Imaging device, distance measuring device, inspection device, imaging method, distance measuring method, inspection method, and program

Abstract

An imaging method calculates a complementary color with respect to a color of an object (G), generates a complementary color random dot pattern, projects the complementary color random dot pattern onto the object (G), and specifies an image region of the object (G) using the random dot pattern obtained by imaging the object (G).

Description

Technical Field

[0001] The present disclosure relates to an imaging device, an imaging method, a distance measuring device and a distance measuring method for measuring the distance to a measurement object (hereinafter simply referred to as "object"), an inspection device and an inspection method for inspecting a defect of the object, and a program. Background Art

[0002] There is known a distance measuring device that projects pattern light onto an object and measures the distance to the object.

[0003] For example, Patent Document 1 discloses a distance measuring device that projects pattern light with randomized brightness levels onto an object. Specifically, Patent Document 1 discloses a distance measuring device including a pattern illumination device having a diffractive optical element, a compound eye imaging device, and a control unit. The diffractive optical element emits pattern light onto the object, and the pattern light randomly arranges pixels having a brightness distribution with three or more levels. The control unit calculates parallax to calculate the distance to the measurement object.

[0004] Patent Document 1: Japanese Patent Application Laid-Open No. 2013-190394 Summary of the Invention

[0005] -Technical Problem to be Solved by the Invention-

[0006] For example, in uniform light, when a random dot pattern of white light is projected onto an object, sometimes pixels in which the signal is cut off at the saturation value (upper limit) or the black level (lower limit) of the image sensor are generated, resulting in damage to the dot pattern and a decrease in the parallax detection rate. In the random dot pattern of white light, pixels having a brightness distribution with three or more levels are randomly arranged.

[0007] The technology of the present disclosure has been completed to solve the above technical problems, and an object thereof is to provide a distance measuring device that prevents the signal from being cut off at the upper limit or the lower limit of the image sensor, thereby suppressing a decrease in the parallax detection rate.

[0008] A Technical Solution for Solving the Technical Problem 1

[0009] The distance measuring device according to the main aspect of the present invention includes a first imaging unit, a second imaging unit, a projection unit, and a measurement unit. The first imaging unit and the second imaging unit are arranged so that their fields of view overlap with each other. The projection unit projects a random dot pattern light of the complementary color of the object onto the overlapping range of the fields of view. The measurement unit performs a stereo corresponding point search process on the first images respectively acquired by the first imaging unit and the second imaging unit to measure the distance to the surface of the object onto which the random dot pattern light is projected.

[0010] Effect of an Invention

[0011] According to the present disclosure, by projecting a random dot pattern of complementary color light and taking a photograph, it is possible to stably detect the dot pattern regardless of the spectral reflectance of the object, and it is possible to suppress a decrease in the parallax detection rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a block diagram schematically showing a situation where pattern light is projected onto an object and photographed;

[0013] Figure 2 is a block diagram showing a structural example of a distance measuring device and its surroundings;

[0014] Figure 3 is a flowchart showing an operation example of the distance measuring device;

[0015] Figure 4 is a graph showing simulation results of a structural example of a first embodiment and a comparative example;

[0016] Figure 5 is a graph showing simulation results of a structural example of a first embodiment and a comparative example;

[0017] Figure 6 is a conceptual diagram showing the operation and function of the distance measuring device of the first embodiment;

[0018] Figure 7 is a diagram for explaining other embodiments;

[0019] Figure 8 is a diagram for explaining other embodiments;

[0020] Figure 9 is a diagram showing an example of an integrated dot pattern;

[0021] Figure 10 is showing Figure 1 a block diagram of a modified example of;

[0022] Figure 11 is a block diagram showing a structural example of a photographing device according to a second embodiment;

[0023] Figure 12 is a flowchart showing an example (one) of a photographing method;

[0024] Figure 13 is a flowchart showing another example (two) of a photographing method;

[0025] Figure 14 is a flowchart showing another example (three) of a photographing method;

[0026] Figure 15 is a block diagram showing a structural example of the inspection device according to the third embodiment;

[0027] Figure 16 is a flowchart showing an example (one) of the inspection method;

[0028] Figure 17 is a flowchart showing another example (two) of the inspection method;

[0029] Figure 18 is a flowchart showing another example (three) of the inspection method;

[0030] Figure 19 is a block diagram showing a structural example of the distance measurement device according to the fourth embodiment;

[0031] Figure 20 is a flowchart showing an example (one) of the distance measurement method;

[0032] Figure 21 is a flowchart showing another example (two) of the distance measurement method;

[0033] Figure 22 is a flowchart showing another example (three) of the distance measurement method;

[0034] Figure 23 is a block diagram showing another structural example of the distance measurement device;

[0035] Figure 24 is a flowchart showing a modified example of the distance measurement method;

[0036] Figure 25 is showing Figure 24 an example of the specific processing of step S24 in Detailed Embodiments

[0037] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0038] It should be noted that the following description of the embodiments is only an example in nature and has no intention of limiting the invention, application object, or use of the present application. That is, the numerical values, shapes, constituent elements, arrangement positions of the constituent elements, connection methods, etc. shown in the following embodiments are an example and are not used to limit the present disclosure. Therefore, the constituent elements in the following embodiments that are not described in the independent claims representing the most general concept of the present disclosure are described as optional constituent elements.

[0039] The main body of the device, system, or method in the present disclosure includes a computer. The computer executes a program to implement the main functions of the device, system, or method in the present disclosure. The computer includes a processor that operates according to the program as the main hardware structure. The processor can implement functions as long as it can execute the program, regardless of its type. The processor is composed of one or more electronic circuits including semiconductor integrated circuits (ICs) or large scale integration (LSIs). Here, although they are called ICs or LSIs, the names may change according to the degree of integration, and they may also be devices called system LSIs, very large scale integration (VLSIs), or ultra large scale integration (ULSIs). A field-programmable gate array (FPGA) or a reconfigurable logic device can also be used for the same purpose, where the FPGA or reconfigurable logic device is programmed after the LSI is manufactured, and the reconfigurable logic device can reconstruct the bonding relationship inside the LSI or set the circuit partitions inside the LSI. The multiple electronic circuits can be integrated on one chip or provided on multiple chips. The multiple chips can be concentrated in one device or included in multiple devices. The program is recorded in a non-transitory recording medium such as a read-only memory (ROM), an optical disc, or a hard disk drive that can be read by the computer. The program can be stored in the recording medium in advance or supplied to the recording medium via a wide area communication network including the Internet and the like.

[0040] <First Embodiment>

[0041] This embodiment relates to a distance measuring device that projects pattern light from a projection unit onto an object and measures the distance to the object, for example, for a stereo camera. More specifically, the distance measuring device according to this embodiment, for example, divides the projection range into a plurality of divided regions, independently adjusts the light intensities of two or more wavelength bands for each divided region, and projects them within the projection range to perform a distance measuring operation.

[0042] As Figure 1 shown, the distance measuring device 1 includes a projection unit 2 and a plurality of imaging units 3. In addition, as Figure 2 shown, the distance measuring device 1 includes a measurement unit 5, a pattern generation unit 6, a control unit 7, and a light source that constitutes the projection unit 2. The functions of the measurement unit 5, the pattern generation unit 6, and the control unit 7 are implemented by the above-mentioned processor, for example.

[0043] One projection unit

[0044] The projection unit 2 has a light source (refer to Figure 2 ), and under the control of the control unit 7, projects uniform light onto a specified projection range 2a. Here, the uniform light only needs to be light with uniform luminous intensity within the projection range (for example, there is no brightness difference such as dot patterns when the light is projected onto a plane perpendicular to the optical axis), and for example, it is white light. The specified projection range 2a can be arbitrarily set as a range including the object G.

[0045] Moreover, the projection unit 2 is configured to be able to project multi-wavelength pattern light onto the specified projection range 2a under the control of the control unit 7.

[0046] The light source used by the projection unit 2 is not particularly limited, and a conventionally known light source (for example, a projector using the DLP (digital light processing) method, the liquid crystal method, etc.) can be used. As the light source of the projector, for example, there are lasers, LEDs (light emitting diodes), mercury lamps, etc.

[0047] Here, if the spectral reflectance of the object G is known, the projection of uniform light can also be skipped. The details will be described in the operation of the parallax information generation device described later.

[0048] -Imaging unit-

[0049] A plurality of imaging units 3 are arranged such that their fields of view 3a overlap with each other. As Figure 1 shown, the above-mentioned specified projection range 2a is set to include the area where the fields of view 3a of each other overlap.

[0050] The imaging units 3 respectively image the range of the field of view 3a under the control of the control unit 7, and output the imaging result to the measurement unit 5. As the imaging unit 3, for example, an RGB camera can be used. Here, the imaging element used in the imaging unit 3 is not particularly limited, and a conventionally known element (for example, an image sensor (photodiode)) can be used.

[0051] In this example, an example in which two imaging units 3 are arranged side by side is shown. That is to say, an example of a binocular stereo camera is shown. It should be noted that the number of imaging units 3 is not limited to two, and can also be three or more.

[0052] -A measurement unit-

[0053] The measurement unit 5 divides the region R in the image (hereinafter, also referred to as the "second image") in which the object G is photographed into a plurality of divided regions RD. This image is an image obtained by projecting uniform light from the projection unit 2 onto the object G and photographing it with one of the photographing units 3 in the photographing unit 3. Then, the complementary color of each divided region RD is calculated and output to the pattern generation unit 6.

[0054] The "complementary color" here is not the complementary color expressed based on human visibility, but the complementary color in the spectral sensitivities of the respective pixels (for example, R pixels, G pixels, B pixels) of the photodiodes (not shown) installed in the photographing unit 3.

[0055] The "complementary color of each divided region RD" here is obtained by performing statistical processing such as averaging, median, and mode on the colors of the pixels included in the divided region RD to obtain a reference color, and calculating the complementary color with respect to this reference color.

[0056] The measurement unit 5 acquires images (hereinafter, also referred to as "first images") respectively acquired by the plurality of photographing units 3 in a state where random dot pattern light (also simply referred to as "random dot pattern") of the complementary color generated by the pattern generation unit 6 is projected. Then, a stereo corresponding point search process is performed on this first image, and the distance D0 to the surface of the object on which the random dot pattern light is projected is measured. Here, the stereo corresponding point search process can be performed, for example, by calculating the matching cost of two images using SSD (sum of squared differences), SAD (sum of absolute differences), NCC (normalized cross-correlation), etc. for each block region and selecting the disparity with the minimum cost.

[0057] - Pattern generation unit -

[0058] Based on the calculation result of the measurement unit 5, the pattern generation unit 6 generates a random dot pattern of light (hereinafter referred to as "complementary color light") corresponding to the calculated complementary color for each projection region corresponding to the divided region RD. The correspondence relationship between the divided region RD and the projection region can be obtained using the positional relationship and optical parameters of the photographing unit 3 and the projection unit 2 in space, for example, by using a homography transformation. In Figure 3 the middle section of the left column, an example of the simulation result of the random dot pattern generated by the pattern generation unit 6 is shown. Regarding Figure 3 , it will be specifically described later.

[0059] - Control unit one

[0060] The control unit 7 has the function of controlling the operation of the distance measuring device 1, and is composed of, for example, a CPU (central processing unit, processor) and a memory that stores programs for the CPU to operate, processing results in the CPU, etc. The control unit 7 includes programs for controlling the operations of the projection unit 2, the imaging unit 3, the measurement unit 5, and the pattern generation unit 6. It should be noted that the program is not limited to being pre-recorded in the memory, and the program can also be provided through electrical lines such as the Internet, or provided from a tangible and non-transitory recording medium such as a memory card. It may also have a setting reception unit such as a keyboard, and can receive settings from the operator to change the control method. The functions of the measurement unit 5 and the pattern generation unit 6 may also be incorporated into the control unit 7. That is to say, the functions of the measurement unit 5, the pattern generation unit 6, and the control unit 7 can also be implemented by a shared processor.

[0061] In the control unit 7, the following controls are executed: (1) controlling the projection unit 2 to project uniform light and the imaging unit 3 to perform an imaging operation; (2) controlling the projection unit 2 to project a random dot pattern of complementary color light and the imaging unit 3 to perform an imaging operation.

[0062] It should be noted that part or all of the functions of the control unit 7 can also be implemented by a hardware circuit, etc. The functions of the control unit 7, the measurement unit 5, and the pattern generation unit 6 can also be implemented by a shared single or multiple microcomputers, CPUs, etc.

[0063] The specific processing content in the control unit 7 will be described in more detail in the following operations of the parallax information generation device.

[0064] 〔Operations of the Parallax Information Generation Device〕

[0065] Next, Figure 3 the operations of the parallax information generation device will be described with reference to the flowchart.

[0066] In Figure 3 step S1, the control unit 7 gives an instruction to the projection unit 2 to project uniform light. In addition, in parallel with step S1, an imaging instruction is given to the imaging unit 3 (step S2).

[0067] As a result, the projection unit 2 projects uniform light within the projection range 2a, and each imaging unit 3 performs imaging of the visual field 3a that includes the projection range 2a where the uniform light is projected. The second image captured by the imaging unit 3 is sent to the measurement unit 5.

[0068] Figure 4 and Figure 5 show the simulation results. Figure 4The upper part of [Figure 0] shows an example of the object G photographed under uniform light. Under uniform light (white light), the upper half region (hereinafter referred to as the upper side region RU) is yellowish green, and the lower half region (hereinafter referred to as the lower side region RL) is blue.

[0069] In the next step S3, the measurement unit 5 divides the second image received from the photographing unit 3 into a plurality of divided regions, and calculates the corresponding complementary color for each divided region RD. In Figure 4 the example of [Figure 5], the measurement unit 5 divides the second image into equal "4 rows × 4 columns = 16" divided regions RD, and calculates the corresponding complementary color for each divided region. That is, in Figure 4 the example of [Figure 7], the upper side region RU is divided into 2 rows × 4 columns = 8 regions, and the lower side region RL is divided into 2 rows × 4 columns = 8 regions, and the complementary color is calculated for each divided region RD. In the embodiment (refer to Figure 4 the middle part of the left column), mutually different complementary colors are obtained as the calculation results between the upper side region RU and the lower side region RL.

[0070] In the next step S4, the pattern generation unit 6 generates a random dot pattern corresponding to the complementary color calculated by the measurement unit 5 for each divided region RD, and sets the light emission intensity of the random dot pattern.

[0071] It should be noted that the method for determining the light emission intensity of the multi-wavelength pattern light constituting the random dot pattern is not particularly limited, and can be set, for example, in the following manner.

[0072] First, the pattern generation unit 6 uses the following formulas (1) and (2) to find the maximum value and the minimum value for each pixel or each region of several pixels × several pixels (hereinafter also collectively referred to as "each pixel").

[0073] Maximum value = ([Illuminance of uniform light] × [Pixel value at saturation of the photodiode for the photographing unit 3]) / [Pixel value obtained by photographing with uniform light]... (1)

[0074] Minimum value = ([Illuminance of uniform light] × [Pixel value of the black level of the photodiode for the photographing unit 3]) / [Pixel value obtained by photographing with uniform light]... (2)

[0075] Next, the pattern generation unit 6 randomly changes the illuminance of the multi-wavelength random dots for each pixel within the range obtained by the above formulas (1) and (2).

[0076] Figure 4 The left column of [Figure 31] shows an example of the embodiment according to the present embodiment, and the middle part shows an example of the random dot pattern of the complementary color light. Figure 4 The right column of [Figure 33] shows a comparative example, and the middle part shows an example of the random dot pattern of white light. ComparisonFigure 4 From the left and right columns, it can be seen that in the left column, since the complementary color is reflected in the random dot pattern light, the overall hue is not the same between the upper region and the lower region.

[0077] It should be noted that when the spectral reflectance of the object G is known and the random dot pattern light of the complementary color of the object can be prepared in advance, steps S1 to S4 can be omitted.

[0078] In the next step S5, the control unit 7 gives an instruction to the projection unit 2 to project the random dot pattern light generated by the pattern generation unit 6. In addition, a shooting instruction is given to the shooting unit 3 in parallel with step S5 (step S6). In Figure 4 The lower part of shows an example of the image after performing demosaicing processing on the captured images of the embodiment and the comparative example.

[0079] Thus, the projection unit 2 projects the random dot pattern light within the projection range 2a, and each shooting unit 3 performs shooting of the object G on which the random dot pattern light is projected, and outputs the captured first image to the measurement unit 5. That is, the first images output from the plurality of shooting units 3 become two or more mutually different images. For example, when there are two shooting units 3, two first images are input to the measurement unit 5.

[0080] In the next step S7, the measurement unit 5 measures the distance to the object G based on the first image captured by the shooting unit 3 in step S6.

[0081] In Figure 5 The upper part of shows an example of the histogram of the first image for each of the upper region RU and the lower region RL of the embodiment (projecting a random dot pattern) and the comparative example (projecting white uniform light). For the comparative example, in the histogram, since pixels whose signals are cut off at the upper and lower limits are generated, the dot pattern is damaged, as Figure 5 shown in the lower right column of, pixels for which parallax cannot be detected are generated (refer to the symbol ND in Figure 5 ). In contrast, in the embodiment, it is confirmed that the histogram does not reach the upper and lower limits, thereby suppressing the reduction in the parallax detection rate.

[0082] As described above, in the present embodiment, (1) the projection unit 2 projects uniform light into the projection range 2a including the object G; (2) the imaging unit 3 captures a second image of the field of view 3a, and the field of view 3a includes the projection range 2a where the uniform light is projected; (3) in the measurement unit 5, the second image is divided into a plurality of divided regions RD, and the complementary color corresponding to each divided region RD is calculated; (4) in the pattern generation unit 6, based on the calculation result of the measurement unit 5, a random dot pattern of the complementary color is generated for each divided region RD; (5) the projection unit 2 projects the random dot pattern light generated by the measurement unit 5 into the projection range 2a; (6) the imaging unit 3 captures a first image of the projection range 2a where the random dot pattern light is projected; (7) in the measurement unit 5, the distance to the object G is measured based on the first image.

[0083] Thus, for example, it is possible to prevent pixels in which signals are cut off at the upper and lower limits from being generated in the photodiode when the photodiode is used as the image sensor of the imaging unit 3. In this way, a decrease in the parallax detection rate can be suppressed.

[0084] Use Figure 6 A more specific description of the functions and effects of the present embodiment is given. In Figure 6 , the left column shows a conceptual diagram of an example (the present embodiment), and the right column shows a conceptual diagram of a comparative example. In this example, when uniform light is projected, both the example and the comparative example can obtain the spectral distribution shown in the upper paragraph.

[0085] In the comparative example (right column), the "random dot pattern of uniform light (white light)" shown in the middle paragraph is applied. As a result, as shown in the lower right column, in the photodiode of the imaging unit 3, the situation where the signal exceeds the upper limit (saturation value) or is lower than the lower limit (black level) of the photodiode occurs. As a result, the dot pattern is damaged and the parallax detection rate decreases.

[0086] In contrast, in the present embodiment, the "random dot pattern light of complementary color" shown in the middle left column is applied, that is, a random dot pattern having the characteristic that the strength of the spectral distribution is opposite to the color of the object is applied. Thus, as shown in the lower left column, in the photodiode of the imaging unit 3, the situation where the signal exceeds the upper limit (saturation value) or is lower than the lower limit (black level) of the photodiode is suppressed. As a result, the dot pattern is not damaged and a decrease in the parallax detection rate can be suppressed.

[0087] 〔Modification example〕

[0088] Figure 10 The structure related to the modification example of the distance measuring device 1 is shown. In Figure 10 the structure, there is one imaging device 3, and Figure 1The structures of the two photographing devices 3 are different. Also, in this modification example, based on the random dot pattern projected by the projection unit 2 and the image photographed by the photographing unit 3, parallax information is generated according to the Figure 3 procedure described, that is, stereo matching is performed. More specifically, in step S7 of the above-described embodiment Figure 3 , when there are two photographing units 3, the distance to the object G is measured based on the two first images. In contrast, in this modification example, the distance to the object G is measured based on the first image photographed by the photographing unit 3 and the random dot pattern already projected by the projection unit 2. The structures and operations other than this are the same as those of the first embodiment, and the same effects can be obtained.

[0089] <Second Embodiment>

[0090] Figure 11 FIG. is a block diagram showing an example of the structure of the photographing device 10 according to the present embodiment. In the following description, for the structures common to the first embodiment, the description may sometimes be simplified or omitted.

[0091] As Figure 11 shown, the photographing device 10 includes a storage unit 11, a projection unit 12, a photographing unit 13, a complementary color calculation unit 14, a measurement unit 15, and a pattern generation unit 16. For example, the arrangements of the projection unit 12 and the photographing unit 13 are the same as those of the projection unit 2 and the photographing unit 3 in the above Figure 10 , and they have the same structure. The functions of the complementary color calculation unit 14, the measurement unit 15, and the pattern generation unit 16 are realized by the above-described processor, for example.

[0092] - Storage Unit

[0093] The storage unit 11 stores color information of the object. In the first embodiment, the color information was obtained by photographing the object in a state where uniform light was projected. However, as in the present embodiment, by storing the color information of the object G in the storage unit 11, the number of photographed frames can be reduced.

[0094] - Complementary Color Calculation Unit

[0095] The complementary color calculation unit 14 calculates the complementary color to the color of the object G using the color information of the object G stored in the storage unit 11.

[0096] It should be noted that the complementary color calculation unit 14 may also project uniform light onto the object using the projection unit 12, photograph the object onto which the uniform light is projected using the photographing unit 13, thereby obtain the color information of the object, and calculate the complementary color to the color of the object using this color information. When the uniform light photographed by the photographing unit 13 is used for calculating the complementary color, the storage unit 11 may not be provided.

[0097] Pattern generation unit 1

[0098] The pattern generation unit 16 generates a random dot pattern of the complementary color calculated by the complementary color calculation unit 14 and outputs it to the projection unit 12. The projection unit 12 projects the random dot pattern or uniform light input from the pattern generation unit 16 onto the object G. Then, the object G onto which the light (random dot pattern or uniform light) of the projection unit 12 is projected is photographed by the photographing unit 13. The image photographed by the photographing unit 13 is output to the measurement unit 15.

[0099] -Measurement unit-

[0100] The measurement unit 15 determines the area of the object G based on the image input from the photographing unit 13. For example, when the distance between the object G and the background is relatively large, the random dot pattern is not projected onto the background but only onto the object G, so the area where the object G is located can be easily determined.

[0101] 〔Shooting method〕

[0102] Next, refer to Figures 12 - 14 to explain the shooting method using the shooting device 10. For example, the shooting device 10 includes one or more processors and has a program for executing the following shooting method.

[0103] It should be noted that in Figures 12 - 14 , the same symbols are used to label the common operations. In the following description, the differences from the above "operation of the parallax information generation device" and the differences between the respective drawings will be mainly described, and repeated descriptions may sometimes be omitted.

[0104] (One)

[0105] Figure 12 An example of obtaining the color information of the object from the shooting result obtained by projecting uniform light in the shooting method is shown.

[0106] Specifically, in Figure 12 , uniform light is projected from the projection unit 12 onto the object G (step S11), and the color information of the object G is obtained by photographing the object G onto which the uniform light is projected by the photographing unit 13 (step S12). Steps S11 and S12 correspond to Figure 3 steps S1 and S2 of

[0107] The complementary color calculation unit 14 divides the image photographed in step S12 into a plurality of divided regions (step S13), obtains the color information of the object G for each divided region RD (step S14), and calculates the corresponding complementary color for each divided region RD (step S15). The processing of steps S13 to S15 corresponds to Figure 3In step S3, the same calculation method as that of the first embodiment can be used.

[0108] In step S16, the pattern generation unit 16 generates a random dot pattern of the complementary color of each divided area RD calculated by the complementary color calculation unit 14, and outputs it to the projection unit 12. Step S16 corresponds to Figure 3 Step S4, and the specific processing content is the same as that of step S4 above.

[0109] In step S17, the projection unit 12 projects the random dot pattern received from the pattern generation unit 16 onto the object G. Step S17 corresponds to Figure 3 Step S5, and the specific processing content is the same.

[0110] In step S18, the photographing unit 13 photographs the object G onto which the random dot pattern is projected. Step S17 corresponds to Figure 3 Step S5, and the specific processing content is the same.

[0111] In step S19, the measurement unit 15 determines the area of the object G based on the image output by the photographing unit 13. That is, it determines the area where the object G is located within the image. As described above, for example, when the distance between the object G and the background is far, the random dot pattern is not projected onto the background but only onto the object, so the area of the object can be easily determined.

[0112] (Second)

[0113] Figure 13 The difference between the photographing method of Figure 12 and the photographing method of Figure 13 is that there is no step S13, that is, the step of dividing into divided areas RD. That is, in

[0114] the image photographed in step S12 is not divided into divided areas RD, but subsequent processing is performed on the entire image.

[0115] (Third)

[0116] Figure 14 The difference between the photographing method of Figure 13 and the photographing method of

[0117] is that the color information of the object stored in the storage unit 11 is used instead of obtaining the color information of the object from the photographing result of the projected uniform light. Figure 13In steps S11 and S12, in step S14, the color compensation calculation unit 14 obtains color information from the storage unit 11.

[0118] Subsequent processing (steps S15 to S19) is the same as that in the above (i) or (ii), and detailed description thereof is omitted here.

[0119] 〔Variant Example〕

[0120] In the above-described photographing methods (i) to (iii), the projection unit 12 may also project the integrated random dot pattern P1 as a random dot pattern onto a part or the entire divided area RD, and the integrated random dot pattern P1 includes an area for projecting uniform light in a part of the pattern. Figure 9 An example of the integrated random dot pattern P1 is shown.

[0121] In Figure 9 an example of the integrated random dot pattern P1 in an area (area RD31) in the divided area RD3 within the projection range R3 is shown, where (1) an area B1 for projecting uniform light is provided at the upper left corner and the lower right corner of the drawing surface of the random dot pattern light, and (2) an area B2 for sparsely projecting uniform light is provided in the random dot pattern light.

[0122] By adopting such a structure, without re-projecting uniform light, it is possible to update the color compensation information of the divided area RD3 based on the information of pixels with uniform light projected such as area B1 or area B2.

[0123] <Third Embodiment>

[0124] Figure 15 It is a block diagram showing an example of the structure of the inspection device 20 according to the present embodiment. In the following description, the differences from the second embodiment will be mainly described, and for the common structures, the description may be simplified or omitted.

[0125] As Figure 15 shown, the inspection device 20 includes a storage unit 21, a projection unit 22, a photographing unit 23, a color compensation calculation unit 24, a measurement unit 25, and a pattern generation unit 26. For example, the arrangements of the storage unit 21, the projection unit 22, the photographing unit 23, the color compensation calculation unit 24, and the pattern generation unit 26 are the same as those of the storage unit 11, the projection unit 12, the photographing unit 13, the color compensation calculation unit 14, and the pattern generation unit 16 in the above Figure 11 and have the same structure. In the present embodiment, the functions of the color compensation calculation unit 24, the measurement unit 25, and the pattern generation unit 26 are realized by the above-described processor, for example.

[0126] -Measurement Unit-

[0127] The measuring unit 25 detects a defect of the object G based on the image captured by the imaging unit 23. For example, the measuring unit 25 detects a defect when the degree of deformation of the pattern exceeds a predetermined threshold. For example, when the object G is strained or a scratch is generated on the surface of the object G, the pattern deformation occurs, and the measuring unit 25 can detect such a defect.

[0128] (Inspection method)

[0129] The following reference Figures 16 - 18 The following describes an inspection method using the inspection device 20. For example, the inspection device 20 includes one or more processors and has a program for executing the following inspection method.

[0130] It should be noted that in Figures 16 - 18 In, with Figures 12 - 14 The same reference numerals are used for common operations and operations common between the drawings. In the following description, the differences from the second embodiment and the differences between the drawings are mainly described, and duplicate descriptions may be omitted.

[0131] (Part 1)

[0132] Figure 16 An example is shown in which color information of the object G is acquired from an image pickup result obtained by projecting uniform light onto the object G and picking up the image in the inspection method.

[0133] exist Figure 16 In the example, steps S11 to S18 correspond to Figure 12 Steps S11 to S18 of the same process are also performed. Figure 16 In the process, step S21 is performed after step S18.

[0134] In step S21, the measuring unit 25 detects a defect of the object G based on the image output by the imaging unit 23. As described above, the measuring unit 25 detects a defect when, for example, the degree of deformation of the pattern in the image output by the imaging unit 23 exceeds a predetermined threshold.

[0135] (Part 2)

[0136] Figure 17 The inspection method and Figure 16 The difference between the inspection method of FIG. 1 and FIG. 2 is that there is no step S13, that is, the step of dividing into divided regions RD. That is, in Figure 17 In the above process, the image captured in step S12 is not divided into the divided regions RD, but the subsequent processing is performed on the entire image.

[0137] It should be noted that regarding the processing of steps S14 to S18 and S21, although there is a difference in whether it is processed according to each divided region RD or the entire image, the same processing method as (one) can be used.

[0138] (Three)

[0139] Figure 18 The inspection method of Figure 17 differs from the inspection method of

[0140] That is to say, in Figure 18 steps S11 and S12 of Figure 17 are omitted, and in step S14, the complementary color calculation unit 24 obtains the color information of the object G from the storage unit 21.

[0141] Subsequent processing (steps S15 to S18, S21) is the same as (one) or (two) above, and detailed description is omitted here.

[0142] 〔Modification example〕

[0143] In the above inspection methods (one) to (three), the projection unit 22 may also project the integrated random dot pattern P1 as a random dot pattern onto a part or the entire divided region RD, and the integrated random dot pattern P1 includes a region for projecting uniform light in a part. As described above, Figure 9 shows an example of the integrated random dot pattern P1.

[0144] <Fourth Embodiment>

[0145] Figure 19 is a block diagram showing an example of the structure of the distance measuring device 30 according to the present embodiment. In the following description, the differences from the second embodiment will be mainly described, and for the common structures, the description may be simplified or omitted.

[0146] As Figure 19 shown, the distance measuring device 30 includes a storage unit 31, a projection unit 32, a photographing unit 33, a complementary color calculation unit 34, a measurement unit 35, and a pattern generation unit 36. In the present embodiment, the arrangements of the storage unit 31, the projection unit 32, the photographing unit 33, the complementary color calculation unit 34, and the pattern generation unit 36 are the same as those in the above Figure 10 and Figure 11The projection unit 11, the projection unit 12, the shooting unit 13, the complementary color calculation unit 14 and the pattern generation unit 16 are the same and have the same structure. Here, the projection unit 32 is, for example, a structured light source. In this embodiment, the functions of the complementary color calculation unit 34, the measurement unit 35 and the pattern generation unit 36 ​​are implemented by, for example, the above-mentioned processor.

[0147] -Measurement Department-

[0148] The measuring unit 35 calculates the distance to the object G based on the image captured by the imaging unit 33. Specifically, the measuring unit 35 calculates the parallax between the pattern on the image output by the imaging unit 33 and the projection pattern projected by the projection unit 32, and calculates the distance to the object G based on the calculated parallax.

[0149] 〔Distance measurement method〕

[0150] The following reference Figures 20 - 22 The distance measuring method (distance measuring method) using the distance measuring device 30 will be described. For example, the distance measuring device 30 includes one or more processors and has a program for executing the following distance measuring method.

[0151] It should be noted that in Figures 20 - 22 In, with Figures 12 - 14 The same reference numerals are used for common operations and operations common between the drawings. In the following description, the differences from the second embodiment and the differences between the drawings are mainly described, and duplicate descriptions may be omitted.

[0152] (Part 1)

[0153] Figure 20 An example is shown in which color information of the object G is acquired from an image pickup result obtained by projecting uniform light in the distance measurement method.

[0154] exist Figure 20 In the example, steps S11 to S18 correspond to Figure 12 Steps S11 to S18 of the same process are also performed. Figure 20 In the process, step S22 is performed after step S18.

[0155] In step S22, the measuring unit 35 calculates the distance to the object G based on the image output by the imaging unit 33. As described above, the measuring unit 35 calculates the parallax between the pattern on the image output by the imaging unit 33 and the projection pattern projected by the projection unit 32, and calculates the distance to the object G based on the calculated parallax.

[0156] (Part 2)

[0157] Figure 21 The distance measurement method and Figure 20The difference in the ranging method is that there is no step S13, i.e., the step of dividing into divided regions RD. That is, in Figure 21 , the image captured in step S12 is not divided into divided regions RD, and subsequent processing is performed on the entire image.

[0158] It should be noted that regarding the processing of steps S14 to S18 and S22, although there is a difference in whether it is processed for each divided region RD or for the entire image, the processing method is the same as that in (i).

[0159] (iii)

[0160] Figure 22 The inspection method of Figure 21 is different from the inspection method of

[0161] in that the color information of the object stored in the storage unit 31 is used instead of obtaining the color information of the object from the shooting result of the projected uniform light. Figure 22 That is, in Figure 21 , steps S11 and S12 of

[0162] are omitted, and in step S14, the complementary color calculation unit 34 obtains the color information from the storage unit 31.

[0163] 〔Modification Example 1〕

[0164] In the above inspection methods (i) to (iii), the projection unit 22 may also project the integrated random dot pattern P1 as a random dot pattern onto a part or the entire divided region RD, and the integrated random dot pattern P1 includes a region for projecting uniform light in a part of the pattern. As described above, Figure 9 shows an example of the integrated random dot pattern P1.

[0165] 〔Modification Example 2〕

[0166] In the above fourth embodiment, an example in which structured light is used as the projection unit 32 to irradiate the object G with light and one shooting unit 33 shoots the object G onto which light (random dot pattern or uniform light) is projected is described, but it is not limited thereto. Figure 10 For example, as shown in the first embodiment (for example

[0167] ), a structure using two shooting units 33 may also be adopted. The structure of this modification example is used for a stereo camera, for example. Figure 1 )

[0168] Figure 23It is a block diagram showing an example of the structure of the distance measuring device 30 according to this modified example. In the following description, the differences from the fourth embodiment will be mainly described, and for the common structures, the description may be simplified or omitted.

[0169] As Figure 23 shown, the distance measuring device 30 according to this modified example is different from Figure 19 in that it includes two photographing units 33. In addition, the distance calculation method of the measurement unit 35 is different from that of the fourth embodiment.

[0170] - Measurement unit -

[0171] The measurement unit 35 calculates the distance to the object G based on the images captured by the photographing unit 33. Specifically, the measurement unit 35 uses the patterns reflected in the two images, calculates the parallax between the two images, and calculates the distance to the object G based on the calculated parallax. The two images are the images captured by the two photographing units 33 arranged at positions with different viewpoints.

[0172] The distance measurement method according to this modified example is the same as steps S11 to S18 of (i) to (iii) of the fourth embodiment, while step S22 is different.

[0173] Specifically, in this modified example, in step S22, the measurement unit 35 calculates the distance to the object G based on the images captured by the two photographing units 33. As described above, the measurement unit 35 uses the patterns reflected in the two images, calculates the parallax between the two images, and calculates the distance to the object G based on the calculated parallax. The two images are the images captured by the two photographing units 33 arranged at positions with different viewpoints.

[0174] 〔Modified Example 3〕

[0175] Figure 24 and Figure 25 are flowcharts showing an example of the distance measurement method according to this modified example. Figure 24 is equivalent to Figure 20 's flowchart. For example, the distance measuring device 30 includes one or more processors and has a program for executing the following distance measurement method.

[0176] It should be noted that in Figure 24 , the same symbols are used for the actions common to Figure 20 . In the following description, the differences from the fourth embodiment will be mainly described, and the repeated descriptions may be omitted sometimes.

[0177] Figure 24 Steps S11 to S13 of Figure 20Steps S11 to S13, and the specific processing content is the same. In Figure 24 After step S13, step S24 is executed.

[0178] In step S24, the processor uses the image captured by projecting uniform light onto the object G to obtain a histogram for each divided area RD. Then, in the divided area RD where the width of the level exceeding the specified frequency threshold in the histogram is equal to or greater than the specified threshold, the distance to the object G is measured using the image captured by projecting uniform light onto the object G. On the other hand, in the divided area RD where the width of the level exceeding the specified frequency threshold in the histogram is less than the specified threshold, the distance to the object G is measured using the image captured in the state where a random dot pattern is projected.

[0179] Figure 25 is a flowchart showing Figure 24 the detailed processing flow of step S24 in

[0180] First, the processor focuses on the first divided area (step S41), calculates the histogram of the focused divided area RD (step S42). The processor also calculates the width of the level exceeding the specified frequency threshold (step S43).

[0181] In the next step S44, the processor determines whether the width of the level calculated in step S43 is equal to or greater than the specified threshold. Then, if the width of the level is equal to or greater than the specified threshold ( "Yes" in S44), uniform light is projected onto the focused divided area (step S45), and the process proceeds to step S47. On the other hand, if the width of the level is less than the specified threshold ( "No" in S44), a random dot pattern is projected onto the focused divided area (step S46), and the processing of steps S461 to S463 is executed.

[0182] Specifically, the processor obtains the color information of the object G in the focused divided area RD (step S461), calculates the complementary color corresponding to the divided area RD (step S462). And the processor generates a random dot pattern of the complementary color of the focused divided area RD according to the complementary color calculated in step S462, and the process proceeds to step S47.

[0183] In step S47, it is determined whether the projection pattern has been determined for all divided areas RD. When there is a divided area RD for which the projection pattern has not been determined ( "No" in S47), the next divided area RD to be focused on is selected, and the processing of steps S42 to S48 is repeated until the projection pattern has been determined for all divided areas RD.

[0184] Then, when the projection pattern has been determined for all the divided regions RD (being "no" in S47), Figure 24 the process of step S24 in

[0185] ends, and the flow proceeds to the next step S25. In step S25, the processor acquires the color information of the region where the complementary color random dot pattern to be projected determined in step S24, that is, the projection object region of the random dot pattern. Then, the processor calculates the corresponding complementary color for each divided region RD which is the projection object of the random dot pattern (step S26), generates the random dot pattern (step S27), and outputs it to the projection unit 12. The processes of steps S25 to S26 correspond to Figure 3 step S3 of Figure 3 , and the process of step S27 corresponds to

[0186] step S3 of Figure 24 , and steps S18, S19, and S22 of Figure 20 correspond to steps S18, S19, and S22 of

[0187] <Other Embodiments>

[0188] Note that the present disclosure is not limited to the description of the above embodiments, and various modifications can be made without departing from the gist thereof. In addition, the structures of the respective embodiments and / or modification examples can be combined as a new embodiment.

[0189] For example, in addition to the structure in the above first embodiment, the measurement unit 5 may also use the second image captured when uniform light is projected to obtain the histogram for each divided region RD. It may also be like this: in the histogram of the second image, in the divided region RD where the width W of the level exceeding the specified frequency threshold Bt is equal to or greater than the specified threshold Wt, the second image is used to measure the distance to the object G; in the divided region RD where the width W of the level exceeding the specified frequency threshold is less than the specified threshold Wt, the first image captured when the random dot pattern is projected is used to measure the distance to the object G.

[0190] Figure 7 An example where the width W1 of the level in the divided region RD1 within the projection range R1 including the object G1 is less than the specified threshold Wt is shown. That is, an example where the width of the histogram is narrower than the specified width is shown. For such a divided region RD1, the measurement unit 5 uses the first image captured when the random dot pattern is projected to measure the distance to the object G.

[0191] On the other hand, as Figure 8As shown, in the divided region RD2 within the projection range R2 that includes the object G2, when the width W2 of the grade is equal to or greater than a specified threshold value Wt, that is, for a divided region where the width of the histogram is wider than the specified width, the measurement unit 5 measures the distance to the object G2 using the second image captured when projecting uniform light.

[0192] By adopting such a configuration, it is sometimes possible to shorten the calculation time or reduce the number of projections of the frames, and thus an effect of shortening the processing time can be obtained.

[0193] In the above-described first embodiment, the projection unit 2 projects the random dot pattern generated by the pattern generation unit 6 into the projection range 2a in Figure 3 step S5, but it is not limited thereto. For example, the projection unit 2 may project the integrated random dot pattern P1 onto a part or the entire divided region RD. The integrated random dot pattern P1 is a pattern that includes a region for projecting uniform light in a part of the random dot pattern light generated by the pattern generation unit 6, and is a pattern in which the projection regions of the random dot pattern light and the uniform light are integrated (see Figure 9 ).

[0194] In the above-described first embodiment, an example is described in which the measurement unit 5 divides the second image acquired by the imaging unit 3 into a plurality of divided regions RD and obtains the complementary color corresponding to each divided region RD, but it is not limited thereto. For example, the measurement unit 5 may also obtain the complementary color of the object G using the second image captured by one of the imaging units 3 in the state where uniform light is projected from the projection unit 2 onto the object G, and output it to the pattern generation unit 6. In this case, in Figure 3 step S3, the entire region of the second image is used for the calculation of the complementary color. The other steps are the same as those described in the above embodiment, and the same effect can be obtained. For example, this method can be applied when the spectral reflectance within the imaging range of the object is constant.

[0195] -Industrial Applicability-

[0196] The distance measuring device of the present disclosure can suppress a decrease in the parallax detection rate, and thus can be used for a stereo camera or the like, and is extremely useful.

[0197] -Symbol Explanation-

[0198] 1 Distance measuring device

[0199] 2 Projection unit

[0200] 3 Imaging unit

[0201] 5 Measurement unit

[0202] 6 Pattern generation unit

[0203] 7 control unit.

Claims

1. A shooting method, characterized in that: A complementary color to a color of an object is calculated, a random dot pattern of the complementary color is generated and then projected onto the object, and an image region of the object is determined using the random dot pattern obtained by photographing the object.

2. The shooting method according to claim 1, characterized in that: Uniform light is projected onto the object, color information of the object is acquired by photographing the object onto which the uniform light is projected, and a complementary color to the color of the object is calculated using the color information.

3. A shooting method, characterized in that: Uniform light is projected onto an object, the object on which the uniform light is projected is photographed, the photographed image is divided into a plurality of divided areas, and the photographing method according to claim 1 is performed for each of the divided areas.

4. The shooting method according to claim 3, characterized in that: The random dot pattern is an integrated random dot pattern including an area for uniform light projection in a portion.

5. A program, characterized in that: The program is used to cause a photographing device including one or more processors to execute the photographing method according to any one of claims 1 to 4.

6. A photographing device, characterized in that: The photographing device comprises: a complementary color calculation unit that calculates a complementary color with respect to a color of an object; a pattern generating unit, which generates a random dot pattern of the complementary color; a projection unit that projects the random dot pattern of complementary colors onto the object; an imaging unit configured to image the object; and A measuring unit is configured to specify a region of the object based on the image outputted from the imaging unit.

7. The photographing device according to claim 6, characterized in that: The complementary color calculation unit calculates a complementary color with respect to a color of the object using the second image acquired by the imaging unit in a state where the projection unit projects uniform light.

8. The photographing device according to claim 7, characterized in that: The complementary color calculation unit divides the second image into a plurality of divided areas, and calculates a complementary color with respect to a color of the object for each of the divided areas.

9. The photographing device according to claim 8, characterized in that: The random dot pattern is an integrated random dot pattern including an area for uniform light projection in a portion.

10. An inspection method, characterized in that: A complementary color to a color of an object is calculated, a random dot pattern of the complementary color is generated and then projected onto the object, and a defect of the object is detected using the random dot pattern obtained by photographing the object.

11. The inspection method according to claim 10, characterized in that: Uniform light is projected onto the object, color information of the object is acquired by photographing the object onto which the uniform light is projected, and a complementary color to the color of the object is calculated using the color information.

12. An inspection method, characterized in that: Uniform light is projected onto an object, the object on which the uniform light is projected is photographed, the photographed image is divided into a plurality of divided areas, and the inspection method according to claim 10 is performed for each of the divided areas.

13. The inspection method according to claim 12, characterized in that: The random dot pattern is an integrated random dot pattern including an area for uniform light projection in a portion.

14. A program, characterized in that: The program is used to cause an inspection device including one or more processors to execute the inspection method according to any one of claims 10 to 13.

15. An inspection device, characterized in that: The inspection device comprises: a complementary color calculation unit that calculates a complementary color with respect to a color of an object; a pattern generating unit, which generates a random dot pattern of the complementary color; a projection unit that projects the random dot pattern of complementary colors onto the object; an imaging unit configured to image the object; and A measuring unit detects a malfunction of the object based on the image output by the imaging unit.

16. The inspection device according to claim 15, characterized in that: The complementary color calculation unit calculates a complementary color with respect to a color of the object using the second image acquired by the imaging unit in a state where the projection unit projects uniform light.

17. The inspection device according to claim 16, characterized in that: The complementary color calculation unit divides the second image into a plurality of divided areas, and calculates a complementary color with respect to a color of the object for each of the divided areas.

18. The inspection device according to claim 17, characterized in that: The random dot pattern is an integrated random dot pattern including an area for uniform light projection in a portion.

19. A distance measurement method, characterized in that: A complementary color to a color of an object is calculated, a random dot pattern of the complementary color is generated and then projected onto the object, and a distance to the object is calculated using the random dot pattern obtained by photographing the object.

20. The distance measurement method according to claim 19, characterized in that: Uniform light is projected onto the object, color information of the object is acquired by photographing the object onto which the uniform light is projected, and a complementary color to the color of the object is calculated using the color information.

21. A distance measurement method, characterized in that: Uniform light is projected onto an object, the object on which the uniform light is projected is photographed, the photographed image is divided into a plurality of divided areas, and the distance measurement method according to claim 19 is executed for each of the divided areas.

22. The distance measurement method according to claim 21, characterized in that: The random dot pattern is an integrated random dot pattern including an area for uniform light projection in a portion.

23. The distance measurement method according to claim 21, characterized in that: A histogram is obtained for each of the divided areas using an image acquired in a state where uniform light is projected. In the segmented area where the width of the level exceeding the predetermined frequency threshold in the histogram is equal to or larger than the predetermined threshold, the distance to the object is measured using the image acquired in the state where the uniform light is projected, In the segmented area where the width of the level exceeding the predetermined frequency threshold in the histogram is smaller than the predetermined threshold, the distance to the object is measured using the image acquired in the state where the random dot pattern is projected.

24. A program, characterized in that: The program is used to cause a distance measuring device including one or more processors to execute the distance measuring method according to any one of claims 19 to 23.

25. A distance measuring device, characterized in that: The distance measuring device comprises: a complementary color calculation unit that calculates a complementary color with respect to a color of an object; a pattern generating unit, which generates a random dot pattern of the complementary color; a projection unit that projects the random dot pattern of complementary colors onto the object; an imaging unit configured to image the object; and A measuring unit calculates a distance to the object based on the image output by the imaging unit.

26. The distance measuring device according to claim 25, characterized in that: The complementary color calculation unit calculates a complementary color with respect to a color of the object using the second image acquired by the imaging unit in a state where the projection unit projects uniform light.

27. The distance measuring device according to claim 26, characterized in that: The complementary color calculation unit divides the second image into a plurality of divided areas, and calculates a complementary color with respect to a color of the object for each of the divided areas.

28. The distance measuring device according to claim 27, characterized in that: The random dot pattern is an integrated random dot pattern including an area for uniform light projection in a portion.

29. The distance measuring device according to claim 27, characterized in that: A histogram is obtained using the second image for each of the segmented regions. In the segmented area where the width of the level exceeding the predetermined frequency threshold in the histogram is equal to or larger than a predetermined threshold, the distance to the object is measured using the second image, In the divided area where the width of the level exceeding the predetermined frequency threshold in the histogram is smaller than the predetermined threshold, the distance to the object is measured using the image acquired by the imaging unit in a state where the projection unit projects the random dot pattern.

Images