Image forming system, image forming method, and program

Abstract

To realize visual inspection of an object with an inexpensive and simple configuration and simple image processing.SOLUTION: An image forming system 100 comprises: an illumination projection unit 12 that irradiates a measurement object with light in a two-dimensional illumination pattern; a plurality of detection units 13 that is installed in different places, and detects intensity of reflected light from the measurement object irradiated with the light in the illumination pattern; an illumination control unit (control unit 11) that causes the illumination projection unit to irradiate the measurement object with light while sequentially changing the illumination pattern from among a plurality of illumination patterns; a first image creation unit (control unit 11) that creates two-dimensionally reconstructed first images respectively for the detection units, from the illumination pattern and the intensity of reflected light corresponding to the illumination pattern; and a second image creation unit (control unit 11) that creates a new second image on the basis of the plurality of first images created respectively for the detection units.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image forming system, an image forming method, and a program. [Background technology]

[0002] Conventionally, quality inspection (appearance inspection) of glossy components, such as components that have been film-processed, plated, or painted, has been carried out visually. Visual inspection typically involves constantly changing the way the light hits the product and the viewing direction to find small scratches and defects, which is a time-consuming and labor-intensive task.

[0003] Recently, there have been attempts to replace such visual inspection with automated inspection using equipment. For example, in Patent Document 1, the timing of irradiation of each area of ​​the microscope's ring illumination is shifted, images are taken for each area, and image processing is performed to remove halation and highlight damaged areas. In addition, when using a ring light, in addition to the above, there are also cases where it is installed at a position away from the target object, and cases where a low-angle light is installed close to the target object. It is also possible to use different types of light depending on the characteristics of the image to be obtained, or to use both types in combination. Furthermore, for the inspection of a larger area, an inspection method is known in which, for example, an object is illuminated with time-varying striped illumination, and images are measured in time series, and then flaws are highlighted by image processing.

[0004] Patent Document 2 also introduces an appearance inspection method that removes halation. Patent Document 2 proposes an inspection method in which, in order to remove halation caused by lighting, instead of directly illuminating the object with multiple lights, a semi-cylindrical diffuser plate is used between the object and the lights to create a secondary diffused dome light source, and the object is observed with one camera using the reflected light of the illumination from this dome light source. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2015-40796 [Patent Document 2] Patent No. 6616040 Summary of the Invention [Problem to be solved by the invention]

[0006] However, as in Patent Document 1, when performing image processing based on multiple images taken under multiple lighting conditions to remove halation or highlight flaws, a configuration is usually adopted in which multiple lighting devices are used and one fixed camera is used. This is because when generating one image based on multiple images, if the viewpoints are different, it is necessary to grasp the positional relationship between each camera and the object to be measured and perform appropriate image processing, making image generation (image synthesis) difficult. This has drawbacks such as the shooting time becoming longer in proportion to the number of lighting conditions, the size of the object being restricted by the size of the lighting equipment, and the need for a mechanism to move the object.

[0007] Furthermore, in Patent Document 2, if the target object becomes large, a larger diffusion plate for the diffusion dome must also be prepared, and since holes in the diffusion plate must be provided in the optical path between the target object and the camera, the entire device and lighting section must also become large-scale.

[0008] Therefore, an object of the present invention is to realize visual inspection of an object using an inexpensive and simple configuration and simple image processing. [Means for solving the problem]

[0009] In order to solve the above problems, the image forming system of the present invention comprises: an illumination projection unit that irradiates a measurement object with light of a two-dimensional illumination pattern; A plurality of detection units are installed at different locations and detect the light intensity of the reflected light from the measurement object due to the light of the illumination pattern; an illumination control unit that sequentially switches the illumination pattern from among the plurality of illumination patterns and causes the illumination projection unit to emit light; a first image generating unit configured to generate a first image that is two-dimensionally reconstructed for each of the detection units from the illumination pattern and the light intensity of the reflected light corresponding to the illumination pattern; a second image generating unit configured to generate a new second image based on a plurality of first images generated for each of the detection units; The present invention is characterized by comprising:

[0010] The image forming method of the present invention further comprises the steps of: Sequentially switching between a plurality of two-dimensional illumination patterns, the illumination pattern is irradiated onto the measurement object, and Detecting the light intensity of the light of the illumination pattern reflected from the measurement object at different locations; generating a plurality of first images reconstructed two-dimensionally for each of the different locations where the light intensity is detected, based on the illumination pattern and the light intensity of the reflected light corresponding to the illumination pattern; The light intensities for each identical pixel in the plurality of first images are processed to generate a new second image.

[0011] In addition, the program of the present invention is A computer of an image forming system including an illumination projection unit that irradiates a measurement object with light of a two-dimensional illumination pattern, and a plurality of detection units that are installed at different locations and detect the light intensity of reflected light from the measurement object due to the light of the illumination pattern, an illumination control unit that sequentially switches the illumination pattern from among the plurality of illumination patterns and causes the illumination projection unit to emit light; a first image generating unit that generates a first image that is two-dimensionally reconstructed for each of the detection units from the illumination pattern and the light intensity of the reflected light corresponding to the illumination pattern; a second image generating unit configured to generate a new second image based on the plurality of first images generated for each of the detection units; Function as. Effect of the Invention

[0012] According to the present invention, it is possible to realize visual inspection of an object with an inexpensive and simple configuration and simple image processing. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a schematic diagram illustrating a schematic configuration of an image forming system. [Diagram 2] FIG. 2 is a block diagram showing a functional configuration of the image forming system. [Diagram 3] 4 is a diagram showing an example of an illumination pattern onto an object by an illumination projection unit. [Figure 4] 13 is an example of an arrangement of an illumination projection unit and a plurality of detection units. [Diagram 5] 13 is an example of an arrangement of an illumination projection unit and a plurality of detection units. [Figure 6] 4 is a flowchart of an image forming process. [Figure 7] 13 is a flowchart of a halation removal process. [Figure 8] 13 is a flowchart of a flaw-emphasizing image generating process. [Figure 9] 13 is a flowchart of a scattering anisotropy image generating process. [Figure 10] 13 is a diagram showing an example of the arrangement of an illumination projection unit and a plurality of detection units in the scattering anisotropy image generation process. [Figure 11] The viewpoint position and light intensity in the scattering anisotropy image generation process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, the embodiments of the present invention will be described with reference to the drawings. However, the embodiments described below are subject to various limitations that are technically preferable for implementing the present invention, but the technical scope of the present invention is not limited to the following embodiments and illustrated examples. In particular, the drawings are simplified for the convenience of explanation, and the ratio of the illumination pattern to the object may differ.

[0015] (Configuration of Image Forming System) 1 is a schematic diagram showing a schematic configuration of an image forming system 100 according to this embodiment. The image forming system 100 is used to output information relating to the appearance of a target object that is primarily a non-luminous object.

[0016] Fig. 1 shows an example of observing a measurement object H having a glossy surface such as a metallic electric saw blade. Fig. 2 is a block diagram showing a functional configuration of an image forming system 100 according to this embodiment. Here, the measurement object H, that is, the electric saw blade, is placed on a pedestal 18.

[0017] As shown in FIGS. 1 and 2, the image forming system 100 includes a control unit 11, an illumination projection unit 12, a detection unit 13, a storage unit 14, a display unit 15, an operation unit 16, a communication unit 17, and the like.

[0018] In the image forming system 100, the illumination projection unit 12 changes a two-dimensional illumination pattern P at specific time intervals and projects the light of the corresponding illumination pattern onto the measurement object H. Then, the detection unit 13 detects reflected light from the measurement object H and sends a time-varying light receiving signal S to the control unit 11. Then, the control unit 11 generates a two-dimensional reconstructed image (first image) using SPI (Single-pixel imaging) technology based on the two-dimensional illumination pattern P changed at specific time intervals and the corresponding one-dimensional time-varying light receiving signal S. Since the reconstructed image is generated based on the illumination pattern P and the received light signal S, one reconstructed image is generated for one detection unit 13 .

[0019] The control unit 11 is composed of a CPU (Central Processing Unit), a RAM (Random Access Memory), etc. The control unit 11 is connected to each part (illumination projection unit 12, detection unit 13, storage unit 14, display unit 15, operation unit 16, communication unit 17) of the image forming system 100, and controls the operation of each part. In addition, the control unit 11 executes various processes including an image forming process described later in cooperation with a program stored in the storage unit 14. The control unit 11 controls the illumination pattern P of the illumination projection unit 12 as an illumination control unit. In addition, the control unit 11 functions as a first image generation unit that acquires light intensity (received light signal S) corresponding to the illumination pattern P from the multiple detection units 13, and generates a two-dimensionally reconstructed first image for each detection unit 13 from the illumination pattern P and the light intensity (received light signal S). In addition, the control unit 11 functions as a second image generating unit that generates a new second image based on the multiple first images generated for each detection unit 13. Furthermore, the control unit 11 generates a plurality of illumination patterns P and stores them in the storage unit 14. For example, if the total number of images required for reconstruction cannot be stored in the storage unit 14, only the number of images that can be stored is generated. In addition, an illumination pattern P may be generated according to the pixel resolution of the reconstructed image. In addition, an illumination pattern P may be generated according to the spatial resolution of the reconstructed image. Moreover, an illumination pattern P may be generated according to the size of the imaging range. Alternatively, the illumination pattern P may be generated from a low spatial frequency. In other words, if 4000 illumination patterns P are required, 100 illumination patterns P are generated from the low frequency side, and at the next timing, 100 illumination patterns P are generated from the next lowest frequency side, and finally, an illumination pattern P with a high frequency is generated. Alternatively, the illumination pattern P may be generated from a high spatial frequency. Note that it is easier to quickly obtain a roughly reconstructed image of the measurement object H by irradiating the illumination projection unit 12 with an illumination pattern P with a low spatial frequency. Also, an illumination pattern with a brightness according to the surface condition of the measurement target H may be generated. In other words, in order to effectively use the dynamic range of the detection unit 13, if a black and white binary image is used as the illumination pattern P and each pixel has a range from 0 to 255, then if the surface of the measurement target H is a metal surface or glass surface that is likely to have a large degree of regular reflection, then a slightly darker illumination pattern P, for example, 0 for black and 15 for white, is used, whereas if the surface is likely to have a small degree of regular reflection, then a brighter illumination pattern P, for example, 0 for black and 255 for white, is used.

[0020] The illumination projection unit 12 is, for example, a projector using a digital mirror device (DMD). Note that the illumination projection unit 12 may be various types of projector and is not limited to a projector using a DMD. The illumination projection unit 12 is controlled by the control unit 11, and for example, changes a two-dimensional illumination pattern P shown in Fig. 3 at specific time intervals and projects the pattern onto the measurement object H. The illumination pattern P is created by the control unit 11. The illumination pattern P may be stored in the storage unit 14 in advance. The resolution of the two-dimensional illumination pattern P projected by the illumination projection unit 12 affects the resolution of the reconstructed image, and the number of two-dimensional illumination patterns P projected also affects the resolution of the reconstructed image. Although the frame rate of the DMD is 60 Hz or 120 Hz, the two-dimensional illumination pattern P per unit time can be increased by the following method. Due to its fundamental characteristics, the DMD is capable of projecting image bits in sequence. Generally, one pixel of an image is composed of 24 bits in total, 8 bits for each of RGB, and each bit corresponds to the clock frequency of the DMD as one image and is projected in sequence. However, an image in which one pixel is expressed by 1 bit cannot express gradations, and only so-called black and white images can be projected. For example, by using an image expressed as a so-called black and white image as the lighting pattern, such as Hadamard illumination, it is possible to project an image 24 times the frame rate per unit time.

[0021] The detection unit 13 measures the time variation of the light intensity (received light signal S) of the reflected light from the measurement object H in response to the light of each illumination pattern P that varies at specific time intervals. The detection unit 13 may be, for example, a two-dimensional sensor such as a CCD (Charge-Coupled Device), a one-dimensional sensor such as a line CCD, an array of photodiodes (photodetectors), a single photodiode (photodetector), or the like. In the schematic diagram shown in FIG. 1, there is one detection unit 13, but in this embodiment, multiple detection units 13 are arranged in different locations. For example, as shown in FIG. 4, there is a form in which the detection units 13 are arranged on a circumference (on a three-dimensional spherical surface) surrounding the object to be measured H. As an example shown in FIG. 5, the detectors 13 may be arranged in a straight line. 4 and 5 are simply drawn on the drawings, and even if the seven detection units 13 are arranged in a dispersed manner when viewed from above, analysis processing can be performed sufficiently.

[0022] The memory unit 14 is composed of a non-volatile semiconductor memory, a hard disk, etc., and stores the programs executed by the control unit 11, data necessary for executing the programs, the illumination pattern, the light receiving signal S of the detection unit 13, various generated images, shooting history, etc.

[0023] The display unit 15 displays an operation screen, the operating status of the image forming system 100, etc. on the display in accordance with the display control of the control unit 11.

[0024] The operation unit 16 includes a group of keys used for input operations such as starting processing, as well as a touch panel that is integrated with the display of the display unit 15, and generates operation signals in response to these operations and outputs them to the control unit 11.

[0025] The communication unit 17 includes a communication interface and communicates with external devices on the network.

[0026] (Image formation processing) The flow of the image forming process will be described with reference to Fig. 6. It is assumed that seven detection units 13 are arranged. First, the control unit 11 generates a predetermined number of illumination patterns P and stores them in the storage unit 14 (step S11). Note that the user may select a predetermined number of illumination patterns P to be irradiated according to the measurement target H from the illumination patterns P stored in advance in the storage unit 14, and the control unit 11 may acquire the illumination patterns P from the storage unit 14.

[0027] Next, the control unit 11 causes the illumination projection unit 12 to irradiate the measurement object H with one of the illumination patterns P stored in the storage unit 14 in step S11 (step S12).

[0028] Next, the control unit 11 acquires, from each of the multiple detection units 13, a light reception signal S representing light received from the object to be measured H (step S13).

[0029] Next, the control unit 11 judges whether or not the predetermined number of illumination patterns P stored in the memory unit 14 in step S11 have all been irradiated by the illumination projection unit 12 (step S14). If all illumination patterns P have been irradiated by the illumination projection unit 12 (step S14; YES), the process proceeds to step S15, and if not all illumination patterns P have been irradiated by the illumination projection unit 12 (step S14; NO), the process proceeds to step S12, where the control unit 11 causes the illumination projection unit 12 to irradiate the next illumination pattern P.

[0030] Next, the control unit 11 reconstructs a two-dimensional image for each of the multiple detection units 13 based on the completed illumination pattern P (all illumination patterns P stored in the memory unit 14 in step S11) and the light receiving signals S for each of the multiple detection units 13, and generates multiple reconstructed images (first images) (step S15). Here, the first image is generated at an angle at which the measurement object H is viewed from each detection section 13. For example, in the embodiment of Fig. 4 or Fig. 5, seven reconstructed images are generated.

[0031] Next, the control unit 11 judges whether the illumination projection unit 12 has emitted a sufficient number of illumination patterns P to generate the first image (step S16). If the illumination has been emitted (step S16; YES), seven first images are completed in step S15, and the process proceeds to step S17. If the illumination has not been completed (step S16; NO), the process proceeds to step S11, where a predetermined number of different illumination patterns P are newly created and the process continues. It should be noted that the judgment in step S16 may not be based on the number of irradiated illumination patterns P, but may be based on calculating the quality (e.g., signal-to-noise ratio) of the reconstructed image (first image) generated in step S15 and determining whether the quality satisfies a predetermined standard. In addition, in generating the first image in step S15, a reconstructed image is generated using all the illumination patterns P generated in step S11 immediately before the generation of the first image, and the first image is generated by adding up the reconstructed images generated from the start of processing to the generation of the first image. That is, the first image (reconstructed image) is generated using all the illumination patterns P generated in step S11 from the start of processing to the generation of the first image. Specifically, when the number of illumination patterns P required for reconstruction is 1000, first, 100 illumination patterns (1 to 100) are irradiated as a predetermined number, and a reconstructed image 1 is generated using the 1 to 100 patterns. Next, 100 illumination patterns (101 to 200) are irradiated as a predetermined number, and a reconstructed image 2 is generated using the 101 to 200 patterns. Next, a predetermined number of illumination patterns (201 to 300) are irradiated, and a reconstructed image 3 is generated using the 201 to 300 patterns, and this is repeated. Then, by adding together reconstructed image 1, reconstructed image 2, reconstructed image 3, etc., a first image (reconstructed image) is generated. Also, the signal intensity of the first image (reconstructed image) may be normalized for each of the multiple detection units 13. For example, the background intensity is removed so that the signal intensity of pixels known to be the background of the measurement object H becomes 0. Also, the minimum intensity of the image may be regarded as the background intensity and the background intensity may be removed so that the minimum intensity becomes 0. Also, the signal intensity of each first image may be multiplied by a constant so that the average intensity of the image becomes the same for each first image.

[0032] Next, the control unit 11 performs a second image generation process (step S17) to generate a new second image using the first images created for the number of the detection units 13. For example, the second image generation process is a halation removal process, a flaw enhancement image generation process, or a scattering anisotropy image generation process, which will be described later. A specific flow of each process will be described later. In the second image generation process, alignment of the multiple first images is not necessary because the reconstructed image is generated by calculation based on the illumination pattern P of known two-dimensional coordinates and the light receiving signal S of one-dimensional information.

[0033] Next, the control unit 11 displays the second image on the display unit 15 (step S18).

[0034] (Second image generation process; halation removal process) The flow of the halation removal process will be explained with reference to Fig. 7. The halation removal process is a process in which a predetermined process is performed on the signal value of each identical pixel in the seven reconstructed images (first images) to create an output image (second image) from which halation has been removed. It should be noted that step S17 in FIG. 6 corresponds to steps S21 to S25 in FIG.

[0035] First, the control unit 11 acquires the signal value of each identical pixel of the first image for each of the multiple detection units 13 (step S21). Note that a specific virtual pixel may be assigned to the first image by user designation or the like. In this case, the control unit 11 acquires the signal value for each virtual pixel.

[0036] Next, the control unit 11 removes abnormal values ​​from the signal values ​​of the same pixel (step S22). For example, N maximum values ​​and N minimum values ​​are removed. That is, film halation is often caused by the specular reflection phenomenon of light on the surface, and the reflectance of that area is an abnormal value that is extremely higher than the surrounding areas, so it is removed. Low abnormal values ​​are also removed. As another example, the median may be used and values ​​other than the median may be removed. Also, abnormal values ​​may be determined using a threshold value. Also, various statistical processing methods may be used to remove abnormal values.

[0037] Next, the control unit 11 calculates the average of the remaining signal values ​​(step S23).

[0038] Next, the control unit 11 judges whether the calculation is completed for all pixels (step S24). If the calculation is completed (step S24; YES), the process proceeds to step S25, and if the calculation is not completed (step S24; NO), the process proceeds to step S21, where the control unit 11 acquires the signal values ​​of the pixels for which the calculation is not completed.

[0039] Next, the control unit 11 generates an image (second image) from the average value calculated for each pixel (step S25).

[0040] (Second image generation process; flaw-emphasizing image generation process) The flow of the flaw-enhanced image generation process will be explained with reference to Fig. 8. The flaw-enhanced image generation process involves comparing the signal values ​​of the same pixels in the seven reconstructed images (first images) with a predetermined processed signal value, extracting pixels whose intensity difference is equal to or exceeds a predetermined ratio, and creating a flaw-enhanced image (second image) based on the signal values, etc. of the extracted pixels. It should be noted that step S17 in Fig. 6 corresponds to steps S31 to S38 in Fig. 8. It should be noted that steps S31 to S35 are the same as steps S21 to S25 in Fig. 7.

[0041] Next, the control unit 11 compares the signal values ​​of each identical pixel in the first images for each of the multiple detection units 13 with the signal values ​​of the same pixel in the image (average value image) generated from the average value in step S35, and extracts signal values ​​of pixels having an intensity difference of more than a predetermined percentage from the multiple first images (step S36).

[0042] Next, the control unit 11 generates a scratch area image based on the signal values ​​of the pixels extracted from the multiple first images (step S37). For example, the signal values ​​of the extracted pixels are subjected to statistical processing such as averaging for each pixel, and the processed signal values ​​are arranged in the original positions of the pixels to generate the scratch area image.

[0043] Next, the control unit 11 superimposes the flaw area image and the average value image to generate a flaw-emphasized image (second image) (step S38).

[0044] (Second image generation process; scattering anisotropy image generation process) The flow of the scattering anisotropy image generation process will be described with reference to Fig. 9. The scattering anisotropy image generation process is a process in which the anisotropy of scattering by a diffusing surface is expressed as, for example, the angle dependency of scattering intensity, and the degree of scattering by the scattering surface is indexed. 6 corresponds to steps S41 to S44 in FIG. 9, and step S41 in FIG. 9 is the same as step S21 in FIG. The detectors may be arranged as shown in FIG. 4 or FIG. 5, but it is preferable to arrange them on a circumference (on a three-dimensional spherical surface) as shown in FIG.

[0045] Next, the control unit 11 evaluates the angle dependency of the scattering intensity based on the position information of the multiple detection units 13, and calculates a feature amount (step S42).

[0046] Here, a method for calculating the feature amount in step S42 will be described. Fig. 10 is a diagram showing the positions (viewpoint positions 0 to 8) of a plurality of detection units 13 and how the light projected from the illumination projection unit 12 is reflected. It is assumed that the upper part of the measurement object H is glossy. The reflected light components include glossy reflected light and diffuse reflected light. The glossy reflected light component is strongly reflected in the direction of the reflection angle corresponding to the angle of incidence between the projected light and the glossy surface, while the diffuse reflected light component is reflected overall.

[0047] Fig. 11 is a diagram showing the light intensity for each viewpoint position. In the case of a glossy member (measurement object H) arranged as shown in Fig. 10, most of the light reflected from the glossy surface is reflected toward viewpoint position 4, so the light intensity at viewpoint position 4 is strong. In the case of a diffusing member, there is no large difference in light intensity for each viewpoint position.

[0048] And because reflection has the above-mentioned properties, by using the following formula (1), the control unit 11 can calculate one feature amount due to gloss for each pixel of the first image for each of the multiple detection units 13. In other words, the control unit 11 can calculate the proportion (one feature amount) of the reflection component due to gloss (for example, viewpoint position 4) among the reflection components. Feature quantity (gloss index) = gloss intensity / (gloss intensity + diffusion intensity) Equation (1) For example, the diffusion intensity may be obtained in advance using another non-glossy sample made of the same material and shape as the measurement object H, and then compared with the gloss intensity obtained in this scattering anisotropy image generation process. Also, in this scattering anisotropy image generation process, the proportion of the reflection component due to gloss at each position can be calculated by taking the minimum value of the signal values ​​of the same pixel in the first image for each of the multiple detection units 13 as the diffusion intensity. It should be noted that the above formula (1) is an example and is not limiting.

[0049] Next, the control unit 11 judges whether the calculation is completed for all pixels (step S43). If the calculation is completed (step S43; YES), the process proceeds to step S44, and if the calculation is not completed (step S43; NO), the process proceeds to step S41, where the control unit 11 acquires the signal values ​​of the pixels for which the calculation is not completed.

[0050] Next, the control unit 11 generates an image (second image) representing the degree of diffusion based on the calculated feature amount (step S44). For example, one feature amount (gloss index) calculated by the above formula (1) for each pixel of the first image for each of the multiple detection units 13 is arranged at the original position of the pixel, and the color is changed depending on the size of the feature amount (gloss index), thereby finally generating one heat map image (second image).

[0051] From the above, the image forming system 100 includes an illumination projection unit 12 that irradiates the measurement object with light of a two-dimensional illumination pattern, a plurality of detection units 13 that are installed in different locations and detect the light intensity of reflected light from the measurement object due to the light of the illumination pattern, an illumination control unit (control unit 11) that sequentially switches between illumination patterns from among the plurality of illumination patterns and irradiates light from the illumination projection unit, a first image generation unit (control unit 11) that generates a two-dimensionally reconstructed first image for each detection unit 13 from the illumination pattern and the light intensity of the reflected light corresponding to the illumination pattern, and a second image generation unit (control unit 11) that generates a new second image based on the plurality of first images generated for each detection unit 13. This makes it possible to realize appearance inspection of an object with an inexpensive and simple configuration and simple image processing. For example, even in cases where there is no distinguishing structure on the object, such as in the case of a film, or in the case of an object that has a variety of shapes and is not necessarily flat, such as a resin molding, it is possible to easily generate a second image using the first image.

[0052] Moreover, the detection unit 13 is a photodiode, and is therefore inexpensive.

[0053] In addition, the illumination projection unit 12 includes a DMD, and the control unit 11 corresponds an image for each bit of the RGB image to the clock frequency of the DMD and causes the illumination projection unit 12 to project the image for each bit of the RGB image as an independent pattern, thereby making it possible to increase the number of two-dimensional illumination patterns per unit time.

[0054] Moreover, the second image generating section (control section 11) performs a predetermined process by comparing the light intensity of each of the same pixels in the multiple first images.

[0055] Moreover, the second image generating section (control section 11) generates, as a second image, an image obtained by removing halation and restoring the image of the portion where halation occurs from the plurality of first images through processing. Therefore, the abnormal value is automatically removed without the user having to determine which detection unit 13 has generated it, and it becomes possible to restore the image of the part where halation has occurred. In other words, since it is only when illumination is applied that it becomes clear how halation occurs, the presence or absence of halation cannot be known in advance unless it is actually input into the device, and this reduces the need for the user to consider the position of the subject and illumination. In addition, even if the object is roughly placed on the device or the object is replaced and the situation changes (such as when a leather sheet comes after a film with a glossy surface), halation can be automatically removed without correcting the position of the detection unit 13 each time. Furthermore, the processing is, for example, statistical processing, and can be easily implemented.

[0056] Moreover, the second image generating section (control section 11) identifies scratches from the plurality of first images through processing, and generates an image in which the scratched portion has been subjected to emphasis processing as a second image. Therefore, even if the object to be observed is roughly positioned with respect to the positions of the illumination projection unit 12 and the detection unit 13, it is possible to easily detect scratches occurring at any location on the object. In other words, it is possible to automatically determine scratches on the object. In particular, when it comes to scratches on an object, the presence or absence, location, and circumstances of the scratches are basically random, making it difficult to predict the condition of the scratches in advance, so being able to roughly position them is useful.

[0057] In addition, the second image generation unit (control unit 11) calculates a feature representing scattering anisotropy from the light intensity of a specific location of the object to be measured in the multiple first images, and generates a scattering anisotropy image as the second image based on the feature. Therefore, it is possible to confirm the scattering anisotropy of an object with an inexpensive and simple configuration and simple image processing.

[0058] In addition, the image formation method involves sequentially switching between multiple two-dimensional illumination patterns to irradiate the object to be measured with light, detecting the light intensity of the light of the illumination pattern reflected from the object to be measured at different locations, generating multiple first images reconstructed for each different location where the light intensity was detected from the illumination pattern and the light intensity of the reflected light corresponding to the illumination pattern, and processing the light intensity of each identical pixel in the multiple first images to generate a new second image, making it possible to achieve visual inspection of the object with an inexpensive and simple configuration and simple image processing.

[0059] In addition, the program causes the computer of the image forming system 100, which includes an illumination projection unit 12 that irradiates the object to be measured with light of a two-dimensional illumination pattern, and a plurality of detection units 13 that are installed in different locations and detect the light intensity of the reflected light from the object to be measured due to the light of the illumination pattern, to function as an illumination control unit (control unit 11) that sequentially switches between illumination patterns from among the plurality of illumination patterns and causes light to be irradiated from the illumination projection unit 12, a first image generation unit (control unit 11) that generates a first image reconstructed for each detection unit 13 from the illumination pattern and the light intensity of the reflected light corresponding to the illumination pattern, and a second image generation unit (control unit 11) that generates a new second image based on the plurality of first images generated for each detection unit 13, thereby making it possible to realize appearance inspection of the object with an inexpensive and simple configuration and simple image processing.

[0060] The embodiment and its modified examples of the present invention have been described above, but the description of the above embodiment is merely a preferred example of the present invention, and the present invention is not limited to this.

[0061] For example, from the viewpoint of object detection, it is desirable to configure the base 18 of the object with a material that does not easily reflect the light of the projection projector. Also, light from sources other than the illumination projection unit 12, such as indoor lighting, is reflected by the object and can reduce the S / N ratio of the analysis device, but the reconstructed image can be created in the SPI analysis while taking this effect into account.

[0062] Incidentally, the reality is that the occurrence of halation cannot be known in advance unless the object is actually put into the device. In other words, since halation also depends on the macroscopic curvature of the object, halation may occur at a certain position of the detection unit 13, but it is unclear at which position of the detection unit 13 it will occur. Also, since halation depends on the state of the object, halation may not occur at all positions. On the other hand, if light is directed in all directions, this means that scattering occurs at that location, which is usually not an abnormal value and is considered to be equivalent to the reflectance of a normal object, so the possibility of halation occurring at all detection units 13 is low. In other words, the reality is that it is difficult to predict the occurrence of halation in advance. In addition, factors that cause the occurrence of halation include the optical axis of the illumination projection unit 12, the direction of the axis of the detection unit 13, the arrangement position, and further the surface condition and direction of the target object. Therefore, when the target object is different, the direction of halation that occurs for light irradiated from the same direction changes.

[0063] In addition, the present embodiment is not limited to the scratch-emphasizing image generation process. The superimposition process may express the outline of the scratch area, and the method of acquiring the average image may not take into consideration the removal of outliers. In addition, a process may be added to determine whether the shape is a scratch or halation by further analyzing the shape. In addition, since the flow in FIG. 8 is obtained by adding steps S36 to S38 to the flow in FIG. 7, it goes without saying that when the halation removal process and the scratch enhancement image generation process are performed together, it is not necessary to perform steps S31 to S35 redundantly.

[0064] Furthermore, the present embodiment is not limited to the scattering anisotropy image generation process. For example, the unevenness of a scratch may be determined from the scattering anisotropy, or a superimposition process may be performed in the same manner as the scratch-enhanced image generation process. Furthermore, before the flow of FIG. 9, an image in which halation has been removed by halation removal processing for each reconstructed image may be applied to the first image in the processing of FIG.

[0065] Also, for example, in the above description, a hard disk or a non-volatile semiconductor memory is used as a computer-readable medium for the program according to the present invention, but the present invention is not limited to this example. As other computer-readable media, a portable recording medium such as a CD-ROM can be applied. Furthermore, a carrier wave can be applied as a medium for providing data of the program according to the present invention via a communication line.

[0066] In addition, the detailed configurations and operations of the devices constituting the image forming system may be modified as appropriate without departing from the spirit and scope of the invention. [Explanation of symbols]

[0067] 100 Image forming system 11 Control section 12 Lighting projection section 13 Detection section 14 Storage section 15 Display 16 Control section 17 Communications Department

Claims

1. an illumination projection unit that irradiates a measurement object with light of a two-dimensional illumination pattern; A plurality of detection units are installed at different locations and detect the light intensity of the reflected light from the measurement object due to the light of the illumination pattern; an illumination control unit that sequentially switches the illumination pattern from among the plurality of illumination patterns and causes the illumination projection unit to emit light; a first image generating unit configured to generate a first image that is two-dimensionally reconstructed for each of the detection units from the illumination pattern and the light intensity of the reflected light corresponding to the illumination pattern; a second image generating unit configured to generate a new second image based on a plurality of first images generated for each of the detection units; An image forming system comprising:

2. 2. The image forming system according to claim 1, wherein the detection unit is a photodiode.

3. The illumination projection unit includes a DMD. the illumination control unit corresponds an image for each bit of an RGB image to a clock frequency of the DMD; 3. The image forming system according to claim 1, wherein an image for each bit of an RGB image is projected onto the illumination projection unit as an independent pattern.

4. 4. The image forming system according to claim 1, wherein the second image generating section performs a predetermined process by comparing the light intensities of the same pixels in a plurality of the first images.

5. The image forming system according to claim 4, wherein the second image generating unit generates, as the second image, an image obtained by removing halation and restoring the image of the portion corresponding to the occurrence of halation through the processing from the plurality of first images.

6. The image forming system according to claim 4 , wherein the second image generating unit identifies scratches from the plurality of first images by the processing, and generates an image in which the scratched portion is emphasized as the second image.

7. The image forming system according to any one of claims 1 to 3, characterized in that the second image generation unit calculates a feature representing scattering anisotropy from the light intensity of each identical pixel in the multiple first images, and generates a scattering anisotropy image as the second image based on the feature.

8. Sequentially switching between a plurality of two-dimensional illumination patterns, the illumination pattern is irradiated onto the measurement object, and Detecting the light intensity of the light of the illumination pattern reflected from the measurement object at different locations; generating a plurality of first images reconstructed two-dimensionally for each of the different locations where the light intensity is detected, based on the illumination pattern and the light intensity of the reflected light corresponding to the illumination pattern; A method of forming an image, the method processing the light intensity of each identical pixel in the plurality of first images to generate a new second image.

9. A computer of an image forming system including an illumination projection unit that irradiates a measurement object with light of a two-dimensional illumination pattern, and a plurality of detection units that are installed at different locations and detect the light intensity of reflected light from the measurement object due to the light of the illumination pattern, an illumination control unit that sequentially switches the illumination pattern from among the plurality of illumination patterns and causes the illumination projection unit to emit light; a first image generating unit that generates a first image that is two-dimensionally reconstructed for each of the detection units from the illumination pattern and the light intensity of the reflected light corresponding to the illumination pattern; a second image generating unit configured to generate a new second image based on the plurality of first images generated for each of the detection units; A program that functions as a

Images