Image inspection system and image inspection method

Abstract

To provide an image inspection system capable of shortening an inspection time required for inspection while suppressing deterioration in the accuracy of inspection.SOLUTION: An image inspection system 100 for inspecting an object using multi-wavelength images comprises: an imaging section that acquires a confirmation image by receiving light from a first object via a spectroscopic section for dispersing the light; and a processing section that performs processing on the confirmation image acquired by the imaging section. The processing section generates each image data for respective wavelengths constituting multi-wavelengths included in the confirmation image on the basis of the confirmation image and spectroscopic characteristics of the spectroscopic section, designates a plurality of first wavelengths included in the multi-wavelengths, calculates a correct answer rate of first combined image data on the basis of the first combined image data obtained by combining each image data in the designated first wavelengths and a predetermined correct answer image corresponding to the first object, and determines a plurality of second wavelengths used for inspecting an inspection image obtained by imaging a second object by the imaging section as the plurality of designated first wavelengths if the correct answer rate is equal to or greater than a threshold value.SELECTED DRAWING: Figure 9A

Description

[Technical Field]

[0001] This disclosure relates to an image inspection system and an image inspection method for inspecting objects using multi-wavelength images. [Background technology]

[0002] By utilizing spectral information from numerous narrow-band frequencies, such as tens of bands, it becomes possible to perform detailed analysis of objects that was impossible with conventional RGB images. Cameras that acquire such multi-wavelength information are called "hyperspectral cameras." Hyperspectral cameras are used in various fields, including food inspection, biological testing, pharmaceutical development, and mineral component analysis. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] International Publication No. 2021 / 192891 [Overview of the project] [Problems that the invention aims to solve]

[0004] When inspecting an object using images captured by a hyperspectral camera, increasing the number of wavelengths included in the image of the object can improve inspection accuracy, but it also increases inspection time. Conversely, using fewer wavelengths shortens inspection time, but reduces inspection accuracy.

[0005] This disclosure provides an image inspection system and an image inspection method that can shorten the inspection time required for inspection while suppressing a decrease in the accuracy of the inspection. [Means for solving the problem]

[0006] One aspect of the present disclosure is an image inspection system for inspecting an object using multi-wavelength images, comprising: an imaging unit that receives light from a first object via a spectrometer that spectrally analyzes light and acquires a confirmation image; and a processing unit that processes the confirmation image acquired by the imaging unit, wherein the processing unit generates image data for each wavelength constituting the multi-wavelengths included in the confirmation image based on the confirmation image and the spectral characteristics of the spectrometer, specifies a plurality of first wavelengths included in the multi-wavelengths, and generates a first combined image data by combining the image data for each of the specified first wavelengths. 、 The first object This shows the target color distribution. Correct answer image data and The misjudgment area is the area where the color distribution differs between the first combination image data and the correct answer image, based on the distribution of light received at each wavelength. Based on this, the system calculates the accuracy rate of the first combination image data, and if the accuracy rate is above a threshold, it determines the plurality of second wavelengths used for inspecting the inspection image obtained by imaging the second object with the imaging unit to be the specified plurality of first wavelengths.

[0007] One aspect of the present disclosure is an image inspection method for inspecting an object using a multi-wavelength image, comprising: an image acquisition step of receiving light from a first object via a spectrometer that spectrally analyzes light and acquiring a confirmation image; and a processing step of processing the acquired confirmation image, wherein the processing step includes generating image data for each wavelength constituting the multi-wavelengths included in the confirmation image based on the confirmation image and the spectral characteristics of the spectrometer; specifying a plurality of first wavelengths included in the multi-wavelengths; and first combined image data obtained by combining the image data for each of the specified first wavelengths. 、 The first object This shows the target color distribution. Correct answer image and The misjudgment area is the area where the color distribution differs between the first combination image data and the correct answer image, based on the distribution of light received at each wavelength. The image inspection method comprises the steps of: calculating the accuracy rate of the first combination image data based on the above; and, if the accuracy rate is above a threshold, determining a plurality of second wavelengths to be used for inspecting the inspection image obtained by imaging the second object to be the specified plurality of first wavelengths. [Effects of the Invention]

[0008] According to the present disclosure, it is possible to shorten the inspection time required for inspection while suppressing a decrease in inspection accuracy.

Brief Description of the Drawings

[0009] [Figure 1] Block diagram of an image inspection system according to an embodiment of the present disclosure [Figure 2A] Diagram showing an example of a confirmation image acquired by a camera [Figure 2B] Diagram showing an example of each image data of each wavelength included in the confirmation image [Figure 3] Conceptual diagram of a correct answer image and a confirmation image [Figure 4A] Diagram showing an example of an area specified in the correct answer image [Figure 4B] Graph showing the amount of light received for each wavelength for each specified area [Figure 4C] Graph showing the degree of contribution for each wavelength number [Figure 5] Diagram showing an example of a state in which the image data of each wavelength included in the confirmation image is arranged in order of degree of contribution [Figure 6] Graph showing an example of the correct answer rate for combined image data obtained by combining each image data of the first wavelength in various combinations [Figure 7] Diagram showing an example of an operation for manually selecting the image data of each wavelength included in the confirmation image [Figure 8A] Flowchart showing an example of the procedure of an image inspection method executed by an image inspection system [Figure 8B] Flowchart showing an example of the procedure of an image inspection method executed by an image inspection system (continuation of FIG. 8A) [Figure 9A] Graph showing an example of the correct answer rate with respect to the number of wavelengths when wavelengths are added and combined in descending order of degree of contribution [Figure 9B] Graph showing an example of the correct answer rate with respect to the number of wavelengths when wavelengths are arbitrarily combined regardless of the degree of contribution [Figure 10] Block diagram of Modification 1 of an image inspection system [Figure 11] Block diagram of Modification 2 of an image inspection system [Figure 12] Block diagram of modified image inspection system, version 3. [Modes for carrying out the invention]

[0010] The embodiments will be described in detail below, with reference to the drawings as appropriate. However, unnecessary details may be omitted. For example, detailed explanations of already well-known matters or redundant explanations of substantially identical configurations may be omitted. This is to avoid the following explanation becoming unnecessarily verbose and to facilitate understanding by those skilled in the art. The accompanying drawings and the following explanation are provided to enable those skilled in the art to fully understand this disclosure and are not intended to limit the subject matter described in the claims.

[0011] Figure 1 is a block diagram of an image inspection system according to an embodiment of the present disclosure. The image inspection system 100 includes a camera 10, an illumination device 20, a control device 30, and a display operation device 40. The image inspection system 100 inspects the characteristics of an imaged object (workpiece, subject) by analyzing the image data of each wavelength constituting the multi-wavelengths contained in the image of the imaged object. In this embodiment, the camera 10, the illumination device 20, the control device 30, and the display operation device 40 are each composed of independent devices. For example, the illumination device 20 may be detachable from the camera 10. The control device 30 is connected to the camera 10, the illumination device 20, and the display operation device 40 via an interface (wired or wireless) not shown. In other words, the camera 10, the illumination device 20, the control device 30, and the display operation device 40 have communication devices.

[0012] Camera 10 is a device capable of capturing images of an object that include multi-wavelength information, such as spectral information of several tens of bands, known as multi-wavelength images. A multi-wavelength image is an image that contains components of many wavelengths (for example, three or more, for example, 20 wavelengths (20 bands)). Camera 10 is also called a hyperspectral camera. Camera 10 is installed, for example, in a fixed position and captures images of an object. Camera 10 has a spectroscopic unit 11, an imaging unit 12, and a storage unit 13.

[0013] The spectral unit 11 may be composed of a transmission type optical element (e.g., a filter or prism) or a reflection type optical element (e.g., a diffraction grating). The spectral unit 11 receives light from an object irradiated with light by the illumination device 20 and spectrally separates it into multiple wavelengths. In other words, the spectral unit 11 is arranged in correspondence with the imaging unit 12 and has a wavelength selection function for each pixel of the imaging unit 12. The spectral unit 11 may be divided into multiple spectral regions. Each spectral region allows a predetermined wavelength to pass through. One or more wavelengths passed through each spectral region may overlap in some respects, or they may not overlap.

[0014] The imaging unit 12 is composed of an image sensor and the like that converts light received via the spectral unit 11 into an electrical signal and captures an image of the object. The wavelength ranges of each pixel of the imaging unit 12 may or may not overlap at least partially.

[0015] The storage unit 13 consists of a memory or disk for storing settings, operating programs, or captured images of the camera 10.

[0016] The illumination device 20 is a device that irradiates an object with illumination light, enabling the camera 10 to image the object. The illumination device 20 has a light-emitting unit 21 and a storage unit 22. The light-emitting unit 21 is a light source capable of irradiating an object with illumination light, and is composed of a light-emitting element or a light bulb, etc. The storage unit 22 is composed of a memory or disk, etc., that stores settings or operation programs for the illumination device 20. The illumination light from the illumination device 20 may be white light or light having at least one predetermined wavelength.

[0017] The control device 30 is a device that controls the image inspection system 100, and is, for example, a PC (Personal Computer). The control device 30 comprises a processing unit that performs various processes on images captured by the camera 10. The control device 30 has a control unit 31, a calculation unit 32, a determination unit 33, and a storage unit 34. The control unit 31, the calculation unit 32, and the determination unit 33 read, for example, an image inspection program stored in the storage unit 34, and control the operation of the control device 30 and, consequently, the image inspection system 100.

[0018] The control unit 31 consists of a processor and other components that are responsible for the overall operation control of the control device 30. The control unit 31 also performs various operations, such as issuing instructions for the control of the camera 10 and the lighting device 20, and setting various parameters (e.g., camera settings, lighting settings).

[0019] The calculation unit 32 is composed of, for example, a processor specialized for calculation processing. The calculation unit 32 performs tasks such as deriving the accuracy rate and contribution rate, which will be described later, generating various images, and specifying the wavelength to be inspected from among the wavelengths contained in the image.

[0020] The determination unit 33 is composed of a processor and the like specialized for various determination processes. The determination process may include, for example, determining whether the accuracy rate of the confirmation image, inspection image, and combined image data described later meets predetermined requirements. The confirmation image is an image obtained by the camera 10 that contains components of all wavelengths and is used for preparing for the inspection (specifying the wavelengths to be used for the inspection). The inspection image is an image obtained by the camera 10 that contains components of all wavelengths and is used for the inspection. The combined image data is data corresponding to an image that contains components of some of the multiple wavelengths included in the confirmation image or inspection image. Note that the confirmation image may be used for the inspection.

[0021] The storage unit 34 is configured as a memory or disk for storing settings or operation programs for the control device 30.

[0022] The display operation device 40 functions as a display device that shows images captured by the camera 10, processing results by the control device 30, etc., and also functions as an operation unit in which the user inputs various operations. The display operation device 40 includes an initial setting unit 41, an inspection condition setting unit 42, an image display unit 43, and a result display unit 44.

[0023] The initial setup unit 41 is one of the operation units that receives user input for initial setup of the display operation device 40, and is composed of an input device such as a keyboard or mouse. The initial setup unit 41 performs initial setup based on this input. The inspection condition setting unit 42 is one of the operation units that receives user input for inspection conditions of the image inspection system 100, and is composed of an input device such as a keyboard or mouse. The inspection condition setting unit 42 sets the inspection conditions based on this input. The inspection conditions include, for example, the number of wavelengths used for image inspection, the accuracy rate used for image inspection, or other inspection conditions. The accuracy rate is an indicator for identifying whether the object being inspected is a good product or a defective product. One input device may also serve as the operation unit for both the initial setup unit 41 and the inspection condition setting unit 42. The set initial conditions and inspection conditions may be sent to the control device 30 and stored in the storage unit 34.

[0024] The image display unit 43 is a display that shows images captured by the camera 10. The result display unit 44 is a display that shows the results of image processing by the control device 30. One display can also serve as both the image display unit 43 and the result display unit 44.

[0025] Alternatively, the initial setting unit 41 and the inspection condition setting unit 42 may be configured using input devices, and the image display unit 43 and the result display unit 44 may be configured using displays separate from these input devices.

[0026] Figure 2A shows an example of a confirmation image GC acquired by camera 10. Figure 2B shows the image data gd for each wavelength included in the confirmation image GC of Figure 2A.

[0027] The illumination device 20 irradiates the first object, which is the object to be inspected by the image inspection system 100, with illumination light. The spectrometer 11 receives the light from the object irradiated by the illumination device 20 and spectrally separates it into multiple wavelengths. The imaging unit 12 converts the light received via the spectrometer 11 into an electrical signal and acquires the confirmation image GC shown in Figure 2A. Therefore, the confirmation image GC is an image showing the first object.

[0028] Similarly, although not shown, the illumination device 20 irradiates the first object, which is the object to be inspected by the image inspection system 100, with illumination light. The spectrometer 11 receives the light from the object irradiated by the illumination device 20 and spectrally separates it into multiple wavelengths. The imaging unit 12 converts the light received via the spectrometer 11 into an electrical signal and acquires the confirmation image GC shown in Figure 2A. Therefore, the confirmation image GC is an image showing the first object.

[0029] In this embodiment, the object (for example, the first object and the second object described later) may be a painted object and may be subject to inspection. The first object is used for preliminary preparation for the inspection of the second object using multi-wavelength imaging. A confirmation image is obtained when the first object is imaged. The second object is the actual object to be inspected. An inspection image is obtained when the second object is imaged. The first and second objects may be of the same kind (objects with similar paint, objects with similar color distribution, predetermined parts or finished products). The first object may also be the actual object to be inspected, and the first object may be one of the second objects. In other words, the same set of wavelengths used in the inspection of one object may be used in the inspection of subsequent objects. For example, the first object may be the first object to be inspected among a set of second objects.

[0030] The control device 30 acquires the confirmation image GC shown in Figure 2A from the camera 10. This confirmation image GC contains images of each wavelength that constitute multiple wavelengths, such as spectral information of several tens of bands, due to the spectral characteristics of the spectroscopic unit 11. The calculation unit 32 of the control device 30 generates image data gd for wavelengths 1 to 20 that constitute multiple wavelengths included in the confirmation image GC, as shown in Figure 2B, based on the confirmation image GC and the spectral characteristics of the spectroscopic unit 11. In each image data gd, the more light received at the wavelength indicated by a predetermined image data gd, the larger the pixel value of this predetermined image data gd becomes (whiter), and the less light received at this wavelength, the smaller the pixel value of this predetermined image data gd becomes (blacker). In this example, the number of wavelengths is 20, but the number of wavelengths is not limited to a specific number; it may be 19 or fewer wavelengths, or 21 or more wavelengths.

[0031] Figure 3 is a conceptual diagram of the correct answer image GA and the verification image GC. The correct answer image GA is, for example, an image of the correct reference object itself, which is acquired in advance by the control device 30 and stored in the memory unit 34. Specifically, it is a sample image that serves as a color sample of the object. In other words, the correct answer image GA is an image that shows the target color distribution for the verification image and the inspection image. The correct answer image GA is an image of the first object, just like the verification image GC. The correct answer image GA is also the verification image GC. The determination unit 33 of the control device 30 determines whether the object matches the reference object and is a correct item (for example, whether or not it has defects) by determining whether the verification image GC obtained by the camera 10 imaging the first object to be inspected matches the correct answer image GA.

[0032] The concept used in this determination is the correct answer rate. The determination unit 33 of the control device 30 compares the light reception amount of each wavelength image included in the confirmation image GC with the light reception amount of each wavelength image in the correct answer image GA, and determines the portion where the color of the confirmation image GC (the distribution of the light reception amount of each wavelength) is different from the color of the correct answer image GA (the distribution of the light reception amount of each wavelength). Then, the calculation unit 32 calculates the area of this determined different portion, that is, the so-called misjudgment area. Furthermore, the calculation unit 32 calculates what percentage of the misjudgment area exists with respect to the area of the object. The correct answer rate is obtained by this series of calculations, and the formula for the correct answer rate is expressed as, for example, (correct answer rate = 1 - ratio of misjudgment area to object area).

[0033] In the example of FIG. 3, the correct answer image GA and the confirmation image GC have an area of 1 cm 2 . The correct answer image GA has a region of a second color whose color is different from the first color in the surrounding area at its central portion, and the area thereof is 0.3 2 cm 2 . On the other hand, the confirmation image GC, similar to the correct answer image GA, has a region of a second color whose color is different from the first color in the surrounding area at its central portion, but the area thereof is 0.2 2 cm 2 , and the size of the region of the second color is different from that of the correct answer image GA. In this case, the correct answer rate of the confirmation image GC is obtained as follows.

[0034] · Area of object: 1 2 cm 2 · Misjudgment area: 0.3 2 - 0.2 2 cm 2 · Correct answer rate: 1 - (0.3 2 - 0.2 2 ) / 1 2 = 95%

[0035] However, the above-described method for calculating the correct answer rate is merely an example, and the method for calculating the correct answer rate is not limited to that described here.

[0036] Figures 4A, 4B, and 4C illustrate the process of determining the contribution of each wavelength. The contribution is a value that indicates the degree to which each wavelength contributes to the calculation of the accuracy rate of the confirmation image, as explained in Figure 3. The contribution is calculated based on the variation in the amount of light received for each wavelength obtained by imaging the first object with the camera 10, according to each region of the first object. In other words, each wavelength that makes up the multi-wavelength spectrum has variation in the amount of light received for each region of the first object (even for a single wavelength, the amount of light received differs depending on the region), and the greater the variation, the easier it is to correctly distinguish and identify the region, and the greater the contribution to the accuracy rate.

[0037] Figure 4A shows the regions specified in the confirmation image or correct answer image for calculating the contribution. Regions a, b, c, and d in Figure 4A are regions used in the confirmation image or correct answer image obtained by imaging the first object when determining the contribution of each wavelength. Before acquiring the confirmation image, the calculation unit 32 of the control device 30 may specify regions a, b, c, and d in the confirmation image or correct answer image and store them in the storage unit 34. Regions a, b, c, and d can be any region, for example, regions that are inspected frequently. Alternatively, the display operation device 40 may receive an operation input from the user via the inspection condition setting unit 42 that encloses an arbitrary region in the confirmation image or correct answer image, specify regions a, b, c, and d, and transmit this region specification information to the control device 30. Then, the calculation unit 32 of the control device 30 may specify regions a, b, c, and d based on the region specification information from the display operation device 40. These region specifications correspond to the specification of regions for which the accuracy rate of the confirmation image, inspection image, or combined image data is later calculated. Furthermore, this designated region may coincide with the region being examined in the actual examination image. Also, regions a, b, c, and d may be designated together as a single region.

[0038] Figure 4B shows a graph illustrating the amount of light received (light received distribution) for each wavelength in a specified region. In Figure 4B, the amount of light received in each region a, b, c, and d differs for each wavelength shown on the horizontal axis, and the variation in the amount of light received in regions a, b, c, and d differs for each wavelength. For example, at wavelengths around 480 nm, the amount of light received is larger in all regions compared to wavelengths longer than 550 nm, but the variation in the amount of light received in each region a, b, c, and d is also larger. Therefore, it can be said that wavelengths around 480 nm have a larger contribution rate than wavelengths longer than 550 nm.

[0039] Specifically, the calculation unit 32 of the control device 30 compares the amount of light received in the designated regions a, b, c, and d and calculates the contribution of each wavelength. For calculating the contribution, multivariate analysis (e.g., principal component analysis) may be used, or the Mahalanobis distance may be used. The Mahalanobis distance is an index that shows how far a group of values ​​is from the mean (centroid) in units of standard deviation, and is also a measure of how similar two data groups are. Data groups with high similarity have a large overlap, and the Mahalanobis distance is short. In this case, the parts that do not overlap represent the differences between the two data groups. For example, the shorter the Mahalanobis distance, the lower the contribution, and the longer the Mahalanobis distance, the higher the contribution.

[0040] Figure 4C shows a graph illustrating the contribution of the selected wavelength to the corresponding wavelength number. In Figure 4C, the horizontal axis represents the wavelength number, and the vertical axis represents the contribution. In Figure 4C, the wavelength numbers are arranged in descending order of contribution. In other words, in Figure 4C, when the contributions of each wavelength calculated by the calculation unit 32 are arranged in descending order, they are wavelength 3, wavelength 7, wavelength 8, wavelength 4, wavelength 5, ..., wavelength 19, wavelength 20, wavelength 18, and the graph shows the change in contribution for each of these wavelengths.

[0041] Furthermore, the regions a, b, c, and d to be examined in the confirmation image do not need to be specified. In other words, the entire confirmation image may be the subject of examination. There may be multiple correct answer images.

[0042] Figure 5 shows the image data for each wavelength included in the confirmation image arranged in order of contribution. The arrangement of image data based on contribution may be performed by the calculation unit 32. The calculation unit 32 arranges the image data for each wavelength included in the confirmation image in order of contribution, as shown in Figure 5, according to the contribution determined in the process described above. This order shown in Figure 5 is the same as the order of wavelength numbers on the horizontal axis in Figure 4C. The control device 30 may have the calculation unit 32 give a display instruction to the display operation device 40, for example, via a communication unit (not shown), and the image display unit 43 of the display operation device 40 may display the arranged image data (wavelength numbers: wavelength 3, wavelength 7, wavelength 8, wavelength 4, wavelength 5, ... wavelength 19, wavelength 20, wavelength 18).

[0043] The calculation unit 32 of the control device 30 specifies multiple wavelengths (first wavelengths) from the multiple wavelengths included in the confirmation image, as shown in the area enclosed by the rectangle in Figures 4B and 4C. In this example, the calculation unit 32 specifies 10 first wavelengths from a total of 20 multiple wavelength bands. The calculation unit 32 may specify multiple first wavelengths by combining multiple wavelengths in order of their contribution, for example. The calculation unit 32 may also specify the number of wavelengths to be specified as the first wavelengths. In this case, the number of wavelengths may be specified based on a threshold described later, or it may be predetermined and stored in the storage unit 34, or it may be specified manually via the inspection condition setting unit 42 of the display operation device 40.

[0044] If, for example, 10 first wavelength bands are specified, the calculation unit 32 may combine the image data for these specified 10 bands to generate a first combined image data. The first combined image data is compared with the correct answer image and becomes data used to derive the correct answer rate in the case of multiple wavelengths. Compared with a confirmation image that takes all wavelengths into account, the first combined image data is data in which some wavelength components are omitted.

[0045] Furthermore, the calculation unit 32 calculates the accuracy rate of the generated first combination image data based on the correct answer image corresponding to the first object. The accuracy rate can be calculated using the method described in Figure 3. In this case, the first combination image data is used instead of the confirmation image in Figure 3. That is, in the comparison between the confirmation image and the correct answer image, an accuracy rate is calculated that takes into account each of the wavelengths included in the confirmation image, and in the comparison between the first combination image data and the correct answer image, an accuracy rate is calculated that takes into account some specific wavelengths (first wavelengths) included in the confirmation image.

[0046] Figure 6 is a graph showing the accuracy rate for the first combined image data, which is a combination of image data of various combinations of the first wavelength.

[0047] The calculation unit 32 combines multiple wavelengths in order of their contribution, as determined by the process described in Figures 4A to 4C, that is, in order of wavelengths with the highest contribution in the confirmation image. It then specifies multiple first wavelength image data and combines the specified image data to generate the first combined image data. In Figure 6, the number of wavelengths combined in order of highest contribution is sequentially increased to generate the first combined image data, and the accuracy rate is plotted on the vertical axis. In this example, the accuracy rate of the image data for wavelength 3, which has the highest contribution, is plotted at the left end of the horizontal axis. Subsequently, the accuracy rates of the first combined image data, which combines image data of wavelengths 7, 8, and so on, in order of contribution, are plotted. At the right end of the horizontal axis, the accuracy rate of the first combined image data, which combines image data of all wavelengths (20 bands), is plotted. Note that the image corresponding to the first combined image data, which combines image data of all wavelengths, is the same as the confirmation image.

[0048] The accuracy of the first combined image data improves as the number of wavelengths (i.e., the number of image data points for each wavelength) used to generate the first combined image data increases. However, although the accuracy of the first combined image data improves with increasing numbers of wavelengths to be combined, the improvement in accuracy eventually saturates. In the graph of Figure 6, for example, after the accuracy of the first combined image data based on image data for 10 bands reaches a predetermined threshold th, no significant improvement in accuracy can be expected even if more image data is added. Therefore, the calculation unit 32 specifies a threshold th to be compared with the accuracy, as shown in Figure 6. The calculation unit 32 then generates the first combined image data using image data with the minimum number of wavelengths (minimum number of bands, e.g., 10 wavelengths) required to reach the threshold th. The threshold th may be a value corresponding to the saturation state of the accuracy, for example.

[0049] Figure 7 shows the operation for manually selecting image data for each wavelength included in the confirmation image. For example, the image data may be displayed in order of contribution, and the selection operation may be performed from there.

[0050] In the examples shown in Figures 4A to 6, the calculation unit 32 of the control device 30 automatically arranges the multi-wavelength image data included in the confirmation image (for example, in order of contribution) and specifies multiple first wavelengths, regardless of the user's intention. In this case, the initial setting unit 41 of the display operation device 40 may set "automatic mode" by accepting user input, for example, from the operation unit. Automatic mode is one of the operating modes of the control device 30 and is a mode that automatically specifies multiple wavelengths (first wavelengths) of the inspection target.

[0051] On the other hand, in the example shown in Figure 7, the initial setting unit 41 of the display operation device 40 may, for example, accept user input from the operation unit and set it to "manual mode," causing the image display unit 43 to display each image data of each wavelength constituting the multi-wavelength system, along with checkboxes. Manual mode is one of the operating modes of the control device 30, and is a mode in which the user manually specifies multiple wavelengths (first wavelengths) to be inspected. The inspection condition setting unit 42 may, in response to user input to the operation unit, select an arbitrary image data from the displayed image data and specify the first wavelength corresponding to the selected image data. In this example, the inspection condition setting unit 42 may, in response to user input to the operation unit, check checkboxes corresponding to wavelengths (image data) to specify multiple first wavelengths and transmit the wavelength specification information to the control device 30. The calculation unit 32 of the control device 30 may then specify multiple first wavelengths based on the wavelength specification information from the display operation device 40. This also allows the calculation unit 32 to specify the number of wavelengths of the first wavelength.

[0052] The image display unit 43 may display image data for each wavelength, as well as identification information for each wavelength (e.g., wavelength number), the contribution of each wavelength, etc. This allows the user to specify a wavelength (image data) via the operation unit of the display operation device 40 while checking the contribution of each wavelength. When a first combined image data is displayed, formed by combining the image data corresponding to the specified first wavelength, consideration may also be given to whether this first combined image data is easy for humans to view. For example, by specifying multiple wavelengths that are close together to generate and display the first combined image data, it becomes easier to clearly see the gradation of the first object. Therefore, when inspecting the image of the second object, displaying a second combined image data (combined image data for inspection) similar to the first combined image data makes it easier to clearly see the gradation of the second object.

[0053] The first combined image data may be a combination of the image data corresponding to the first wavelength and not necessarily an image itself (it may not even reach the stage of generating an image), or it may be an image (integrated image) obtained by combining (integrating) the image data corresponding to the first wavelength. Similarly, the second combined image data may be a combination of the image data corresponding to the second wavelength and not necessarily an image itself (it may not even reach the stage of generating an image), or it may be an image (integrated image) obtained by combining (integrating) the image data corresponding to the second wavelength.

[0054] Next, we will explain the inspection of a second object using multi-wavelength imaging.

[0055] As described above, the calculation unit 32 of the control device 30 acquires a confirmation image based on the imaging of the first object. The calculation unit 32 compares the combination image data for wavelength specification with the correct answer image and calculates the accuracy rate of the combination image for wavelength specification. If this accuracy rate satisfies a predetermined accuracy rate (i.e., is above a threshold th), the calculation unit 32 determines a plurality of first wavelengths included in the combination image data for wavelength specification (for example, a specific number of 10 wavelengths (for example, wavelengths with high contribution)) as a plurality of second wavelengths to be used for inspecting the inspection image obtained when the second object is imaged by the camera 10.

[0056] Furthermore, the calculation unit 32 acquires an inspection image based on imaging of a new second object of the same type as the first object, and uses the combination image data for inspection (second combination image data) that includes the determined multiple second wavelength components. The calculation unit 32 compares the combination image data for inspection with the correct answer image and calculates the accuracy rate of the combination image data for inspection. The determination unit 33 then determines whether the second object is good or defective based on this accuracy rate. For example, the determination unit 33 determines that the second object is good if the accuracy rate is greater than or equal to a threshold th, and determines that the second object is defective if the accuracy rate is less than a threshold th. The method for deriving the accuracy rate during inspection and the method for specifying the area to be inspected may be the same as the method for specifying the wavelength for inspection. In addition, although it has been illustrated that the threshold for determining whether the second object is good or defective is the same as the threshold compared with the accuracy rate for the wavelength specification mentioned above, these thresholds may be different.

[0057] The calculation unit 32 may sequentially image multiple second objects, sequentially acquire inspection images, sequentially generate combination image data for inspection, and sequentially calculate the accuracy rate of the combination image data for inspection. The determination unit 33 may then sequentially determine whether the second object is a good product or a defective product.

[0058] Therefore, when inspecting an inspection image, the image inspection system 100 does not inspect all wavelengths but uses a designated second wavelength for inspection, thus reducing the time required for image inspection of the second object compared to when all wavelengths are used for inspection. Furthermore, even if the number of wavelengths used for inspection of the inspection image is reduced, the image inspection system 100 can suppress a decrease in the inspection accuracy of the inspection image because the wavelength that can obtain an accuracy rate of th or higher using the confirmation image of the first object is determined as the wavelength to be inspected.

[0059] Figures 8A and 8B are flowcharts illustrating an example of the procedure for an image inspection method performed by the image inspection system 100.

[0060] First, the control device 30 acquires the correct image of the reference object to be used as a reference (step S1). For example, when the inspection condition setting unit 42 issues an instruction to acquire the correct image via the operation unit, the control device 30 instructs the control unit 31 to acquire the correct image from a predetermined server via a network (not shown) and store it in the storage unit 34.

[0061] Next, the control unit 31 of the control device 30 takes an image of the target object, acquires a confirmation image, and stores it in the storage unit 34 (step S2). The calculation unit 32 specifies predetermined regions in the correct answer image and the confirmation image, as shown in regions a, b, c, and d in Figure 4A (step S3). These regions a to d correspond to the areas to be inspected.

[0062] Next, the calculation unit 32 of the control device 30 calculates the accuracy rate of the confirmation image relative to the correct answer image in the designated area, for example, in the manner shown in Figure 3 (step S4). In this case, the calculation unit 32 may calculate the accuracy rate for each image data of each designated first wavelength. That is, the calculation unit 32 may compare the image data of a single wavelength with the correct answer image and calculate the accuracy rate for each single wavelength. Furthermore, the calculation unit 32 calculates the contribution of each wavelength to the accuracy rate, for example, in the manner shown in Figures 4A to 4C (step S5).

[0063] Next, the calculation unit 32 determines whether or not to automatically specify the wavelength to be inspected (step S6). For example, if the initial setting unit 41 has set the system to automatic mode, the control unit 31 automatically specifies the wavelength to be inspected (step S6; YES). If the initial setting unit 41 has set the system to manual mode, the control unit 31 manually specifies the wavelength to be inspected (step S6; No).

[0064] When wavelengths are automatically specified, the calculation unit 32 specifies multiple first wavelengths from the multiple wavelengths included in the confirmation image in the manner shown in Figures 4C to 6 (step S7). In this case, the calculation unit 32 may specify multiple first wavelengths according to inspection conditions such as the number of wavelengths and the accuracy rate (i.e., threshold th). Specifically, the calculation unit 32 may specify a predetermined number of first wavelengths in order of highest accuracy rate or contribution, or it may specify multiple first wavelengths whose accuracy rate or contribution exceeds a predetermined threshold set in advance. The calculation unit 32 combines each image data corresponding to the specified first wavelengths and calculates the accuracy rate of the first combination image data by comparing the first combination image data with the correct answer image (step S9).

[0065] On the other hand, if the wavelength is not automatically specified (when the wavelength is manually specified), the calculation unit 32 manually specifies image data by specifying multiple first wavelengths in the manner shown in Figure 7 (step S8). The calculation unit 32 generates a first combined image data by combining each image data corresponding to the specified first wavelength, and calculates the accuracy rate of the first combined image data by comparing the first combined image data with the correct answer image (step S9).

[0066] Next, the determination unit 33 of the control device 30 determines whether the accuracy rate of the first combination image data calculated in step S9 is equal to or greater than a predetermined threshold th as shown in Figure 6 (step S10). If the accuracy rate is less than the predetermined threshold th (step S10; NO), the calculation unit 32 manually specifies the image data by specifying multiple first wavelengths again, for example, as shown in Figure 7 (step S8). Then, the calculation unit 32 performs step S9 again, and the determination unit 33 performs step S10 again.

[0067] In step S8, the determination unit 33 may instruct the display operation device 40 to display a guide (recommendation display) for specifying the first wavelength, and the image display unit 43 of the display operation device 40 may display this guide according to the instructions from the control device 30. The guide display makes it easier for the user to make a manual specification in step S8. The guide display is a display that makes it easy to identify what is recommended to be specified as the first wavelength, such as combinations of image data that have a high correct answer rate or contribution. For example, the wavelength number of the recommended wavelength may be highlighted (e.g., displayed in bold, displayed in color). For example, the frame of the image data or checkbox frame of the recommended wavelength may be highlighted (e.g., displayed with a thick frame, displayed with a colored frame).

[0068] If the accuracy rate of the first combination image data calculated in step S9 is equal to or greater than a predetermined threshold th as shown in Figure 6 (step S10; YES), the control unit 31 determines the plurality of second wavelengths to be used for inspecting inspection images obtained by imaging a new second object to be inspected in the future to be the plurality of first wavelengths specified above (step S11). The information of the determined plurality of first wavelengths may be stored in the storage unit 34.

[0069] After step S11 is completed, proceed to Figure 8B. Camera 10 captures an image of the second object, similar to the first object, and obtains an inspection image, which is then transmitted to the control device 30. The determination unit 33 of the control device 30 obtains the inspection image from camera 10 (step S12). The determination unit 33 then determines the inspection image, that is, whether the second object matches the reference object (step S13). In this case, the determination unit 33 synthesizes the image data corresponding to the specified second wavelength included in the inspection image to generate a second combined image data, and calculates the accuracy rate of the second combined image data by comparing the second combined image data with the aforementioned correct answer image. In other words, the accuracy rate when using a specified number of second wavelengths is calculated. The determination unit 33 may then determine, for example, that the second object is a good product if the accuracy rate is equal to or greater than a threshold th, and that the second object is a defective product if the accuracy rate is less than the threshold th. The determination unit 33 transmits information on whether the product is good or defective to the display operation device 40 via the communication device of the control device 30. The result display unit 44 of the display operation device 40 then acquires the inspection result information via the communication device of the display operation device 40 and displays the inspection result. The user (inspector) can confirm and understand the displayed inspection result.

[0070] Therefore, steps S1 to S11 in Figure 8A are the preparation stage for inspecting the second object to be inspected (the stage for specifying the wavelength to be used for inspection). Steps S12 and S13 in Figure 8B are the inspection stage in which the multiple first wavelengths specified in the preparation stage are used as multiple second wavelengths for inspection, and the second object is inspected using images of other wavelengths.

[0071] Figures 9A and 9B are graphs showing examples of the accuracy rate for the number of combined wavelengths. Figure 9A shows the accuracy rate of the second combined image data for the number of wavelengths when wavelengths are added and combined in order of their contribution. Figure 9A shows essentially the same graph as Figure 6. Figure 9B shows the accuracy rate of the second combined image data for the number of wavelengths when wavelengths are combined arbitrarily without depending on their contribution.

[0072] In Figure 9A, wavelengths are added in order of their contribution to generate a second combined image data by synthesizing the image data of each combined wavelength. In this case, the control device 30 can obtain an accuracy rate with the second combined image data of approximately 10 wavelength combinations, similar to the second combined image data of all 20 wavelength combinations in the random case shown in Figure 9B. As a result, the image inspection system 100 can maintain a high accuracy rate while suppressing the amount of data to be processed, even when using second combined image data obtained by combining wavelengths in order of their contribution to the second combined image data (i.e., inspection image) obtained with all wavelength combinations. Therefore, the image inspection system 100 can, as a result, suppress a decrease in the accuracy of image inspection of the object while shortening the inspection time required for image inspection.

[0073] Figure 10 is a block diagram of Modification 1 of the image inspection system. In Modification 1, the camera unit 10A includes the camera 10 and the illumination device 20 shown in Figure 1. That is, the camera unit 10A is a device that integrates the functions of the camera 10 and the illumination device 20.

[0074] Figure 11 is a block diagram of a modified image inspection system, Part 2. In Part 2, the control unit 30A includes the control device 30 and the display operation device 40 shown in Figure 1. That is, the control unit 30A is a device that integrates the functions of the control device 30 and the display operation device 40.

[0075] Figure 12 is a block diagram of Modification 3 of the image inspection system. In Modification 3, the camera unit 10A includes the camera 10 and lighting device 20 in Figure 1, and the control unit 30A includes the control device 30 and display operation device 40 in Figure 1. In other words, Modification 3 is a combination of the camera unit 10A of Modification 1 and the control unit 30A of Modification 2.

[0076] Thus, according to the image inspection system 100 of this embodiment, the image inspection system 100 can shorten the inspection time while suppressing the decrease in inspection accuracy of the object to be inspected using multi-wavelength images. Furthermore, by increasing the wavelengths included in the second combination image data, this second combination image data approaches the image quality of the inspection image, thus approaching the image quality of the correct answer image and improving the accuracy rate. However, the amount of image data increases, which increases the processing load and thus the processing time. The image inspection system 100 can achieve a suitable balance between improving inspection accuracy and shortening inspection time. In addition, the image inspection system 100 can efficiently detect color unevenness and the like of the object to be inspected (second object) by using wavelengths with a high contribution as the inspection wavelength (second wavelength).

[0077] As described above, the image inspection system 100 of this embodiment is an image inspection system that inspects an object using a multi-wavelength image, and comprises an imaging unit (e.g., camera 10) that receives light from a first object via a spectral unit 11 that spectrally analyzes light and acquires a confirmation image GC, and a processing unit (e.g., control device 30) that processes the confirmation image GC acquired by the imaging unit. The processing unit generates image data gd for each wavelength that constitutes the multi-wavelengths included in the confirmation image GC based on the spectral characteristics of the confirmation image GC and the spectral unit 11. The processing unit specifies a plurality of first wavelengths included in the multi-wavelengths and calculates the accuracy rate of the first combination image data based on a first combination image data obtained by combining the image data gd for each specified first wavelength and a predetermined correct answer image GA corresponding to the first object. If the accuracy rate is greater than or equal to a threshold th, the processing unit determines a plurality of second wavelengths to be used for inspecting the inspection image obtained by imaging a second object with the imaging unit to be the specified plurality of first wavelengths.

[0078] As a result, the image inspection system 100 can determine a plurality of first wavelengths that constitute the first combination image data, satisfying a threshold th or higher accuracy rate, by omitting some of the wavelengths included in the confirmation image GC to generate a first combination image data and comparing it with the correct answer image GA. Therefore, when actually performing an image inspection of the second object, the system can perform an image inspection of the second object, satisfying a threshold th or higher accuracy rate, by omitting some of the wavelengths included in the inspection image to generate a second combination image data and comparing it with the correct answer image GA. Furthermore, since some of the wavelengths in the inspection image are omitted from the second combination image data, the amount of image data is small, thus reducing the processing load related to comparing the second combination image data with the inspection image. Consequently, the image inspection system 100 can shorten the inspection time required for inspection while suppressing a decrease in inspection accuracy.

[0079] The processing unit may also specify regions a to d for calculating the accuracy rate in the confirmation image GC.

[0080] This allows the image inspection system 100 to specify which parts of the confirmation image it wishes to obtain accuracy rates or image inspection results for.

[0081] Furthermore, the processing unit may calculate the contribution of each wavelength to the accuracy rate based on the variation in the amount of light received for each wavelength obtained by imaging the first object by the imaging unit, corresponding to each region of the first object, and specify a plurality of first wavelengths based on the contribution.

[0082] This allows the image inspection system 100 to determine multiple first wavelengths, taking into account their contribution to the accuracy rate, that is, the ease with which each position of the first object can be identified.

[0083] Furthermore, the processing unit may specify multiple first wavelengths by combining multiple wavelengths in order of their contribution.

[0084] This allows the image inspection system 100 to determine multiple wavelengths that are easy to identify for each position of the first object as multiple first wavelengths.

[0085] Furthermore, the processing unit may sequentially increase the number of wavelengths combined in order of their contribution until the accuracy rate reaches a threshold th or higher.

[0086] As a result, the image inspection system 100 can minimize the number of specified first wavelengths while satisfying the desired accuracy rate in image inspection. Therefore, the image inspection system 100 can minimize the processing load related to image inspection.

[0087] Furthermore, the processing unit may specify a plurality of first wavelengths in response to input operations to the operation unit (for example, the initial setting unit 41 or the inspection condition setting unit 42).

[0088] This allows the image inspection system 100 to manually specify a first wavelength desired by the user. For example, the image inspection system 100 can specify the first wavelength by taking into account the naturalness of the combined image data based on the specified first wavelength.

[0089] The processing unit may also specify the number of wavelengths of the designated first wavelength.

[0090] This allows the image inspection system 100 to generate combined image data, taking into account the wavelength count of the specified first wavelength.

[0091] The processing unit may also specify a threshold th that is compared with the correct answer rate.

[0092] This allows the image inspection system 100 to adjust the inspection accuracy of the image inspection of the second object by adjusting the threshold th.

[0093] The imaging unit may also receive light from the second object via the spectroscopic unit 11 to acquire an inspection image. The processing unit may generate image data for each wavelength constituting multiple wavelengths based on the inspection image and the spectral characteristics of the spectroscopic unit 11, and inspect the second object based on a second combined image data obtained by combining the determined image data for each of the second wavelengths and the correct answer image GA.

[0094] This allows the image inspection system 100 to use a first wavelength, specified using a confirmation image of a first object, as the second wavelength for inspecting the inspection image obtained by imaging a second object. In other words, the image inspection system 100 can perform image inspection using the inspection image with the same inspection accuracy and inspection time as image inspection using the confirmation image. [Industrial applicability]

[0095] This disclosure is useful for image inspection systems and image inspection methods that can shorten the inspection time required for inspection while suppressing a decrease in the accuracy of the inspection. [Explanation of Symbols]

[0096] 10 Cameras 10A Camera Unit 11 Spectroscopic section 12 Imaging Unit 13 Storage section 20 Lighting devices 21. Lighting unit 22 Memory section 30 Control device 30A Control Unit 31 Control Unit 32 Arithmetic section 33 Judgment section 34 Storage section 40 Display operation device 41 Initial setting section 42 Inspection Condition Setting Unit 43 Image display section 44 Result display area 100 Image Inspection Systems

Claims

1. An image inspection system that uses multi-wavelength images to inspect objects, An imaging unit receives light from a first object via a spectrometer that spectrally separates light and acquires a confirmation image, A processing unit that processes the confirmation image acquired by the imaging unit, Equipped with, The aforementioned processing unit, Based on the confirmation image and the spectral characteristics of the spectroscopic unit, image data for each wavelength constituting the multiple wavelengths included in the confirmation image is generated. Specify multiple first wavelengths included in the multi-wavelength spectrum, The first combined image data, obtained by combining each image data of a specified first wavelength, is compared with the correct answer image showing the target color distribution of the first object. Based on the misjudgment area, which is the area of ​​the portion where the color distribution differs based on the distribution of light received at each wavelength between the first combination image data and the correct answer image, the accuracy rate of the first combination image data is calculated. If the accuracy rate is above a threshold, the plurality of second wavelengths used for inspecting the inspection image obtained by imaging the second object with the imaging unit are determined to be the specified plurality of first wavelengths. Image inspection system.

2. The processing unit specifies the region in the confirmation image for which the accuracy rate is calculated. The image inspection system according to claim 1.

3. The aforementioned processing unit, Based on the variation in the amount of light received for each wavelength obtained by imaging the first object with the imaging unit, corresponding to each region of the first object, the contribution of each wavelength to the accuracy rate is calculated. Based on the contribution, the plurality of first wavelengths are specified. The image inspection system according to claim 1 or 2.

4. The processing unit specifies the plurality of first wavelengths by combining a plurality of wavelengths in order of their contribution, The image inspection system according to claim 3.

5. The processing unit sequentially increases the number of wavelengths to be combined in order of their contribution until the accuracy rate exceeds the threshold. The image inspection system according to claim 4.

6. The processing unit, in response to an input operation to the operation unit, specifies the plurality of first wavelengths. The image inspection system according to claim 1 or 2.

7. The processing unit specifies the number of wavelengths of the designated first wavelength. The image inspection system according to any one of claims 1 to 6.

8. The processing unit specifies the threshold to be compared with the correct answer rate. The image inspection system according to any one of claims 1 to 7.

9. The imaging unit receives light from the second object via the spectroscopy unit to acquire the inspection image. The aforementioned processing unit, Based on the inspection image and the spectral characteristics of the spectroscopic unit, image data for each wavelength constituting multiple wavelengths is generated. The second object is inspected based on the second combined image data obtained by combining the image data of the plurality of second wavelengths that have been determined, and the correct answer image. The image inspection system according to any one of claims 1 to 8.

10. An image inspection method that uses multi-wavelength images to inspect an object, An image acquisition step involves receiving light from a first object via a spectrometer that spectrally separates light and acquiring a confirmation image, A processing step which involves processing the acquired confirmation image, It has, The processing step described above is: The steps include generating image data for each wavelength constituting the multi-wavelengths included in the confirmation image based on the confirmation image and the spectral characteristics of the spectroscopic unit, A step of specifying multiple first wavelengths included in the multi-wavelength range, The steps include: comparing a first combined image data obtained by combining image data of each specified first wavelength with a correct answer image showing the target color distribution of the first object, and calculating the accuracy rate of the first combined image data based on the misjudgment area, which is the area of ​​the color distribution that differs based on the distribution of light received at each wavelength between the first combined image data and the correct answer image; If the accuracy rate is above a threshold, the steps include determining the plurality of second wavelengths used for inspecting the inspection image obtained by imaging the second object to be the specified plurality of first wavelengths, Image inspection method having

Images