Inspection system, image processing method, and defect inspection apparatus

Abstract

The present application provides an inspection system, an image processing method, and a defect inspection apparatus that can effectively suppress noise even when a periodic state change is measured. The present inspection system includes an image processing section that suppresses noise of a complex image in which each pixel is represented by a complex number representing a periodic change in a vibration state of an inspection object. The image processing section acquires a similarity of pixels included in an object image region defined in the complex image and pixels included in a plurality of reference image regions defined separately from the object image region in the complex image by comparing the complex numbers representing the respective pixels with each other, and performs a noise suppression process of the object image region using a weight based on the acquired similarity.

Description

Technical Field

[0001] This invention relates to an inspection system, an image processing method, and a defect inspection device. Background Technology

[0002] Previously, defect inspection devices that measure the vibration state of the object being inspected were known. Such devices are disclosed, for example, in Japanese Patent Application Publication No. 2017-219318.

[0003] The defect inspection apparatus disclosed in Japanese Patent Application Publication No. 2017-219318 includes: an excitation unit for exciting elastic waves onto an object to be inspected; an illumination unit for stroboscopic illumination of a measurement area on the surface of the object to be inspected; and a displacement measuring unit. The displacement measuring unit is configured to measure the displacement in the front-rear direction of each point in the measurement area at at least three different phases of the elastic wave by controlling the phase of the elastic wave and the timing of the stroboscopic illumination. Furthermore, by measuring the displacement in the front-rear direction of each point in the measurement area, the defect inspection apparatus measures the vibration state (amplitude and phase) of each point in the measurement area. Moreover, Japanese Patent Application Publication No. 2017-219318 discloses a structure for generating an image based on the measured vibration state (amplitude and phase) of each point in the measurement area.

[0004] Although not described in Japanese Patent Application Publication No. 2017-219318, noise sometimes appears in the generated image when an image is generated by measuring a measurement area. For example, in the defect inspection apparatus described in Japanese Patent Application Publication No. 2017-219318, when an image is generated by measuring the state of vibration using laser interferometry, noise is generated in the generated image due to thermal noise caused by the irregular movement of electrons in the conductor or vibration of the device. In this case, noise suppression processing is generally performed on the generated image to suppress the noise included in the image. However, in the defect inspection apparatus described in Japanese Patent Application Publication No. 2017-219318, when an image is generated by measuring the state of periodically changing vibration of an object being inspected (the object being inspected) that emits elastic waves, the noise state of the generated image sometimes varies according to the state of periodically changing vibration. In this case, it is considered that noise suppression of the generated image is insufficient to adequately suppress noise that varies according to the state of vibration. Therefore, it is desirable to effectively suppress noise even when periodic state changes are measured. Summary of the Invention

[0005] The present invention was made to solve the problems described above. One object of the present invention is to provide an inspection system, image processing method and defect inspection device that can effectively suppress noise when measuring periodic state changes.

[0006] To achieve the aforementioned objective, the inspection system of the first aspect of the present invention includes: an excitation unit for exciting elastic waves onto an inspection object; a measurement unit for measuring the vibration state of the inspection object excited by the excitation unit; and an image processing unit configured to suppress noise in a complex image based on the measurement results of the measurement unit, wherein each pixel in the complex image is represented by a complex number representing a periodic change in the vibration state of the inspection object, and the image processing unit obtains the similarity between pixels included in an object image region defined in the complex image and pixels included in a plurality of reference image regions defined separately from the object image region in the complex image by comparing the complex numbers representing each pixel with each other, and performs noise suppression processing of the object image region using a weight based on the obtained similarity.

[0007] The image processing method of the second aspect of the present invention includes the following steps: acquiring a complex image in which each pixel is represented by a complex number representing a periodic state change; acquiring an object image region defined in the acquired complex image, and a plurality of reference image regions defined separately from the object image region in the complex image; acquiring the similarity between pixels included in the acquired object image region and pixels included in the plurality of reference image regions by comparing the complex numbers representing each pixel with each other; and performing noise suppression processing on the object image region using a weight based on the acquired similarity.

[0008] The defect inspection apparatus of the third aspect of the present invention includes: an excitation unit for exciting an elastic wave onto an inspection object; an irradiation unit for irradiating the inspection object on which the elastic wave has been excited by the excitation unit with a laser; a measurement unit for interfering a reflected laser, which is a laser irradiated by the irradiation unit and reflected from the inspection object, with a reference laser, which is a laser irradiated by the irradiation unit, and for photographing and measuring the interference light after the reflected laser and the reference laser interfere, thereby measuring the vibration state of the inspection object on which the elastic wave has been excited by the excitation unit; and a control unit configured to suppress noise in a complex image for defect inspection, which is a complex number representing each pixel representing a periodic change in the vibration state of the inspection object, based on the measurement result of the measurement unit. The control unit obtains the similarity between pixels included in an object image region defined in the complex image and pixels included in a plurality of reference image regions defined separately from the object image region in the complex image by comparing the complex numbers representing each pixel with each other, and performs noise suppression processing of the object image region using a weight based on the obtained similarity.

[0009] In the inspection system of the first aspect, the image processing method of the second aspect, and the defect inspection apparatus of the third aspect, the similarity between pixels included in an object image region defined in a complex image and pixels included in multiple reference image regions defined separately from the object image region in the complex image is obtained by comparing complex numbers representing each pixel with each other. Then, noise suppression processing of the object image region is performed using weights based on the obtained similarity. Thus, by performing noise suppression processing of the object image region defined in the complex image, noise in the complex image can be suppressed. Therefore, since the complex image, in which each pixel is represented by a complex number representing a periodic state change, reflects the periodic state change of vibration, unlike the case where noise suppression processing is performed on an image represented by real values, noise can be effectively suppressed even when the noise state changes according to the state of vibration by suppressing noise in the complex image. As a result, noise can also be effectively suppressed when periodic state changes are measured. Attached Figure Description

[0010] Figure 1 This is a diagram used to illustrate the structure of the inspection system based on the first embodiment.

[0011] Figure 2 This is a schematic diagram showing an example of a vibration state image based on the first embodiment.

[0012] Figure 3 This is a diagram used to illustrate the timing of measurements based on the first embodiment.

[0013] Figure 4 It is a diagram used to illustrate the measurement of interference light, as well as the acquisition and correction of complex images.

[0014] Figure 5 It is a diagram used to illustrate the extraction of object image regions from complex images and the reference image regions.

[0015] Figure 6 It is a diagram representing the noise distribution on the complex plane.

[0016] Figure 7 It is a diagram used to illustrate an approximation of the noise distribution at a signal point.

[0017] Figure 8 This is a schematic diagram showing the display unit based on the first embodiment.

[0018] Figure 9 This is a flowchart used to illustrate the image processing method based on the first embodiment.

[0019] Figure 10This is a block diagram showing the structure of the inspection system based on the second embodiment.

[0020] Figure 11 It is a graph representing the vibration state image where low-frequency components have been removed and noise has been suppressed.

[0021] Figure 12 This is a block diagram showing the structure of the inspection system based on the third embodiment.

[0022] Figure 13 This diagram illustrates the acquisition of the average values ​​of the pixels of the object image region and the reference image region based on the third embodiment. Detailed Implementation

[0023] Hereinafter, embodiments embodying the present invention will be described with reference to the accompanying drawings.

[0024] [First Implementation Method]

[0025] (Inspect the overall structure of the system)

[0026] Reference Figures 1 to 8 The inspection system 100 based on the first embodiment of the present invention will be described.

[0027] like Figure 1 As shown, the inspection system 100 based on the first embodiment includes: a vibrator 1, an irradiation unit 2, a speckle shear interferometer 3, a control unit 4, a signal generator 5, a display unit 6, and an operation unit 7. Furthermore, the vibrator 1 is an example of the "excitation unit" in the claims, and the speckle shear interferometer 3 is an example of the "measuring unit" in the claims. Additionally, the control unit 4 is an example of the "image processing unit" in the claims.

[0028] The vibrator 1 and the irradiation unit 2 are connected to the signal generator 5 via cables. In addition, the speckle shear interferometer 3, the signal generator 5, the display unit 6, and the operation unit 7 are connected to the control unit 4 via cables.

[0029] Vibrator 1 excites elastic waves in the measurement area Pa of the object being inspected, P. Specifically, vibrator 1 is configured to contact the object being inspected, P, and converts the AC signal from signal generator 5 into mechanical vibration, thereby generating excitation vibration to excite elastic waves in the measurement area Pa. Furthermore, the frequency of the elastic wave generated by vibrator 1 on the object being inspected is equal to the frequency of the vibration (excitation vibration) of vibrator 1. That is, the frequency of the AC signal input to vibrator 1 is equal to the frequency of the elastic wave generated by vibrator 1 on the object being inspected, P.

[0030] The irradiation unit 2 irradiates the measurement area Pa of the inspection object P, which has been subjected to elastic waves excited by the vibrator 1, with laser light. The irradiation unit 2 includes a laser source (not shown). The laser light emitted from the laser source is extended through the illumination lens 21 to irradiate the entire measurement area Pa on the surface of the inspection object P. In addition, the irradiation unit 2 irradiates the laser at predetermined times based on an electrical signal from the signal generator 5. That is, the irradiation unit 2 irradiates the inspection object P with laser light in correspondence with the elastic waves excited by the vibrator 1. The laser source is, for example, a laser diode, irradiating a laser with a wavelength of 785 nm (near-infrared light).

[0031] In the first embodiment, the speckle shear interferometer 3 measures the vibration state in the measurement region Pa of the inspection object P, which is excited by the vibrator 1 to produce elastic waves. Specifically, the speckle shear interferometer 3 uses laser interferometry to interfere with the laser light reflected from the irradiation unit 2 and the measurement region Pa of the inspection object P, and then captures the interfered laser light (interference light). More specifically, the speckle shear interferometer 3 uses laser interferometry to interfere with the reflected laser light (reflected laser and reference laser) at two different points in the measurement region Pa of the inspection object P excited by the vibrator 1. In addition, the speckle shear interferometer 3 includes a beam splitter 31, a first reflecting mirror 32a, a second reflecting mirror 32b, a condenser lens 33, an image sensor 34, and a phase shifter 35.

[0032] Beam splitter 31 includes a semi-reflecting mirror. Beam splitter 31 is positioned at the location where the reflected laser light is incident on the measurement area Pa of the object being inspected P. Furthermore, beam splitter 31 causes the incident laser light to... Figure 1 The light path, as shown by straight line L1, is reflected towards the first reflecting mirror 32a, and as... Figure 1 The light path, as shown by the dashed line L2, passes through towards the second reflector 32b. Additionally, the beam splitter 31 allows the laser light reflected from and incident on the first reflector 32a to pass through as... Figure 1 The light path, as shown by straight line L1, passes through to the image sensor 34 side, and the laser reflected by the second mirror 32b is like... Figure 1 The image is reflected toward the image sensor 34 side as shown by the dashed line L2.

[0033] The first reflecting mirror 32a is configured to be at a 45-degree angle relative to the reflecting surface of the beam splitter 31 in the optical path of the laser reflected by the beam splitter 31. The first reflecting mirror 32a causes the laser reflected by the beam splitter 31 to reflect towards the beam splitter 31.

[0034] The second reflector 32b is configured such that it is slightly tilted at an angle of 45 degrees relative to the reflecting surface of the beam splitter 31 in the optical path of the laser transmitted through the beam splitter 31. The second reflector 32b reflects the laser transmitted through the beam splitter 31 and incident towards the beam splitter 31 side.

[0035] A focusing lens 33 is positioned between the beam splitter 31 and the image sensor 34, allowing the laser light passing through the beam splitter 31 to pass through. Figure 1 The straight line L1 in the middle and the laser reflected by the beam splitter 31 ( Figure 1 The dashed line L2 in the image focuses the light.

[0036] Image sensor 34 captures an image of the interfered laser light. Image sensor 34 has multiple detection elements arranged in a planar shape, positioned relative to the laser light that is reflected in beam splitter 31, then reflected by first mirror 32a, and transmitted through beam splitter 31. Figure 1 The straight line L1 in the beam splitter 31, and the laser light reflected by the second mirror 32b and then reflected by the beam splitter 31 after passing through the beam splitter 31. Figure 1 The image sensor 34 includes, for example, a complementary metal oxide semiconductor (CMOS) image sensor or a charge-coupled device (CCD) image sensor. (The image is shown in the dashed line L2 in the image.)

[0037] The phase shifter 35 is positioned between the beam splitter 31 and the first reflector 32a, and is controlled by the control unit 4 to adjust the transmitted laser light ( Figure 1 The phase shift (displacement) of the straight line L1 in the laser beam. Specifically, the phase shifter 35 is configured to change the optical path length of the transmitted laser beam.

[0038] In the speckle shear interferometer 3, for example, the laser light reflected from the first reflecting mirror 32a at position Pa1 on the surface of the measurement region Pa is used. Figure 1 The straight line L1 in the figure and the laser light reflected by position Pa2 on the surface of the measurement area Pa and the second reflecting mirror 32b are ( Figure 1 The dashed lines L2 in the image sensor 34 interfere with each other and are incident on the same location (detected in the same element). Positions Pa1 and Pa2 are positions separated from each other by a small distance. The lasers reflected from the different positions (Pa1 and Pa2) in the various regions of the measurement area Pa (reflected laser and reference laser) are guided by the speckle shear interferometer 3 to form interference light and are incident on the image sensor 34.

[0039] The control unit 4 controls the various parts of the speckle shear interferometer 3 and the operation of the signal generator 5. The control unit 4 is, for example, a computer including a central processing unit (CPU). Furthermore, the control unit 4 includes a storage device for storing various parameters and programs. Details regarding the control of the control unit 4 will be described below.

[0040] The signal generator 5, based on the control performed by the control unit 4, outputs an AC signal for controlling the vibration of the vibrator 1 and the timing of laser irradiation of the irradiation unit 2. The signal generator 5 may include, for example, a field-programmable gate array (FPGA).

[0041] Display unit 6 displays an image generated by control unit 4. Specifically, display unit 6 displays a vibration state image E (described later) generated by control unit 4. Figure 2 The display unit 6 includes a liquid crystal display or an organic electroluminescent (EL) display, etc.

[0042] The operation unit 7 checks the input operations performed by the operator. The operation unit 7 is, for example, a clicking device such as a keyboard and mouse. In addition, the operation unit 7 outputs an operation signal based on the received input operation to the control unit 4.

[0043] The inspection object P is, for example, a coated steel sheet. Defect Q is an undesirable part of the measurement area Pa that occurs internally (on the surface), including cracks or peeling.

[0044] (Controlled by the control department)

[0045] like Figure 2 As shown, the control unit 4 generates a vibration state image E for inspecting defect Q based on the interference light measured by the speckle shear interferometer 3. The vibration state image E is a generated image of the vibration state of the elastic wave generated by the excitation of the measurement area Pa of the inspection object P, which can be visually identified. For example, the vibration state image E is an image that can identify the location of defect Q (defective area) included in the inspection object P by visually recognizing the discontinuity of the vibration state.

[0046] Furthermore, in the first embodiment, the control unit 4 performs noise suppression processing on the generated vibration state image E. Specifically, in the first embodiment, the control unit 4 generates a complex image D (refer to) for defect inspection based on the measurement results obtained by the speckle shear interferometer 3. Figure 4 In the complex image D, each pixel is represented by a complex number representing the periodic change in the vibration state of the object P in the measurement area Pa. Furthermore, the control unit 4 is configured to suppress noise in the vibration state image E by performing noise suppression processing on the generated complex image D. Additionally, in Figure 2 The diagram shows a vibration state image E, including noise without noise suppression processing. The control processing performed by the control unit 4 will be explained in detail below.

[0047] <Measurement of Interference Light Performed by the Control Department>

[0048] The control unit 4 measures the vibration state of the inspection object P by irradiating the measurement area Pa of the inspection object P, which emits elastic waves, with a laser. Specifically, the control unit 4 changes the phase of the laser using a phase shifter 35, and acquires (captures) an interference light image based on the intensity pattern of the interference light captured by each detection element of the image sensor 34 to measure the vibration state in the measurement area Pa of the inspection object P. The interference light image is an image capturing the speckle pattern of the interference light.

[0049] Specifically, the control unit 4 uses an actuator (not shown) to operate the phase shifter 35 disposed within the speckle shear interferometer 3, thereby changing the phase of the transmitted laser. This results in a change in the phase difference between, for example, the laser reflected at position Pa1 and the laser reflected at position Pa2. Multiple detection elements of the image sensor 34 detect the intensity of the interference light obtained from the interference of the two lasers.

[0050] In addition, such as Figure 3 and Figure 4 As shown, the control unit 4 controls the vibration of the vibrator 1 and the timing of the laser irradiation by the irradiation unit 2 via the signal generator 5. While changing the phase shift, it captures images of the interference light at each detection element of the image sensor 34. The control unit 4 changes the phase shift by λ / 4 each time, capturing a total of 37 interference light images at each phase shift (0, λ / 4, λ / 2, 3λ / 4) at timing t (t = 0 to 7) when the phase difference of the elastic wave is T / 8, and five images taken before and after each phase shift (0, λ / 4, λ / 2, 3λ / 4) when the light is off. Here, λ is the wavelength of the laser. Additionally, T is the period of the elastic wave generated by the excited object P.

[0051] Then, based on the captured interference light images, the control unit 4 calculates the light phase (the phase difference between the two optical paths when the phase shift is zero) Φ using equation (1), taking the brightness values ​​I0 to I3 of four images where the elastic wave phase timing t (t = 0 to 7) is the same and the laser phase shift differs by λ / 4 each time. t .

[0052] Φ t =-arctan{(I3-I1) / (I2-I0)}…(1)

[0053] In addition, the control unit 4 uses the least squares method to determine the optical phase Φ t By performing a sine wave approximation, the approximation coefficients A, φ, and C in equation (2) are obtained.

[0054] Φ t=Acos(φ+tπ / 4)+C=Be×p(tπ / 4)+C…(2)

[0055] Where B is the complex amplitude, as expressed in equation (3).

[0056] B=Ae×p(iφ): Complex amplitude…(3)

[0057] Furthermore, the steps for determining the phase shift or the timing of laser irradiation are not limited to this. In this case, the calculation formula becomes different from equations (1) to (3).

[0058] Furthermore, the complex amplitude represented by equation (3) can be expressed as I(x) = a(x) + ib(x) according to Euler's theorem. I(x) is a complex number representing the state of the phase of the interference light detected by the detection element in the image sensor 34 corresponding to the coordinate x of the measurement area Pa (the vibration state of the object under inspection P). Thus, in the first embodiment, the control unit 4 generates a complex image D based on the measurement result of the interference light captured by the image sensor 34 of the speckle shear interferometer 3. The complex image D has a plurality of pixels arranged in a planar shape in a manner corresponding to each detection element of the image sensor 34. Moreover, in the complex image D, each pixel is represented by the complex number I(x) obtained by equations (1) to (3). The complex number I(x) represents the measured vibration state (the phase and amplitude of the periodically changing optical path difference). In addition, the complex number I(x) represents the complex number of pixels at coordinate x of the complex image D corresponding to the coordinates of the measurement area Pa.

[0059] <Correction of the degree of interference>

[0060] Here, as Figure 4 As shown, the control unit 4 is configured to correct the complex image D based on the interference degree coh(x). Specifically, in the first embodiment, the control unit 4 is configured to acquire the measured interference degree coh(x) of the interfering light. Furthermore, the control unit 4 is configured to weight the complex numbers I(x) representing each pixel of the complex image D based on the acquired interference degree coh(x), thereby correcting the complex numbers I(x) representing each pixel. Here, coh(x) represents the interference degree at coordinate x.

[0061] Specifically, the control unit 4 measures the degree of interference coh(x) in each pixel of the complex image D based on the changes in brightness values ​​I0 to I3 of four images where the phase of the elastic wave is at the same timing t (t = 0 to 7) and the phase shift of the laser differs by λ / 4 each time. That is, when the shift amount is changed using the phase shifter 35, the degree of interference coh(x) is small in pixels where the changes in brightness values ​​I0 to I3 are small. On the other hand, when the shift amount is changed using the phase shifter 35, the degree of interference coh(x) is large in pixels where the changes in brightness values ​​I0 to I3 are large.

[0062] Then, the control unit 4 uses the following equation (4) to correct the interference degree coh(x) of each pixel of the complex image D.

[0063] [Formula 1]

[0064]

[0065] Furthermore, I'(x) is the corrected value of the pixel at coordinate x. Additionally, C n This is a standardized term used to set the sum of the weights to 1. Additionally, N... k It is the number of pixels (core size) around the pixel of interest (the pixel to be corrected) used for correction, γ c This is a parameter that adjusts the magnitude of the influence of the interference degree. According to Equation (4), the control unit 4 interpolates (corrects) pixels with small interference degree coh(x) by weighting the pixels with large interference degree coh(x) from the surrounding pixels. Furthermore, the control unit 4 performs correction of the interference degree coh(x) based on Equation (4) for all pixels of the complex image D.

[0066] <Noise Suppression Processing>

[0067] Furthermore, in the first embodiment, the control unit 4 is configured to perform noise suppression processing on the complex image D, whose interference level coh(x) has been corrected. In the following description, the complex image D and the complex number I(x) represent the values ​​after the correction based on the interference level coh(x).

[0068] like Figure 5 As shown, in the first embodiment, the control unit 4 acquires the object image region z(x) defined in the acquired complex image D. i ), and in the complex image D, with respect to the object image region z(x) i Multiple reference image regions z(x) are separately defined. j (j: 1~N). Specifically, the control unit 4 obtains the object image region z(x) by extracting a specified region from the complex image D. iThen, control unit 4 compares the complex image D with the object image region z(x). i Extract the image region z(x) separately from the object image region. i Multiple defined regions of the same size are used to obtain multiple (N) reference image regions z(x). j ).

[0069] Object image region z(x) i () includes coordinates x i The defined region, including the object image region z(x). i ) is based on coordinates x i The region centered at the center has 7×7 pixels. That is, the object image region z(x i ) is derived from the complex number I(x) i The pixel represented by z(x) is a set of 49 pixels centered at the pixel. Additionally, the object image region z(x) i The region z(x) is the area to be subjected to noise suppression processing. Furthermore, referring to the image region z(x)... j ) is the image region z(x) of the object. i N distinct regions surrounding the image region z(x). For example, referring to the image region z(x) j ) is based on coordinates x1 to x2. N Each of the regions centered on the image has 7×7 pixels. That is, referring to the image region z(x j Each of ) is composed of complex numbers I(x) j (complex number I(x1)~complex number I(x)) N The set of 49 pixels centered at the pixel represented by )). From the coordinate x i The defined region S(x) centered on i Extract the reference image region z(x) from (for example, a region comprising 21×21 pixels) j Furthermore, the reference image region z(x) can be extracted from the entire complex image D. j Additionally, referring to the image region z(x) j It can be compared with the object image region z(x) i They share a portion of pixels. Similarly, in multiple reference image regions z(x) j Different reference image regions z(x) j They may share pixels with each other.

[0070] Furthermore, the control unit 4 communicates with multiple reference image regions z(x) j The comparison of z(x) with respect to the object image region z(x) i Noise suppression processing is performed.

[0071] Specifically, in the first embodiment, the control unit 4 obtains the acquired object image region z(x) by comparing the complex numbers I(x) representing each pixel with each other. i The pixels included in ) and multiple reference image regions z(x) j The control unit 4 then obtains weights based on the obtained similarity of the pixels included in the image region z(x). Additionally, the control unit 4 performs non-local mean processing as a noise suppression process, which performs weighted analysis on multiple reference image regions z(x) based on the obtained weights. j The values ​​of each part of the image are averaged to suppress the image region z(x). i (noise).

[0072] <Non-local mean processing>

[0073] Here, the control unit 4 is configured to assign weights to the signal distribution (noise distribution) during non-local mean processing, thereby performing optimal noise suppression processing on the inspection system 100. The weights represent multiple reference image regions z(x) j Each of the object image regions z(x) in ) i The similarity between ).

[0074] like Figure 6 As shown, the noise included in the complex image D has a unique noise distribution. For example, when the complex image D does not contain noise, the phase and amplitude of the vibrations of each of the 7×7 pixels in a region comprising 7×7 pixels are approximately equal. Therefore, when the complex number I(x) of each extracted pixel is plotted on the complex plane, the plotted signal points cluster at approximately the same points. On the other hand, when the complex image D contains noise, when the complex number I(x) of each extracted pixel is plotted on the complex plane, the plotted signal points spread out according to a distribution with expansion (noise distribution).

[0075] Furthermore, the noise distribution expands significantly depending on the intensity of the detected signal. That is, the noise distribution increases according to the amplitude of the periodic change in the vibration state of the object P in the measurement area Pa. For example, in the portion of the vibration state image E where the vibration state is an antinode, the amplitude increases, thus the expansion of the noise distribution increases. Conversely, in the portion where the vibration state is a node, the amplitude decreases, thus the expansion of the noise distribution is smaller.

[0076] like Figure 7 As shown, in the first embodiment, the control unit 4 represents the object image region z(x) iThe noise distribution of signal points on the complex plane of each pixel's complex number I(x) is approximated by a two-dimensional normal distribution, which is biased towards the signal points on the complex plane relative to the origin based on the magnitude of the amplitude of the periodic change in the vibration state of the object under inspection P.

[0077] Specifically, when the true signal point in a pixel of the complex image D without noise is defined as u = a + ib, the control unit 4 approximates the distribution of the actual signal points (including the noise distribution) in the complex image D as following a normal distribution N(u|Σ). Considering the augmented matrix that expands by a factor of λ1 relative to the direction of angle θ (signal point direction) and by a factor of λ2 in the orthogonal direction, when the extent of expansion from the origin to the signal point direction (θ) is set to λ1 and the extent of expansion to the orthogonal direction orthogonal to the signal point direction is set to λ2, it is expressed as u = |u|cosθ + i|u|sinθ. Therefore, the covariance matrix Σ of the normal distribution N(u|Σ) is expressed as shown in the following equation (5).

[0078] [Formula 2]

[0079]

[0080] Furthermore, the control unit 4 determines λ1 and λ2 as shown in the following equation (6), and approximates the noise distribution in a two-dimensional normal distribution that expands according to the magnitude of the amplitude in the direction of the signal point.

[0081] [Formula 3]

[0082] λ1=n c +|u| γ n r

[0083] λ2=n c …(6)

[0084] Here, n c n is a parameter representing the magnitude of isotropically expanding noise independent of the signal. r γ is a parameter that represents the magnitude of noise dependent on the signal as it expands from the origin toward the signal point, while γ is a parameter that represents the degree of dependence on the amplitude of the signal point.

[0085] Furthermore, in the first embodiment, the control unit 4 is configured to obtain the object image region z(x) based on the approximate noise distribution by performing the following computational processing. i The pixels included in ) and the reference image region z(x) j The similarity of pixels included in the calculation is evaluated, and noise suppression processing is performed.

[0086] Specifically, in the suppressed object image region z(x) i In the case of noise, if the object image region z(x) is included... i Let u(x) be the region of true values ​​in pixels where there is no noise. i ), will be presumed to be in the object image region z(x) i The region in the pixel whose noise has been suppressed (the region after noise suppression) is set as u'(x). i As shown in equation (7), the best estimate is obtained by minimizing the expected loss after the fact.

[0087] [Formula 4]

[0088]

[0089] In addition, p(u(x) i )|z(x i )) is the observation z(x i u(x) at time i The conditional probability distribution of ). Furthermore, according to equation (7), if the loss function L is set as the squared error, then u'(x) i The best estimated value of u'(x) i ) opt It can be represented by equation (8).

[0090] [Formula 5]

[0091]

[0092] Moreover, equation (8) can be rewritten according to Bayes' theorem, as in equation (9).

[0093] [Formula 6]

[0094]

[0095] Here, p(z(x) i )|u(x i The true value is u(x). i z(x) at time i The conditional probability distribution of p(u(x)). i )) is u(x i The prior distribution of ).

[0096] Moreover, due to p(z(x) i )|u(x i )) and p(u(x i Since it is impossible to measure an accurate value, the control unit 4 acquires a representation of the value from the object image region z(x). i The reference image region z(x) is extracted from the area surrounding the image. jThe complex number I(x) of each pixel in ) j Furthermore, if p(u(x) is distributed beforehand... i If the distribution follows a uniform distribution, then u'(x) i It can be approximated as in equations (9) to (10).

[0097] [Formula 7]

[0098]

[0099]

[0100] Furthermore, as shown in equation (10), the control unit 4 is configured to replace u(x) as an unknown value. j Using multiple (N) reference image regions z(x) j The actual measured values ​​of each pixel in ) (complex number I(x) j To approximate u'(x) i ).

[0101] Furthermore, in the object image region z(x i Each of the n pixels included in ) is represented as z(x i，k (k: 1~n), and will refer to the image region z(x) j Each of the n pixels included in ) is represented as z(x j，k In the case of (k: 1~n), z(x) i，k )|z(x j，k ) is approximated as following a normal distribution N(z(xi, k )|Σ). Furthermore, p(z(x) i，k )|z(x j，k Let it be as shown in equation (11).

[0102] [Formula 8]

[0103]

[0104] Additionally, k represents the image region z(x) of the object. i ) and reference image region z(x) j The relative coordinates of each pixel. Within the object image region z(x) i ) and reference image region z(x) j In the case of 7×7 pixels (with n=49), k=1~49.

[0105] Through the computational processing described above, the control unit 4 approximates the noise distribution of the signal points as a normal distribution following the covariance matrix Σ shown in equation (5), and through the computational processing shown in equations (12) to (14), it approximates the noise distribution of the object image region z(x) as a normal distribution. i Noise suppression processing is performed.

[0106] [Formula 9]

[0107]

[0108]

[0109]

[0110] Furthermore, in equations (12) to (14), p(z(x) i，k )|z(x j，k )) represents z(x) as an object-based image region i ) and multiple reference image regions z(x) j The control unit 4 is configured to assign weights to the similarity scores of each element (x, z, x). i，k )|z(x j，k )) is used to refer to multiple reference image regions z(x) j The weights when averaging. Additionally, as shown in equation (13), in p(z(x) i，k )|z(x j，k In the image region z(x) of the object, i Each pixel (z(x)) i，k The complex number I(x) i，k The real part a(x) i，k ) and the imaginary part b(x) i，k ) and multiple reference image regions z(x) j Each pixel (z(x) j，k The complex number I(x) j，k The real part a(x) j，k ) and the imaginary part b(x) j，k The differences between each pair are used as the comparison results to calculate the similarity.

[0111] Control unit 4 uses the weights p(z(x) expressed by equation (13) i，k )|z(x j，k For multiple reference image regions z(x) j The values ​​of each element are weighted and averaged to obtain the region where noise is presumed to be suppressed (the noise-suppressed region), i.e., u'(x). i Control unit 4 acquires the acquired u'(x). i The object image region z(x) was suppressed as noise.i ), thus affecting the image region z(x) of the object. i Noise suppression processing is performed.

[0112] Furthermore, the control unit 4 is configured to sequentially extract (define) the target image region z(x) for the entire complex image D. i ), and sequentially extract the object image regions z(x) i The same noise suppression process is performed, thereby suppressing noise in the entire complex image D.

[0113] <Generation of Vibration State Images>

[0114] Moreover, such as Figure 8 As shown, in the first embodiment, the control unit 4 is configured to generate a vibration state image E based on a complex image D whose noise has been suppressed by performing noise suppression processing. This vibration state image E is capable of visually identifying the vibration state of the inspection object P in the measurement area Pa. For example, the control unit 4 generates a vibration state image E as a dynamic image by rotating the phase of the complex I(x) of each pixel in the noise-suppressed complex image D and converting the real part into a time-series image. Furthermore, the control unit 4 can acquire the amplitude or phase of the complex I(x) of each pixel in the noise-suppressed complex image D as real values, and use the amplitude or phase acquired as real values ​​as brightness values ​​to generate a vibration state image E as a static image. Then, the control unit 4 displays the vibration state image E on the display unit 6 so that discontinuous areas of the vibration state can be visually identified as defects Q (defective areas) generated inside the inspection object P. Furthermore, in Figure 8 In, it means from Figure 2 The vibration state image E suppresses the noise state.

[0115] (Parameter change)

[0116] Furthermore, in the first embodiment, the control unit 4 is configured to make the parameter n, which represents the extent of expansion of the distribution when approximating the noise distribution,... c Parameter n r And the parameter n in parameter γ c and parameter n r Change. Specifically, the operator performs a visual inspection of the vibration status image E displayed on the display unit 6, which is handled by the operation unit 7, and changes the parameter n. c and parameter n r The control unit 4 is configured to change the parameter n based on the input operation received by the operation unit 7. c and parameter n r This allows for adjustments to the expansion of the two-dimensional normal distribution when approximating the noise distribution in noise suppression processing.

[0117] Furthermore, in the first embodiment, the control unit 4 is configured to store the representation of the image region z(x) of the object. i ) and multiple reference image regions z(x) j The comparison result is obtained by comparing the complex numbers I(x) of the pixels included in each of the regions of the object image z(x). For example, as shown in equation (13), z(x) represents the object image region. i ) and multiple reference image regions z(x) j The difference between the real parts a(x) and the difference between the imaginary parts b(x) of the complex numbers I(x) of each pixel in the equation is obtained by changing the parameter n. c and parameter n r The control unit 4 is configured such that, during the initial processing of suppressing noise in the complex image D, the parameter n will remain unchanged. c and parameter n r The comparison results and other calculation results, which remain unchanged even under certain circumstances, are stored in the storage device.

[0118] Furthermore, in the first embodiment, the control unit 4 is configured such that, after performing one noise suppression process, the operation unit 7 receives the parameter n representing the degree of expansion of the distribution. c and parameter n r In the case of a changed operation, use the changed parameter n. c and parameter n r And the stored comparison results, after adjusting the expansion of the two-dimensional normal distribution, are used to approximate the noise distribution again, thereby obtaining the similarity again. The weight p(z(x) for noise suppression processing (non-local mean processing) is then obtained again using the above equation (13). i，k )|z(x j，k Then, use the obtained weights p(z(x)). i，k )|z(x j，k Noise suppression processing is performed again.

[0119] (Regarding defect inspection methods)

[0120] Next, refer to Figure 9 The image processing method performed by the inspection system 100 based on the first embodiment will be described. The image processing method of the first embodiment is executed by control processing performed by the control unit 4.

[0121] First, in step 401, based on the measurement results obtained by the speckle shear interferometer 3, a complex image D is acquired, wherein each pixel in the complex image D is represented by a complex number I(x) representing the periodic change of the vibration state of the object P in the measurement area Pa.

[0122] Next, in step 402, the object image region z(x) defined in the acquired complex image D is obtained. i ) and in the complex image D, the region z(x) of the object image. i Multiple reference image regions z(x) are separately defined. j ).

[0123] Next, in step 403, the acquired object image region z(x) is obtained by comparing the complex numbers I(x) representing each pixel with each other. i The pixels included in ) and multiple reference image regions z(x) j The similarity of pixels included in the image is calculated. Specifically, the object image region z(x) is obtained by comparing the differences between the real parts a(x) and imaginary parts b(x) of the complex numbers I(x). i ) and the reference image region z(x) j The similarity of z(x) to the object image region z(x), where the complex number I(x) represents the similarity of z(x) to the object image region z(x). i ) and referencing the image region z(x) j Each of the pixels included in the array has the same relative coordinates. Furthermore, j = 1 to N. Additionally, the acquired comparison results are stored.

[0124] Next, in step 404, based on the acquired similarity, a method for comparing multiple reference image regions z(x) is obtained. j The weights p(z(x) are used for weighting. i，k )|z(x j，k )).

[0125] Next, in step 405, based on the obtained weights p(z(x) i，k )|z(x j，k For the object image region z(x) i Noise suppression processing is performed. Specifically, this is done by using the obtained weights p(z(x)). i，k )|z(x j，k For multiple reference image regions z(x) j The weighted average is then applied to obtain the object image region z(x) that has undergone noise suppression processing. i (equation (12) u'(x) i )).

[0126] Next, in step 406, the object image region z(x) is executed. i Noise suppression processing is used to suppress noise in the complex image D.

[0127] Next, in step 407, a vibration state image E is generated based on a noise-suppressed complex image D. This vibration state image E is capable of visually identifying the vibration state of the object P in the measurement area Pa. The generated vibration state image E is then displayed on the display unit 6.

[0128] Next, in step 408, it is determined whether the operation unit 7 has accepted an input operation, the input operation being used to adjust the parameter n, which represents the extent of expansion of the distribution when approximating the noise distribution. c Parameter n r and parameter n in parameter γ c and parameter n r Make changes. After determining that the change was accepted, the change was made to parameter n. c and parameter n r If a change is made to the input, proceed to step 409. If it is not determined that the input used to modify parameter n has been accepted... c and parameter n r If a change is made to the input, the control process ends.

[0129] In step 409, based on the input operation received by the operation unit 7, the parameter n is... c and parameter n r Make the changes. Then, use the changed parameter n. c and parameter n r The similarity is recalculated based on the comparison results stored in step 403, and the process returns to step 404. Then, by re-executing steps 404 to 407, the process is repeated using the modified parameter n. c and parameter n r The noise suppression process of the complex image D is performed, and the vibration state image E is generated again and displayed on the display unit 6.

[0130] (Effects of the first implementation method)

[0131] The following effect is obtained in the first embodiment.

[0132] In the inspection system 100 of the first embodiment, as described above, the object image region z(x) delineated in the complex image D is obtained by comparing the complex numbers I(x) representing each pixel with each other. i The pixels included in the complex image D and the object image region z(x) i Multiple reference image regions z(x) are separately defined. j The similarity of pixels included in ) is then used. Then, a weight p(z(x) is applied based on the obtained similarity. i，k )|z(x j，k )), execute the image region z(x)i Noise suppression processing is performed on the object image region z(x) defined in the complex image D. i Noise suppression processing can suppress noise in the complex image D. Therefore, the complex image D, where each pixel is represented by a complex number I(x) indicating periodic state changes, reflects the periodic state changes of the vibration. Thus, unlike noise suppression processing performed on images represented by real values, noise suppression of the complex image D can effectively suppress noise even when the noise state changes according to the vibration state. Consequently, noise can also be effectively suppressed when measuring periodic state changes.

[0133] Furthermore, in the first embodiment, further effects can be obtained by configuring it as follows.

[0134] That is, in the first embodiment, as described above, the control unit 4 (image processing unit) is configured to represent the object image region z(x) i The noise distribution of signal points on the complex plane of each pixel's complex number I(x) is approximated by a two-dimensional normal distribution, and similarity is obtained based on the approximated distribution. This two-dimensional normal distribution is biased towards the direction of signal points on the complex plane relative to the origin, depending on the magnitude of the amplitude of the periodic change in the vibration state of the object P under inspection. If configured in this way, even if there is a deviation in the expansion of the noise distribution on the complex plane, approximating it with a two-dimensional normal distribution that expands towards the direction of signal points relative to the origin on the complex plane can approximate the noise distribution as corresponding to the actual biased expansion of noise. Therefore, even if there is a deviation in the expansion of the noise distribution on the complex plane, the similarity can be calculated more appropriately. As a result, when using similarity-based weights p(z(x)... i，k )|z(x j，k In noise processing, it can suppress noise more effectively.

[0135] Furthermore, in the first embodiment, as described above, the inspection system 100 includes an operation unit 7, which handles the processing of parameter n. c and parameter n r The operation to change the parameter n c and parameter n r This indicates the degree of expansion of the distribution when approximating the noise distribution. If configured this way, the operator can easily adjust the degree of expansion of the noise distribution. Therefore, the degree of noise suppression in the complex image D can be easily adjusted.

[0136] Furthermore, in the first embodiment, as described above, the control unit 4 (image processing unit) is configured to store the comparison result, that is, the image region representing the object z(x) i ) and multiple reference image regions z(x) j The comparison result is obtained by comparing the complex numbers I(x) of the pixels included in each of the distributions, and takes into account the parameter n used to represent the extent of expansion of the distribution. c and parameter n r In the case of a change operation, the parameter n, representing the extent of expansion of the distribution, is used after the change. c and parameter n r And the stored comparison results are used to obtain similarity. If configured in this way, then the parameter n, which represents the extent of the distribution expansion, is... c and parameter n r In the case of changes, for the image region z(x) representing the object i ) and multiple reference image regions z(x) j The value of the comparison result obtained by comparing the complex numbers I(x) of the pixels included in each of the distributions does not change. Therefore, by storing the comparison result calculated once, the value of the parameter n representing the extent of the distribution expansion is also unchanged. c and parameter n r With the changes made, the time required to re-perform noise suppression processing on the complex image D can be reduced.

[0137] Furthermore, in the first embodiment, as described above, the control unit 4 (image processing unit) is configured to perform non-local mean processing as noise suppression processing, wherein the non-local mean processing uses weights p(z(x) based on the acquired similarity. i，k )|z(x j，k For multiple reference image regions z(x) j Average the values ​​to suppress the image region z(x). i The noise is such that, due to the use of non-local means processing, i.e., using similarity-based weights p(z(x)). i，k )|z(x j，k To simply refer to the image region z(x) j The averaged nonlocal mean is applied to the obtained weights p(z(x)). i，k )|z(x j，k Compared to thresholding and other methods, this approach can more easily suppress the z(x) region of the object image. i (noise).

[0138] Furthermore, in the first embodiment, as described above, the inspection system 100 includes an irradiation unit 2 that irradiates a laser onto an inspection object P excited by an elastic wave from a vibrator 1 (excitation unit). A speckle shear interferometer 3 (measurement unit) is configured to cause interference between a reflected laser (which is the laser irradiated by the irradiation unit 2 and reflected from the inspection object P) and a reference laser (which is the laser irradiated by the irradiation unit 2), and to capture and measure the interference light after the reflected laser and the reference laser interfere. A control unit 4 (image processing unit) is configured to suppress noise in a complex image D generated based on the measurement results of the captured interference light. With this configuration, since the interference light of the measured laser changes periodically due to the vibration of the elastic wave, by representing the measurement results with a complex image D and suppressing noise, the noise component included in the measurement results can be effectively suppressed.

[0139] Furthermore, when a laser is irradiated onto the object P that emits elastic waves, the vibration of the elastic waves causes changes in the speckle pattern due to variations in the irradiation angle of the laser relative to the surface of the object P, or changes in the speckle pattern caused by displacement of the surface of the object P. This results in a biased and expanded distribution of noise at signal points on the complex plane representing the complex number I(x) of each pixel in the complex image D. Therefore, as in the first embodiment, the complex image D is generated based on the measurement results of the interference light, and the noise distribution of the signal points is approximated as a two-dimensional normal distribution that expands in the direction of the signal points according to the magnitude of the amplitude. This suppresses noise to correspond to the noise distribution characteristic of the complex image D generated based on the measurement results of the interference light.

[0140] Furthermore, in the first embodiment, as described above, the control unit 4 (image processing unit) is configured to acquire the interference degree coh(x) of the measured interference light, and is configured to weight the complex number I(x) representing each pixel of the complex image D based on the acquired interference degree coh(x), thereby correcting the complex number I(x) representing each pixel. Here, in pixels with a small interference degree coh(x), the measured vibration state becomes an inaccurate value. Therefore, when calculating the similarity in a state including the complex number I(x) representing pixels with a small interference degree coh(x), the accuracy of noise suppression processing decreases. In contrast, in the first embodiment, the control unit 4 is configured to acquire the interference degree coh(x) of the measured interference light, and is configured to weight the complex number I(x) representing each pixel of the complex image D based on the acquired interference degree coh(x), thereby correcting the complex number I(x) representing each pixel. If configured in this way, by adjusting the weight of the complex number I(x) representing pixels with small interference coh(x) to reduce it, the processing of using the complex number I(x) as an inaccurate value to suppress noise can be suppressed. As a result, even when the complex image D includes pixels with small interference coh(x), the reduction in the accuracy of noise suppression processing can be suppressed.

[0141] Furthermore, in the first embodiment, as described above, the control unit 4 (image processing unit) is configured to generate a vibration state image E based on a complex image D whose noise has been suppressed by performing noise suppression processing. This vibration state image E allows for visual identification of the vibration state of the object being inspected, P. With this configuration, noise in the generated vibration state image E can be suppressed. Therefore, by visually identifying the generated vibration state image E, the inspection operator can easily identify the distribution of the vibration state of the object being inspected, P. Thus, when inspecting for a defect Q in the object being inspected, by visually identifying the vibration state image E, the continuity of the vibration state of the object being inspected, P, can be identified, and therefore the defect Q in the object being inspected can be easily identified.

[0142] (Effects of the image processing method based on the first embodiment)

[0143] In the image processing method of the first embodiment, the following effect can be obtained.

[0144] In the image processing method of the first embodiment, by configuring it as described above, the object image region z(x) defined in the complex image D is obtained by comparing the complex numbers I(x) representing each pixel with each other. i The pixels included in the complex image D and the object image region z(x) i Multiple reference image regions z(x) are separately defined. jThe similarity of pixels included in ) is then used. Then, a weight p(z(x) is applied based on the obtained similarity. i，k )|z(x j，k To execute the object image region z(x) i Noise suppression processing is performed on the object image region z(x) defined in the complex image D. i The noise suppression processing can suppress noise in the complex image D. Therefore, the complex image D, where each pixel is represented by a complex number I(x) indicating periodic state changes, reflects the periodic state changes of vibration. Thus, unlike the case of performing noise suppression processing on an image represented by real values, by suppressing noise in the complex image D, noise can be effectively suppressed even when the noise state changes according to the vibration state. As a result, an image processing method can be provided that can effectively suppress noise even when measuring periodic state changes.

[0145] [Second Implementation]

[0146] Next, refer to Figure 10 and Figure 11 The structure of the inspection system 200 based on the second embodiment of the present invention will be described. In the second embodiment, it is configured to suppress the background component of the vibration state image E by removing low-frequency components. Furthermore, structures identical to those in the first embodiment are illustrated using the same symbols, and their descriptions are omitted.

[0147] like Figure 10 As shown, the inspection system 200 based on the second embodiment includes a control unit 204. Furthermore, the measurement of interference light performed by the speckle shear interferometer 3 based on the second embodiment is the same as in the first embodiment. Moreover, the control unit 204 generates a complex image D based on the measurement results obtained by the speckle shear interferometer 3, using the same control processing as the control unit 4 in the first embodiment. Furthermore, the control unit 204 is an example of the "image processing unit" mentioned in the claims.

[0148] Moreover, such as Figure 11 As shown, in the vibration state image E generated based on the complex image D, the vibration state changes drastically (high frequency) near the defect Q, but in the background area outside the defect Q, the vibration state changes more steadily (low frequency). Therefore, in the second embodiment, the control unit 204 is configured to generate a vibration state image E200 in which low-frequency components are removed and noise is suppressed.

[0149] Specifically, similar to the control unit 4 in the first embodiment, the control unit 204 obtains the object image region z(x) from the generated complex image D. i ) and multiple reference image regions z(x)j (j: 1~N). Furthermore, similar to the control unit 4 in the first embodiment, the control unit 204 suppresses the target image region z(x) i The noise is used to obtain the noise for multiple reference image regions z(x) j The weights p(z(x) are used for weighting. i，k )|z(x j，k (Refer to formula (13)).

[0150] Furthermore, the control unit 204 refers to multiple reference image regions z(x) j The control unit 204 performs a process to remove low-frequency components from the reference image area z(x). j Perform low-cutoff filter processing, such as using Fourier transform, as a process to remove low-frequency components.

[0151] Furthermore, in the second embodiment, the control unit 204 is configured to perform nonlocal mean processing as a noise suppression process, wherein the nonlocal mean processing uses weights p(z(x) based on the acquired similarity. i，k )|z(x j，k For multiple reference image regions z after removing low-frequency components lowcut (x j Average the values ​​to suppress the image region z(x). i (noise).

[0152] That is, the control unit 204 uses multiple reference image regions z(x) before removing low-frequency components. j ) and the object image region z(x) i The weights p(z(x) are obtained based on the similarity between them. i，k )|z(x j，k This is used to analyze multiple reference image regions z after removing low-frequency components. lowcut (x j The weighted average is then applied to obtain the object image region z(x) where noise is suppressed and low-frequency components are removed. i ) i.e. u lowcut '(x i u lowcut '(x i ) is by passing z lowcut (x j Substitute it into equation (12) and express it as in equation (15) below.

[0153] [Formula 10]

[0154]

[0155] Furthermore, the control unit 204 is based on the object image region z(x) where noise is suppressed and low-frequency components are removed. i The control unit 204 uses the same processing as in the first embodiment to generate a vibration state image E200, in which the low-frequency components have been removed and the noise has been suppressed, based on the complex image D. Furthermore, the other structures of the second embodiment are the same as in the first embodiment.

[0156] [Effects of the Second Embodiment]

[0157] In the second embodiment, the following effect can be obtained.

[0158] In the second embodiment, as described above, the control unit 204 (image processing unit) is configured to remove multiple reference image regions z(x) j The low-frequency components of ) are processed, and nonlocal mean processing is performed as a noise suppression process, which is achieved by using weights p(z(x) based on the acquired similarity. i，k )|z(x j，k For multiple reference image regions z after removing low-frequency components lowcut (x j Average the values ​​to suppress the image region z(x). i Noise from the image D is also present. Here, when low-frequency components are included as background components in the complex image D, and similarity is obtained and weights are calculated after prior removal of low-frequency components, accurate similarity cannot be obtained, thus reducing noise suppression effectiveness. Furthermore, when low-frequency components are removed after noise suppression processing, harmonic components (artifacts) that were not noticeable before removal due to non-local mean processing are enhanced. In contrast, in the second embodiment, the control unit 204 is configured to remove multiple reference image regions z(x) j The low-frequency components of ) are processed, and nonlocal mean processing is performed as a noise suppression process, which is achieved by using weights p(z(x) based on the acquired similarity. i，k )|z(x j，k For multiple reference image regions z after removing low-frequency components lowcut (x j Average the values ​​to suppress the image region z(x). i The noise. If configured in this way, then in the case that low-frequency components are included as background components in the complex image D, the weights p(z(x)) can be calculated based on similarity. i，k )|z(x j，kIn the case of not removing low-frequency components, but using the calculated weights p(z(x)... i，k )|z(x j，k The reference image region z, after averaging and removing low-frequency components. lowcut (x j The weights are then weighted and averaged. Therefore, when calculating the weights p(z(x)... i，k )|z(x j，k In the case of [the specific method], low-frequency components can be removed while suppressing the decrease in accuracy, and the enhancement of harmonic components caused by non-local region processing can be suppressed. As a result, when low-frequency components are included as background components in the complex image D, noise can be effectively suppressed while removing background components.

[0159] Furthermore, other effects based on the second embodiment are the same as those of the first embodiment.

[0160] [Third Implementation Method]

[0161] Next, refer to Figure 12 and Figure 13 The structure of the inspection system 300 based on the third embodiment of the present invention will be described. In this third embodiment, the system is configured such that when inspecting the object image region z(x)... i Each pixel of ) and the reference image region z(x) j When comparing individual pixels, the average value is used. Furthermore, for structures identical to those in the first embodiment, the same symbols are used in the illustrations, and their descriptions are omitted.

[0162] like Figure 12 As shown, the inspection system 300 based on the third embodiment includes a control unit 304. Furthermore, the measurement of interference light performed by the speckle shear interferometer 3 based on the third embodiment is the same as in the first embodiment. Moreover, the control unit 304 generates a complex image D based on the measurement results obtained by the speckle shear interferometer 3 through the same control processing as the control unit 4 in the first embodiment. Furthermore, the control unit 304 is an example of the "image processing unit" mentioned in the claims.

[0163] Furthermore, similar to the control unit 4 in the first embodiment, the control unit 304 obtains the object image region z(x) from the generated complex image D. i ) and multiple reference image regions z(x) j (j: 1~N).

[0164] Moreover, such as Figure 13 As shown, in the third embodiment, the control unit 304 is configured to control the image region z(x) representing the object. i Each pixel (z(x))i，k The complex number I(x) i，k The average value I avr (x i ) and represent the reference image region z(x) j Each pixel (z(x)) j，k The complex number I(x) j，k The average value I avr (x j The similarity is obtained by comparison.

[0165] Specifically, the control unit 304 calculates the z(x) region representing the image area of ​​the object. i The complex number I(x) includes n pixels. i，k The average value I avr (x i ). I avr (x i ) is represented as I avr (x i ) = a avr (x i )+ib avr (x i Similarly, the control unit 304 calculates the z(x) region representing the object image area. j The complex number I(x) includes n pixels in ) j，k The average value I avr (x j Average value I avr (x j ) is represented as I avr (x j ) = a avr (x j )+ib avr (x j Furthermore, k = 1 to n.

[0166] The control unit 304 calculates the average value I. avr (x i ) and average value I avr (x j The difference between z(x) and z(x) is used to determine the image region of the object. i ) and the reference image region z(x) j Similarity is obtained by comparison and by means of the average value I. avr (x i ) and average value I avr (x j To obtain the reference image region z(x) j The weights p(z(x) during averaging i )|z(x j That is, the control unit 304 uses aavr (x i ) and b avr (x i ) to replace a(x) in equation (15) i，k ) and b(x i，k Each of the ) and using a avr (x j ) and b avr (x j ) to replace a(x j，k ) and b(x j，k Each of the following is used to obtain the weights p(z(x)). i )|z(x j )).

[0167] That is, in the third embodiment, it is configured to refer to the image region z(x) j The control unit 304, unlike the control unit 4 in the first embodiment which performs averaging based on similarity weights for each pixel of the reference image region z(x), is configured to perform averaging based on similarity weights for each pixel of the reference image region z(x). j Perform a weighted average based on a similarity score.

[0168] Furthermore, the control unit 304 refers to the reference image area z(x) j The values ​​are weighted and averaged, and noise suppression is performed to obtain the object image region z(x). i ), to suppress noise in complex image D.

[0169] Furthermore, the control unit 304 is configured to store the object image region z(x) i The average value I avr (x i ), and referencing the image region z(x) j The average value I avr (x j Furthermore, the control unit 304 is configured to, when controlling parameter n c and parameter n r If changes have been made, use the changed parameter n. c and parameter n r and the stored average value I avr (x i ) and average value I avr (x j The weights are calculated using this method.

[0170] Furthermore, the other structures of the third embodiment are the same as those of the first embodiment.

[0171] [Effects of the Third Implementation]

[0172] In the third embodiment, the following effect can be obtained.

[0173] In the third embodiment, as described above, the control unit 304 (image processing unit) is configured to process the image region z(x) representing the object. i The complex number I(x) of each pixel of ) i，k The average value I avr (x i ) and represent the reference image region z(x) j The complex number I(x) of each pixel of ) j，k The average value I avr (x j The similarity is obtained by comparing the z(x) region of the object image with the z(x) region. If constructed in this way, the similarity is then compared with the z(x) region of the object image. i The individual pixels included in the image and multiple reference image regions z(x) j Compared to comparing individual pixels in a complex image D, this method reduces the number of steps required to calculate the comparison results. Therefore, it shortens the time required for computational processing to calculate similarity, and consequently, the time required to suppress noise in the complex image D.

[0174] Furthermore, other effects based on the third embodiment are the same as those of the first and second embodiments.

[0175] [Variation Example]

[0176] Furthermore, the embodiments disclosed herein should be considered illustrative in all respects and not restrictive. The scope of the invention is defined by the claims rather than by the description of the embodiments described above, and includes all modifications (variations) within the meaning and scope equivalent to the claims.

[0177] For example, in the first to the third embodiments, an inspection system 100 (200, 300) is shown in which the vibrator 1 (excitation unit), the irradiation unit 2, the speckle shear interferometer 3 (measuring unit), and the control unit 4, control unit 204, and control unit 304 (image processing unit) are separately provided; however, the present invention is not limited to this. In the present invention, the excitation unit, the irradiation unit, the measuring unit, and the control unit may also be integrally formed as a defect inspection device. In this case, the control unit of the defect inspection device has the same structure as the control unit 4, control unit 204, and control unit 304 (image processing unit) in the first to the third embodiments.

[0178] Furthermore, in the first to third embodiments, an example was shown where the noise distribution of the signal points was approximated by a two-dimensional normal distribution. However, the present invention is not limited to this. The two-dimensional normal distribution extends towards the signal point direction relative to the origin on the complex plane, depending on the magnitude of the amplitude of the periodic change in the vibration state of the object under inspection, P. In the present invention, the noise distribution of the signal points may also be approximated with a certain extension regardless of the magnitude of the amplitude of the periodic change in the vibration state. Alternatively, the noise distribution of the signal points may be approximated by a normal distribution that extends isotropically without deviation. Furthermore, the noise distribution of the signal points may also be approximated by extending not only in the direction of the signal points but also in orthogonal directions orthogonal to the direction of the signal points, depending on the magnitude of the amplitude of the periodic change in the vibration state.

[0179] Furthermore, in the first to the third embodiments, a parameter n is shown that represents the extent of expansion of the distribution when approximating the noise distribution. c and parameter n r Examples of variations are given, but the invention is not limited to these. For example, it can be configured to change the three parameters n. c n r And all of γ, can also be configured to only be able to change three parameters n. c n r And any one of γ. For example, the appropriate three parameters n can be preset in a way that corresponds to the material of the object being inspected, P. c n r And the value of γ. Additionally, it is assumed that the parameter n... c The optimal value varies depending on the photographic environment, such as lighting; on the other hand, the parameter n... r And the optimal value of parameter γ does not change due to the material of the object being inspected, P. In this case, it can also be configured such that, by performing a calibration operation beforehand, parameter n is set according to the object being inspected, P. r And under the state of parameter γ, change parameter n c At the same time, noise suppression is adjusted.

[0180] Furthermore, in the first to the third embodiments, an example is shown where the comparison result is stored, i.e., for the image region z(x) representing the object. i ) and multiple reference image regions z(x) j The comparison result is obtained by comparing the complex numbers I(x) of the pixels included in each of the components, but the present invention is not limited thereto. For example, the comparison result may not be stored, but rather the result may be changed each time the parameter n is changed. c and parameter n r The comparison results are recalculated at that time.

[0181] Furthermore, in the first to the third embodiments, an example is shown whereby the weighted reference image region z(x) is used. j Average the image region z(x) to perform the average. i This is an example of nonlocal mean processing for noise suppression, but the invention is not limited thereto. For example, the obtained weights p(z(x)) can also be used. i，k )|z(x j，k The reference image region z(x) is above the specified threshold. j The noise-suppressed object image region z(x) is calculated using this method. i Alternatively, the reference image region z(x) can also be obtained. j It is the squared average rather than a simple average.

[0182] Furthermore, in the second embodiment, an example is shown where multiple reference image regions z(x) are used before removing low-frequency components. j ) and the object image region z(x) i The weights p(z(x) are obtained based on the similarity between them. i，k )|z(x j，k For multiple reference image regions z after removing low-frequency components lowcut (x j The values ​​are weighted and averaged to obtain the object image region z(x) where noise is suppressed and low-frequency components are removed. i However, the present invention is not limited thereto. For example, after removing low-frequency components, the weight p(z(x) can be obtained based on similarity. i，k )|z(x j，k And for the reference image area z lowcut (x j Weights are then applied and averaged. Alternatively, weights p(z(x)) can be used. i，k )|z(x j，k For multiple reference image regions z(x) j After weighting and averaging, low-frequency components are removed.

[0183] Furthermore, in the third embodiment, the acquisition of the object image region z(x) is shown. i ) and reference image region z(x) j The average value of each pixel (I) avr (x i ) and I avr (x j Examples are given, but the invention is not limited thereto. For example, the median or most frequent value may be obtained instead of the average.

[0184] Furthermore, in the first to third embodiments, an example was shown of measuring the interference light, which is an interferometric laser beam, using a speckle shear interferometer 3; however, the present invention is not limited thereto. For example, a Michelson interferometer or similar device can also be used to measure the interference light. Additionally, the vibration state of the outer surface can be measured using optical measurement methods such as digital image correlation or moiré sampling. Furthermore, a time-of-flight (ToF) camera can also be used to measure the vibration state of the object P under inspection.

[0185] Furthermore, in the first to third embodiments, an example was shown where each pixel of the entire complex image D was corrected according to the interference degree coh(x), but the present invention is not limited thereto. For example, pixels with small interference degree coh(x) may be extracted only from the complex image D and corrected accordingly.

[0186] Furthermore, in the first to third embodiments, an example was shown of generating a vibration state image E capable of visually identifying the vibration state of the inspection object P in the measurement area Pa; however, the present invention is not limited thereto. For example, the area presumed to be a defect Q can also be detected by detecting the continuity of vibration. In this case, an image or coordinate values ​​that can identify the area presumed to be a defect Q can also be output.

[0187] Furthermore, in the first to the third embodiments, an example was shown where a vibrator 1 was used as an excitation unit to generate elastic waves in the object under inspection P, but the present invention is not limited thereto. For example, elastic waves can also be generated by a loudspeaker that emits sound.

[0188] [form]

[0189] Those skilled in the art will understand that the exemplary embodiments described above are specific examples of the following forms.

[0190] (Project 1)

[0191] An inspection system, comprising:

[0192] The excitation section generates elastic waves on the object under inspection.

[0193] The measuring unit measures the vibration state of the object under inspection after the elastic wave is excited by the excitation unit; and

[0194] The image processing unit is configured to suppress noise in a complex image based on the measurement results from the measurement unit, wherein each pixel in the complex image is represented by a complex number representing a periodic change in the vibration state of the object under inspection.

[0195] The image processing unit

[0196] By comparing the complex numbers representing each pixel with each other, the similarity between pixels included in the object image region defined in the complex image and pixels included in multiple reference image regions defined separately from the object image region in the complex image is obtained.

[0197] Noise suppression processing of the object image region is performed using weights based on the acquired similarity.

[0198] (Project 2)

[0199] According to the inspection system described in Project 1, the image processing unit is configured to approximate the noise distribution of signal points on a complex plane representing each pixel of the object image region in a manner that follows a two-dimensional normal distribution, and to obtain the similarity based on the approximate distribution, wherein the two-dimensional normal distribution is biased towards the direction of the signal points on the complex plane relative to the origin based on the magnitude of the amplitude of the periodic change in the vibration state of the inspected object.

[0200] (Project 3)

[0201] The inspection system according to Project 2 also includes an operation unit that accepts operations to change parameters, the parameters representing the extent of expansion of the distribution when approximating the noise distribution.

[0202] (Project 4)

[0203] According to the inspection system described in Project 3, the image processing unit is configured to store comparison results, i.e., comparison results obtained by comparing the plurality of pixels included in each of the plurality of reference image regions with each other, and to obtain the similarity using the changed parameter representing the extent of expansion of the distribution and the stored comparison results when an operation is received to change the parameter representing the extent of expansion of the distribution.

[0204] (Project 5)

[0205] According to any one of Projects 1 to 4, in the inspection system, the image processing unit is configured to perform non-local mean processing as the noise suppression processing, wherein the non-local mean processing suppresses noise in the object image region by averaging the plurality of reference image regions based on the acquired similarity weights.

[0206] (Project 6)

[0207] According to the inspection system described in Project 5, the image processing unit is configured to remove low-frequency components from the plurality of reference image regions and perform non-local mean processing as the noise suppression processing. The non-local mean processing suppresses noise in the object image region by averaging the plurality of reference image regions after removing low-frequency components using weights based on the acquired similarity.

[0208] (Project 7)

[0209] According to any one of items 1 to 6, in the inspection system, the image processing unit is configured to obtain the similarity by comparing the average of the complex numbers representing each pixel of the object image region with the average of the complex numbers representing each pixel of the reference image region.

[0210] (Project 8)

[0211] The inspection system according to any one of items 1 to 7 further includes an irradiation unit.

[0212] The irradiation unit irradiates the inspection object with laser light, which has generated the elastic wave by the excitation unit.

[0213] The measuring unit is configured to cause interference between a reflected laser, which is the laser irradiated by the irradiation unit and reflected from the object under inspection, and a reference laser, which is the laser irradiated by the irradiation unit, and to photograph and measure the interference light after the reflected laser and the reference laser interfere.

[0214] The image processing unit is configured to suppress noise in the complex image generated based on the measurement results of the captured interference light.

[0215] (Project 9)

[0216] According to the inspection system described in Project 8, the image processing unit is configured to acquire the interference degree of the measured interference light, and is configured to weight the complex numbers representing each pixel of the complex image according to the acquired interference degree, thereby correcting the complex numbers representing each pixel.

[0217] (Project 10)

[0218] According to any one of Items 1 to 9, the inspection system wherein the image processing unit is configured to generate a vibration state image capable of visually identifying the vibration state of the inspected object based on the complex images whose noise has been suppressed by performing the noise suppression processing.

[0219] (Project 11)

[0220] An image processing method includes the following steps:

[0221] Obtain a complex image in which each pixel is represented by a complex number indicating a periodic state change;

[0222] Obtain the object image region defined in the acquired complex images, and a plurality of reference image regions defined separately from the object image region in the complex images;

[0223] The similarity between pixels in the acquired object image region and pixels in the plurality of reference image regions is obtained by comparing complex numbers representing each pixel with each other; and

[0224] Noise suppression processing of the object image region is performed using weights based on the acquired similarity.

[0225] (Project 12)

[0226] A defect inspection device, comprising:

[0227] The excitation section generates elastic waves on the object under inspection.

[0228] The irradiation unit irradiates the inspection object, which has been excited by the elastic wave by the excitation unit, with a laser.

[0229] The measuring unit causes interference between a reflected laser (which is the laser irradiated by the irradiation unit and reflected from the object under inspection) and a reference laser (which is the laser irradiated by the irradiation unit), and then photographs and measures the interference light after the reflected laser and the reference laser interfere, thereby measuring the vibration state of the object under inspection where the elastic wave is excited by the excitation unit; and

[0230] The control unit is configured to suppress noise in a complex image for defect inspection, where each pixel is represented by a complex number representing the periodic change in the vibration state of the object under inspection, based on the measurement results of the measurement unit.

[0231] The control unit

[0232] By comparing the complex numbers representing each pixel with each other, the similarity between pixels included in the object image region defined in the complex image and pixels included in multiple reference image regions defined separately from the object image region in the complex image is obtained.

[0233] Noise suppression processing of the object image region is performed using weights based on the acquired similarity.

Claims

1. An inspection system, wherein, include: The excitation section generates elastic waves on the object under inspection. The measuring unit measures the vibration state of the object under inspection, which has been excited by the elastic wave by the excitation unit. as well as The image processing unit is configured to suppress noise in a complex image based on the measurement results from the measurement unit, wherein each pixel in the complex image is represented by a complex number representing a periodic change in the vibration state of the object under inspection. The image processing unit By comparing the complex numbers representing each pixel with each other, the similarity between pixels included in the object image region defined in the complex image and pixels included in multiple reference image regions defined separately from the object image region in the complex image is obtained. Noise suppression processing of the object image region is performed using weights based on the acquired similarity, and The image processing unit is configured to approximate the noise distribution of signal points on a complex plane representing each pixel of the object image region by distributing them according to a two-dimensional normal distribution that is biased and extended on the complex plane, and to obtain the similarity based on the approximate distribution.

2. The inspection system according to claim 1, wherein, The image processing unit is configured to approximate the noise distribution of the signal points on the complex plane representing each pixel of the object image region in a manner that follows a two-dimensional normal distribution, and to obtain the similarity based on the approximate distribution, wherein the two-dimensional normal distribution is biased towards the signal points on the complex plane relative to the origin based on the magnitude of the amplitude of the periodic change in the vibration state of the object under inspection.

3. The inspection system according to claim 2, wherein, It also includes the operating unit, The operation unit accepts operations to change parameters, which represent the degree of expansion of the distribution when approximating the noise distribution.

4. The inspection system according to claim 3, wherein, The image processing unit is configured to store comparison results, namely, comparison results obtained by comparing the complex number of pixels included in each of the plurality of reference image regions to each other, and to obtain the similarity using the modified parameter representing the extent of expansion of the distribution and the stored comparison results when an operation is received to change the parameter representing the extent of expansion of the distribution.

5. The inspection system according to claim 1, wherein, The image processing unit is configured to perform nonlocal mean processing as the noise suppression processing. The nonlocal mean processing suppresses noise in the object image region by averaging the plurality of reference image regions based on the weights of the acquired similarity.

6. The inspection system according to claim 5, wherein, The image processing unit is configured to remove low-frequency components from the plurality of reference image regions and perform non-local mean processing as the noise suppression processing. The non-local mean processing suppresses noise in the object image region by averaging the plurality of reference image regions after removing low-frequency components using weights based on the acquired similarity.

7. The inspection system according to claim 1, wherein, The image processing unit is configured to obtain the similarity by comparing the average of the complex numbers representing each pixel of the object image region with the average of the complex numbers representing each pixel of the reference image region.

8. The inspection system according to claim 1, wherein, It also includes an irradiation section. The irradiation unit irradiates the inspection object, which has been excited by the excitation unit with a laser, with a laser beam. The measuring unit is configured to cause interference between a reflected laser, which is the laser irradiated by the irradiation unit and reflected from the object under inspection, and a reference laser, which is the laser irradiated by the irradiation unit, and to photograph and measure the interference light after the reflected laser and the reference laser interfere. The image processing unit is configured to suppress noise in the complex image generated based on the measurement results of the captured interference light.

9. The inspection system according to claim 8, wherein, The image processing unit is configured to acquire the interference degree of the measured interference light, and is configured to weight the complex numbers representing each pixel of the complex image according to the acquired interference degree, thereby correcting the complex numbers representing each pixel.

10. The inspection system according to claim 1, wherein, The image processing unit is configured to generate a vibration state image capable of visually identifying the vibration state of the object under inspection, based on the complex image in which noise has been suppressed by performing the noise suppression processing.

11. An image processing method, wherein, Includes the following steps: Obtain a complex image in which each pixel is represented by a complex number representing a periodic state change; Obtain the object image region defined in the acquired complex images, and a plurality of reference image regions defined separately from the object image region in the complex images; The similarity between pixels in the acquired object image region and pixels in the plurality of reference image regions is obtained by comparing the complex numbers representing each pixel with each other. as well as Noise suppression processing of the object image region is performed using weights based on the acquired similarity. The step of obtaining the similarity includes approximating the noise distribution of signal points on the complex plane representing each pixel of the object image region in a manner that follows a two-dimensional normal distribution that is biased and extended on the complex plane, and obtaining the similarity based on the approximated distribution.

12. A defect inspection device, wherein, include: The excitation section generates elastic waves on the object under inspection. The irradiation unit irradiates the inspection object, which has been excited by the elastic wave by the excitation unit, with a laser. The measuring unit causes interference between a reflected laser (which is the laser irradiated by the irradiation unit and reflected from the object under inspection) and a reference laser (which is the laser irradiated by the irradiation unit), and then photographs and measures the interference light after the reflected laser and the reference laser interfere, thereby measuring the vibration state of the object under inspection where the elastic wave is excited by the excitation unit; and The control unit is configured to suppress noise in a complex image for defect inspection, where each pixel is represented by a complex number representing the periodic change in the vibration state of the object under inspection, based on the measurement results of the measurement unit. The control unit By comparing the complex numbers representing each pixel with each other, the similarity between pixels included in the object image region defined in the complex image and pixels included in multiple reference image regions defined separately from the object image region in the complex image is obtained. Noise suppression processing of the object image region is performed using weights based on the acquired similarity. The control unit is configured to approximate the noise distribution of signal points on a complex plane representing each pixel of the object image region by distributing them according to a two-dimensional normal distribution that is biased and extended on the complex plane, and to obtain the similarity based on the approximate distribution.

Images