Defect inspection device, method of operating the defect inspection device, and operating program of the defect inspection device

Abstract

This invention provides a defect inspection device, an operating method for the defect inspection device, and an operating program for the defect inspection device, which enable highly accurate inspection of foreign matter within the flow path in a short time for defect inspection of flow path devices. [Solution] The defect inspection apparatus is a defect inspection apparatus for inspecting defects in a flow channel device having a flow channel, and comprises a processor and an imaging apparatus for imaging the flow channel device, which includes a first imaging apparatus for capturing a first image and a second imaging apparatus having an optical system with a shallower depth of field than the first imaging apparatus and capturing a second image with a higher magnification than the first image, and the processor performs a first determination to detect a candidate foreign object located in a position overlapping with the flow channel based on the first image which has a wider imaging range than the second image, and performs a second determination to determine whether or not the candidate exists in the flow channel in the thickness direction of the flow channel device based on the second image which is in focus on the flow channel.

Description

Technical Field

[0001] The technology of the present disclosure relates to a defect inspection device, a method for operating the defect inspection device, and an operation program for the defect inspection device.

Background Art

[0002] As an example, there is known a flow path device having a flow path through which a specimen of a liquid derived from a living body flows, and for optically measuring the specimen. The flow path device is also called a micro flow path chip, a micro flow chip, etc. because of its small size or narrow flow path. The flow path device is manufactured, for example, by forming grooves constituting the flow path in a flat transparent plastic.

[0003] Patent Document 1 describes an image inspection device for inspecting defects during the manufacture of a flow path device. The image inspection device images the entire range of the flow path device, and based on the captured image, inspects the presence or absence of defects including foreign matters such as chips, and determines the flow path device in which a defect is detected as a defective product.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] As a particularly important inspection item in the defect inspection of a flow path device, there is an inspection item called in-flow-path foreign matter inspection for determining the presence or absence of foreign matters that exist in the flow path and block the flow path. This is because if there are foreign matters in the flow path, it may affect the measurement of the specimen.

[0006] In the image inspection apparatus described in Patent Document 1, an overall image is acquired by capturing the entire range of the flow channel device from a direction that views the flow channel device from above, and inspection is performed based on the overall image. However, with such an overall image alone, it was sometimes not possible to accurately determine whether or not foreign objects visible in the overall image were present in the flow channel. Many flow channel devices are thin, and the flow channels are also narrow. Therefore, it is necessary to accurately determine whether or not foreign objects are present in the flow channel in the thickness direction of the flow channel device, but the overall image has a wide range of focus in the thickness direction of the flow channel device, making it difficult to determine whether or not foreign objects are present in the flow channel.

[0007] Therefore, it is conceivable to improve the accuracy of determining whether or not foreign matter is present in the channel by inspecting the entire channel device over an extended period of time using a microscope or similar device. However, inspecting the entire range of the channel device with a microscope or similar device is too time-consuming and therefore difficult to implement in practice. Thus, there has been a need to achieve more accurate inspection of foreign matter in the channel in a shorter amount of time.

[0008] The technology disclosed herein provides a defect inspection device, an operating method for the defect inspection device, and an operating program for the defect inspection device, which enable highly accurate inspection of foreign matter in the flow path in a short time for defect inspection of flow path devices. [Means for solving the problem]

[0009] The defect inspection apparatus of the technology disclosed herein is a defect inspection apparatus for inspecting defects in a flow channel device having a flow channel, comprising a processor and an imaging apparatus for imaging the flow channel device, comprising a first imaging apparatus for capturing a first image and a second imaging apparatus having an optical system with a shallower depth of field than the first imaging apparatus and capturing a second image with a higher magnification than the first image, wherein the processor performs a first determination to detect a candidate foreign object located in a position overlapping with the flow channel based on the first image which has a wider imaging range than the second image, and performs a second determination to determine whether or not the candidate exists in the flow channel in the thickness direction of the flow channel device based on the second image which is in focus on the flow channel.

[0010] The thickness of the channels formed in the flow channel device may be on the order of micrometers.

[0011] The thickness of the channel formed in the flow channel device may be 100 μm or less.

[0012] The flow channel device may include a transparent plate-like member having a groove-forming surface on which grooves serving as flow channels are formed, and a transparent film that seals the grooves.

[0013] The film thickness may be on the order of micrometers.

[0014] The film thickness may be 200 μm or less.

[0015] The film may also be bonded to the plate-shaped member by welding.

[0016] The fluid channel device may be a fluid channel device that moves the liquid in the channel by electrophoresis.

[0017] The second imaging device may be a digital microscope.

[0018] The processor may perform a morphological determination based on the second image to determine whether the candidate shape should be evaluated as a defect.

[0019] The processor may perform morphological determination only on candidates that were determined to be within the flow path in the second determination.

[0020] The processor may perform morphological determination using a machine learning model.

[0021] The processor may perform the first determination based on the first image based on a binarized image obtained by binarizing the pixel values.

[0022] In the flow path of the flow path device, when there is a circular area, the processor may convert the circular area of the first image into a strip area that extends linearly by polar coordinate transformation, and detect candidates from the strip area.

[0023] The operation method of the defect inspection device according to the technology of the present disclosure is an operation method of a defect inspection device for inspecting defects of a flow path device having a flow path, which includes obtaining a first image of the flow path device captured by a first imaging device, and obtaining a second image of the flow path device captured by a second imaging device having an optical system with a shallower depth of field than the first imaging device, where the second image has a higher magnification than the first image. Based on the first image with a wider imaging range than the second image, a first determination is executed to detect candidates for foreign matter at positions overlapping the flow path, and based on the second image focused on the flow path, a second determination is executed to determine whether the candidates exist within the flow path in the thickness direction of the flow path device.

[0024] The operation program of the defect inspection device according to the technology of the present disclosure is an operation program of a defect inspection device for inspecting defects of a flow path device having a flow path, obtaining a first image of the flow path device captured by a first imaging device, obtaining a second image of the flow path device captured by a second imaging device having an optical system with a shallower depth of field than the first imaging device, where the second image has a higher magnification than the first image, executing a first determination to detect candidates for foreign matter at positions overlapping the flow path based on the first image with a wider imaging range than the second image, and executing a second determination to determine whether the candidates exist within the flow path in the thickness direction of the flow path device based on the second image focused on the flow path, and causing the processor of the defect inspection device to execute the processes including these.

Advantages of the Invention

[0025] The technology according to the present disclosure can perform highly accurate inspection for foreign matter in the flow path in a short time in the defect inspection of the flow path device.

Brief Description of the Drawings

[0026] [Figure 1] It is a diagram showing the functions of the defect inspection device. [Figure 2] This figure shows an example of the appearance of a flow channel device. [Figure 3] This figure shows an example of an immunoassay procedure using electrophoresis, which is one example of an application of a fluidic device. [Figure 4] This figure shows an example of a cross-section of a fluid channel device. [Figure 5] This figure shows an example of a schematic configuration of a defect inspection device. [Figure 6] This is a flowchart illustrating an example of the overall processing procedure for defect inspection. [Figure 7] A flowchart illustrating an example of the macro inspection process. [Figure 8] This diagram conceptually illustrates an example of a macro inspection procedure. [Figure 9] This figure shows an example of an overall image of a flow channel device. [Figure 10] A flowchart illustrating an example of the processing procedure for micro-inspection. [Figure 11] This figure shows an example of a microscopic image when foreign matter is present in the flow path. [Figure 12] This figure shows an example of a microscopic image when there are no foreign objects in the flow path. [Figure 13] This figure shows an example of how a machine learning model processes data. [Figure 14] This figure shows a modified example of macro inspection, part 1. [Figure 15] This conceptually illustrates the process in Modification Example 1. [Figure 16] This figure shows a modified example of macro inspection, part 2. [Figure 17] This diagram conceptually illustrates the polar coordinate transformation in Modification Example 2. [Figure 18] This figure shows an example of scanning the entire thickness range of a fluidic device during micro-inspection. [Modes for carrying out the invention]

[0027] [First Embodiment] As shown in Figure 1, the defect inspection device 10 according to the technology of this disclosure inspects for defects that occurred during the manufacturing of the flow channel device 30 by performing image inspection on the flow channel device 30 manufactured on the production line. The flow channel device 30 is used, for example, in immunoassays of biological fluid samples such as blood. The flow channel device 30 is used, for example, in immunoassays based on the LBA-EATA method (Liquid-phase Binding Assay and Electrokinetic AnalyteTransport Assay).

[0028] As shown in Figures 1 and 2, the flow channel device 30 includes, as an example, a rectangular plate-shaped main body 33 and a cylindrical portion 34. The main body 33 has a flow channel 31 through which liquid flows, and the flow channel 31 includes a linear microchannel 31A and a circular well 31B connected to the microchannel 31A. The main body 33 is composed of a substrate 38 and a film 39. The flow channel 31 is formed by grooves formed on the back side of the substrate 38. The cylindrical portion 34 is provided on the front side of the substrate 38. The substrate 38 and the cylindrical portion 34 are injection-molded products made of transparent plastic and are integrally formed. The cylindrical portion 34 is provided for supplying liquid to the flow channel 31, discharging liquid from the flow channel 31, and inserting electrodes. The well 31B is a region corresponding to the bottom side of the cylindrical portion 34. In the art of this disclosure, the substrate 38 is an example of a transparent "plate-shaped member" having a groove-forming surface in which grooves as flow channels 31 are formed.

[0029] A film 39 is provided on the back surface of the substrate 38, and the flow channel 31 is covered by the film 39 (see also Figure 4). The film 39 is also transparent. As shown in the back view of Figure 2, when the main body 33 is viewed from the film 39, the flow channel 31 can be observed through the film 39.

[0030] As an example, the outer dimensions of the main body 33 are approximately 20 mm in width in the short side direction (X direction) and approximately 50 mm to 60 mm in length in the long side direction (Y direction). The microchannel 31A is a micrometer-order channel with a width of several tens of micrometers, and includes straight and curved sections. The well 31B is provided with a cylindrical section 34 that protrudes in the thickness direction (Z direction). The well 31B and the cylindrical section 34 function as an inlet for samples and liquids, an electrode insertion port for inserting pin-shaped electrodes, etc. In the thickness direction (Z direction) perpendicular to the XY plane, the thickness of the channel device 30, including the height of the cylindrical section 34, is approximately 7 mm.

[0031] As shown in Figure 3, in the LBA-EATA method, first, reagents such as a primary labeled antibody solution (e.g., an antibody labeled with DNA (Deoxyribo-Nucleic Acid)), a secondary labeled antibody solution (e.g., an antibody labeled with fluorescence), and buffer are dispensed into channel 31 in addition to the sample. After dispensing, pressure is applied to the microchannel 31A, filling the channel 31, including the microchannel 31A, with liquid. Then, by applying voltage, the primary labeled antibody moves within the microchannel 31A according to the principle of isokinetic electrophoresis. As the primary labeled antibody is concentrated, an immunocomplex is formed between the primary labeled antibody and the antigen and secondary labeled antibody, forming a concentrated layer of immunocomplexes. In parallel with the immune reaction, B / F (Bound / Free) separation proceeds to separate the immunocomplexes from the unreacted fluorescently labeled antibody. Subsequently, the position of the cathode to which the voltage is applied is changed, and capillary gel electrophoresis (CGE) is performed. By electrophoresis, the immune complex is separated from other components within the microchannel 31A and guided to the measurement region 36 (see Figure 2). Then, the immune complex is quantitatively measured in the measurement region 36 using laser-induced fluorescence (LIF). Laser-induced fluorescence is a measurement method that quantitatively measures the immune complex by irradiating the measurement region 36 with laser light, exciting the fluorescently labeled antibodies contained in the immune complex, and detecting the fluorescence emitted after excitation.

[0032] Figure 4 shows a cross-section of the main body 33 in the microchannel 31A, with the back side of the main body 33, i.e., the side to which the film 39 is attached, facing upwards. As described above, the substrate 38 is an injection-molded product made of transparent plastic, and grooves constituting the channel 31 are formed therein. The plastic material is, for example, polymethyl methacrylate (PMMA) resin. The thickness TB of the substrate 38 excluding the channel 31 is, for example, approximately 1500 μm. The depth DP in the thickness direction of the microchannel 31A is, for example, approximately 30 μm. The width WFP perpendicular to the flow direction of the microchannel 31A varies depending on the location, but for example it is approximately 40 μm to approximately 170 μm. The film 39 is, for example, made of acrylic, and its thickness TF is, for example, approximately 130 μm. The film 39 is, for example, heat-welded to the surface of the substrate 38 where the grooves constituting the microchannel 31A are formed, so as to cover the grooves.

[0033] The reason for bonding the film 39 by welding, such as heat welding, is as follows: If the film 39 is bonded to the substrate 38 using an adhesive, the adhesive may enter the microchannel 31A and become a foreign substance 51 that adversely affects the measurement of the immunoassay. Therefore, welding without the use of adhesive is selected for bonding the film 39.

[0034] As shown in Figure 5, the defect inspection apparatus 10 comprises a first imaging device 11, a second imaging device 12, a displacement meter 13, a base 14, and a control unit 16. The base 14 is the part on which the flow channel device 30 to be inspected is set. A marker 14A indicating a reference position is provided on the surface of the base 14. The flow channel device 30 is set on the base 14 with the back side, to which the film 39 is attached, facing upward. As a result, the state of the flow channel 31 formed on the substrate 38, including the film 39, is imaged by the first imaging device 11 and the second imaging device 12.

[0035] The first imaging device 11 and the second imaging device 12 are imaging devices that image the fluid channel device 30 set on the base 14. The first imaging device 11 is, for example, a camera having a first optical system 11A and a first sensor unit 11B. The first optical system 11A forms an image of the fluid channel device 30 on the detection surface. The first optical system 11A has a field of view that includes the entire range including the fluid channel 31 of the fluid channel device 30 as the imaging range SA1. For example, a general solid lens made of glass or plastic is used in the first optical system 11A.

[0036] The first optical system 11A is positioned with its optical axis aligned with the thickness direction (Z direction) of the flow channel device 30 in order to image the entire range in the XY plane when the flow channel device 30 is viewed from above. The focus position of the first optical system 11A is adjusted so that it is in focus on the flow channel device 30 on the base 14. The first sensor unit 11B has an image sensor having a detection surface on which a plurality of detection elements composed of photoelectric conversion elements are arranged. The image sensor is a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, etc.

[0037] The second imaging device 12, for example, has a second optical system 12A and a second sensor unit 12B, and is a digital microscope capable of magnifying and observing objects. Furthermore, the second imaging device 12 has a focus control mechanism 12C that can adjust the focus position of the second optical system 12A. The second sensor unit 12B is an image sensor similar to that of the first sensor unit 11B.

[0038] The second optical system 12A has a field of view that defines the imaging range SA2 as a small portion of the flow channel device 30. The second optical system 12A has a higher magnification than the first imaging device 11, but a narrower field of view. The magnification of the second optical system 12A is, for example, about 10 to 40 times. The second optical system 12A has a larger numerical aperture (NA) and a shallower depth of field than the first optical system 11A. Depth of field refers to the range in front of and behind the position of the focused subject in the optical axis direction. The second optical system 12A, like the first optical system 11A, is positioned with its optical axis aligned with the thickness direction (Z direction) of the flow channel device 30. The second optical system 12A of the microscope has a shallower depth of field compared to the first optical system 11A of a normal camera.

[0039] The second optical system 12A is an optical system that employs a liquid lens as an example. Compared to solid lenses, liquid lenses have a faster focusing speed. The focus control mechanism 12C is capable of adjusting the focus on the order of micrometers. By employing a liquid lens in the second optical system 12A, the focus control mechanism 12C can perform highly accurate and rapid focusing adjustments.

[0040] The first imaging device 11 captures an overall image WI, with an imaging range SA1 encompassing the entire range of the fluid channel device 30. The second imaging device 12 captures a microscopic image PI, with an imaging range SA2 that is narrower than the imaging range SA1 of the first imaging device 11 and encompasses only a portion of the fluid channel device 30. The overall image WI is an example of the "first image" relating to the technology of this disclosure, and the microscopic image PI is an example of the "second image" relating to the technology of this disclosure.

[0041] The first optical system 11A has a deeper depth of field than the second optical system 12A. Therefore, the overall image WI captured by the first optical system 11A has a wider range of focus both before and after the in-focus area. The optical performance of the first optical system 11A is as follows as an example. In the first optical system 11A, the magnification is approximately 1.5x, the numerical aperture (NA) is approximately 0.08, and the depth of field DF1 is approximately 300 μm. That is, the depth of field DF1 of the first optical system 11A (see Figure 4) is wider than the sum of the thickness TF of the film 39 of the flow channel device 30 (approximately 130 μm as an example, as mentioned above) and the depth DP of the flow channel 31 in the thickness direction (approximately 30 μm as an example). Therefore, for example, if the focus of the first optical system 11A is set to the center of the flow channel 31 in the thickness direction, the range including the surface of the main body 33 will be in focus. Therefore, the overall image WI can capture foreign matter 51 present in the flow channels 31 in the thickness direction of the main body 33 and on the surface of the film 30. Furthermore, since the imaging range SA1 of the overall image WI includes the entire range in the XY plane when viewing the flow channel device 30 from above, the overall image WI can capture foreign matter 51 present in the entire range in the XY plane of the main body 33.

[0042] In contrast, the optical performance of the second optical system 12A is as follows, as an example. In the second optical system 12A, the magnification is approximately 10x, the NA is approximately 0.3, and the depth of field DF2 is approximately 3.5 μm. That is, the depth of field DF2 of the second optical system 12A (see Figure 4) is, for example, less than or equal to the depth DP in the thickness direction of the channel 31 (for example, approximately 30 μm). Therefore, when the channel 31 is focused in the thickness direction (Z direction) of the main body 33, the area before and after the focused position, i.e., the area other than the channel 31, will be out of focus and blurred. Therefore, in the microscope image PI focused on the channel 31, foreign matter 51 that exists within the depth of field DF2, including the focused position, is clearly visible. On the other hand, foreign matter 51 that exists in parts other than the channel 31 in the thickness direction (Z direction), such as the surface of the main body 33, will not be in focus. Therefore, in a microscope image PI focused on the channel 31, foreign matter 51 present in areas other than the channel 31 will either be blurred or not visible.

[0043] The first imaging device 11 and the second imaging device 12 can each be moved in the XY plane by a moving mechanism 21. The moving mechanism 21 selectively moves the first imaging device 11 and the second imaging device 12 to an imaging position on the base 14 capable of imaging the fluidic device 30.

[0044] The displacement meter 13 is used to measure the precise position of the surface of the flow channel device 30. For example, the displacement meter 13 is a laser displacement meter that measures the surface position by irradiating the surface of the flow channel device 30 with laser light. If individual differences are not considered, the thickness of the flow channel device 30 is constant if the specifications are the same. Also, if the height of the base 14 on which the flow channel device 30 is set is constant, the subject distance to the first imaging device 11 and the second imaging device 12 will also be constant. In that case, the focusing position in the first imaging device 11 and the second imaging device 12 will also be constant. However, strictly speaking, there are individual differences in the thickness of the flow channel device 30, so the subject distance changes slightly. In the case of the second imaging device 12 of a microscope, the depth of field is shallow, so focus control is critical, and individual differences in the thickness of the flow channel device 30 have an effect. The displacement meter 13 is used to adjust the focus by measuring the position of the flow channel device 30, taking into account the individual differences in the thickness of the flow channel device 30.

[0045] The control unit 16 comprehensively controls the entire defect inspection device 10. The control unit 16 is composed of, for example, a personal computer and has a processor 24, memory 25, storage 26, and a display 27. The processor 24 is, for example, a CPU (Central Processing Unit). The memory 25 is, for example, RAM (Random Access Memory) and is used as working memory for the processor 24.

[0046] The storage device 26 is a data storage device such as a hard disk drive or a solid-state drive. The storage device 26 stores basic programs such as the operating system, as well as application programs that cause the processor 24 to function as a defect inspection device 10. Program 28 is an example of an application program and is an example of an "operation program" related to the technology of this disclosure. The display 27 displays the operation screen, the overall image WI and the microscope image PI, as well as inspection results.

[0047] The processor 24 functions as the control unit 16 of the defect inspection device 10 by loading and executing the program 28 into the memory 25. Furthermore, by executing the program 28, the processor 24 functions as an image analysis processing unit for the overall image WI and the microscopic image PI. As a result, the processor 24 performs defect inspection based on the images. Below, we will describe a defect inspection, specifically a channel foreign matter inspection, which checks whether foreign matter is present in the channel 31 of the channel device 30. In addition to the channel foreign matter inspection, the defect inspection items performed by the defect inspection device 10 also include dimensional inspections, such as checking whether the dimensions of the channel device 30 conform to specifications.

[0048] As described above, the flow channel device 30 is used in immunoassays, which involve optically measuring a test substance while a liquid flows through the flow channel 31. Therefore, if foreign matter 51 is present in the flow channel 31, it may contaminate the liquid being measured, or the foreign matter 51 may block the flow channel 31. Such a flow channel device 30 is a defective product. The inspection for foreign matter in the flow channel is an inspection to detect such defective flow channel devices 30.

[0049] In the in-flow channel foreign matter inspection, the processor 24 performs a first determination as a macro inspection, based on the overall image WI, to detect candidate foreign matter 51 located in a position overlapping with the flow channel 31, and a second determination as a micro inspection, based on a microscope image PI focused on the flow channel 31, to determine whether or not candidate foreign matter 51 exists in the flow channel in the thickness direction of the flow channel device 30.

[0050] The following describes the in-flow channel foreign matter inspection performed by the defect inspection device 10, using Figures 6 to 13. Figure 6 is a flowchart showing the overall processing procedure for in-flow channel foreign matter inspection. As shown in Figure 6, first, in step ST1000, the manufactured flow channel device 30 is set on the base 14. The flow channel device 30 is set on the base 14 with the cylindrical portion 34 facing downwards and the back surface of the main body portion 33, that is, the back surface on the film 39 side, facing upwards. This allows the first imaging device 11 and the second imaging device 12 to observe the flow channel 31 through the film 39.

[0051] In step ST1100, the processor 24 moves the first imaging device 11 to an imaging position above the base 14 by controlling the movement mechanism 21, and causes the first imaging device 11 to capture an overall image WI of the flow channel device 30. The processor 24 acquires the overall image WI from the first imaging device 11. In step ST1200, the processor 24 performs a macro inspection based on the overall image WI.

[0052] Figure 7 is a flowchart showing the macro inspection process based on the overall image WI of step ST1200 shown in Figure 6. The relative positional relationship between the marker 14A on the base 14 and the position where the flow channel device 30 is set is known, and the region in which the flow channel 31 is formed within the flow channel device 30 is also known. Therefore, in step ST1210 shown in Figure 7, the processor 24 detects the marker 14A from the overall image WI and extracts the region of the flow channel 31, which is the inspection target region for macro inspection, using the marker 14A as a reference point. As a result, as shown as <Region Extraction> in Figure 8, the region of the flow channel device 30 including the flow channel 31 is extracted from the overall image WI.

[0053] Furthermore, the area of ​​the overall image WI that is extracted as the region to be inspected for macro inspection can be changed as needed. In this example, the entire flow path 31 is inspected, but a part of it may be excluded.

[0054] Alternatively, the inspection target area including the channel 31 can be extracted from the overall image WI by pattern matching based on a known pattern of the channel 31. However, in the case of pattern matching, if the type or specifications of the channel device 30 change and the pattern of the channel 31 is altered, it is necessary to store the new pattern. A method based on the relative positional relationship with the marker 14A, as in this example, can flexibly handle changes in the pattern of the channel 31.

[0055] Next, in step ST1220, the processor 24 performs contrast enhancement processing on the overall image WI. In the overall image WI, there is a difference in density between the channel 31 and the other parts, so the channel 31 is enhanced by the contrast enhancement processing, as shown in <Contrast Enhancement> in Figure 8.

[0056] In step ST1230, the processor 24 detects the channel 31 by applying a binarization process to the overall image WI. As shown in Figure 8, when the overall image WI, in which the channel 31 has been enhanced by contrast enhancement processing, is subjected to binarization processing, an image is obtained in which the channel 31 and the other parts are clearly distinguished. The processor 24 detects the channel 31 from the binarized image. Then, in the image in which the channel 31 has been detected, the processor 24 performs a hole-filling process on the inside of the channel 31 in order to distinguish the inside from the outside of the channel 31. As a result, the inside and outside of the channel 31 can be clearly distinguished in the overall image WI.

[0057] In step ST1240, the processor 24 determines whether the flow path detection in step ST1230 was performed correctly. For example, the processor 24 measures the area of ​​the flow path 31 and compares it with the area according to the specifications of the flow path device 30 to determine whether the flow path detection was performed correctly. If the determination result in step ST1240 is incorrect (N in ST1240), the processor 24 proceeds to error handling and stops the subsequent inspection. If the determination result in step ST1240 is correct (Y in ST1240), the processor 24 proceeds to step ST1250.

[0058] In step ST1250, the processor 24 detects foreign object candidates that are located in the XY plane, overlapping with the flow path 31, and that meet the size requirements. In step ST1250, the processor 24 first detects foreign object candidates that overlap with the flow path 31 based on the binarized overall image WI as shown in Figure 9. In Figure 9, the identification frame BB is displayed for foreign object candidates that overlap with the flow path 31. The processor 24 also measures the size of the detected foreign object candidates 51 and detects only foreign object candidates 51 that are larger than or equal to a preset specified size. Foreign object 51 smaller than the specified size is unlikely to be a foreign object 51 that should be removed as a defect, and is therefore removed. When step ST1250 is completed, the processor 24 proceeds to step ST1300 in the flowchart of Figure 6.

[0059] In step ST1300 of Figure 6, the processor 24 determines whether a candidate foreign object exists in the flow path within the XY plane. If no candidate foreign object is detected in the macro inspection (N in step ST1300), the processor 24 proceeds to step ST2200 without performing a micro inspection, determines that there are no defects in the flow path foreign object inspection, and declares the flow path foreign object inspection a success. On the other hand, if a candidate foreign object is detected in the macro inspection (Y in step ST1300), the processor 24 proceeds to step ST1400.

[0060] In step ST1400, the processor 24 determines whether the number of foreign object candidates detected in the macro inspection is within a predetermined number. If the number of foreign object candidates exceeds the predetermined number (N in step ST1400), the process proceeds to step ST2100 without performing a micro inspection, and a defect is determined in the in-flow channel foreign object inspection, resulting in a failure of the in-flow channel foreign object inspection. On the other hand, if the number of foreign object candidates is within the predetermined number (Y in step ST1400), the processor 24 proceeds to step ST1500.

[0061] In step ST1500, the processor 24 performs a microinspection based on the microscope image PI. Figure 10 is a flowchart showing the microinspection process in step ST1500. In step ST1510, the processor 24 acquires a microscope image PI focused on the channel 31 at the location of the foreign object candidate detected in the macroinspection. The processor 24 moves the second imaging device 12 to an imaging position above the base 14 by controlling the movement mechanism 21. The second imaging device 12 is then aligned with the location of the foreign object candidate detected in the macroinspection in the XY plane. In the Z direction, the processor 24 focuses on the channel 31 by controlling the focus control mechanism 12C. The second imaging device 12 performs the imaging operation with the channel 31 in focus.

[0062] In Figures 11 and 12, since the channel 31 also has a depth DP, the second imaging device 12 scans in the direction of the depth DP (Z direction) within the channel 31 by controlling the focus control mechanism 12C. As a result, the processor 24 acquires microscopic images PI at multiple locations in the thickness direction within the channel 31.

[0063] In step ST1520, the processor 24 uses a machine learning model to perform foreign object detection and morphological determination based on the acquired microscope image PI. The machine learning model may include, for example, a convolutional neural network used for object detection within an image. The machine learning model is trained to detect foreign objects 51 of a desired morphology from the input microscope image PI.

[0064] The microscope images PI shown in Figures 11 and 12 were both acquired at a location where the channel 31 and the foreign object 51 overlap in the XY plane, and are microscope images PI focused on the channel 31 in the Z direction. In such microscope images PI, if the foreign object 51 is present in the channel 31 in the Z direction, the foreign object 51 is visible in the microscope image PI as shown in Figure 11. On the other hand, if the foreign object 51 is located in a position in front of or behind the channel 31 in the Z direction, such as when the foreign object 51 is present on the surface of the main body 33, the foreign object 51 is not visible in the microscope image PI, or it is blurred, as shown in Figure 12. As mentioned above, this is because the depth of field DF2 of a digital microscope like the second imaging device 12 is shallower than the depth of field DF1 of the first imaging device 11.

[0065] As shown in Figure 13, if the machine learning model detects a foreign object 51 based on the input microscope image PI, it outputs an image with a recognition frame BB attached to the detected foreign object 51, for example. When the microscope image PI shown in Figure 11 is input, the machine learning model detects the foreign object 51 from the microscope image PI and adds the recognition frame BB. On the other hand, when the microscope image PI shown in Figure 12 is input, the machine learning model cannot detect the foreign object 51 from the microscope image PI.

[0066] In this way, by performing a microscopic inspection based on the microscopic image PI acquired by the second imaging device 12, which has a depth of field DF2 within the depth DP of the channel 31, it is possible to determine whether or not foreign matter 51, which is located in a position overlapping with the channel 31 in the XY plane, is also present in the channel 31 in the Z direction.

[0067] In addition to detecting foreign objects, the machine learning model also simultaneously determines the morphology of potential foreign objects present in the channel 31. The potential foreign objects detected in the macro inspection include not only foreign objects 51 with shapes that should be evaluated as defects, but also potential foreign objects with shapes that should not be evaluated as defects. In Figure 13, potential foreign object 52 is, for example, a component contained in the material of the main body 33, and one example is an ultraviolet absorber. Since such potential foreign object 52 should not be evaluated as a defect, it is excluded from the targets to be detected as foreign object 51. Potential foreign object 52 has a morphological difference from the foreign object 51 that should be evaluated as a defect. Potential foreign object 53 is a looseness in the shape of the inner wall of the channel 31. Such potential foreign object 53 is also excluded from the targets to be detected as foreign object 51. Potential foreign object 53 also has a morphological difference from the foreign object 51 that should be evaluated as a defect. The machine learning model is trained to recognize such morphological differences and not detect potential foreign object 52 as foreign object 51. By using such a machine learning model, foreign object detection and morphological determination are performed simultaneously during micro-inspection. When the micro-inspection in step ST1500 is completed, the processor 24 proceeds to step ST1600 in Figure 6.

[0068] In step ST1600 in Figure 6, the processor 24 determines, based on the results of the micro-inspection, whether a candidate foreign object is present in the flow path 31 in the Z direction. As shown in Figure 11, if there is a foreign object 51 present in the flow path 31 (Y in step ST1600), the processor 24 proceeds to step ST1700. On the other hand, as shown in Figure 12, if there is no foreign object 51 in the flow path 31 (N in step ST1600), the process proceeds to step ST1900.

[0069] In step ST1700, the processor 24 determines whether the shape and size of the foreign object candidate meet the conditions. Regarding the shape of the foreign object candidate, the processor 24 determines whether it meets the conditions based on the results of the morphological determination shown in Figure 13 in the micro-inspection. As shown in Figure 13, in this example a machine learning model is used for morphological determination, so foreign object candidates 52 and 53, which do not meet the shape conditions, are excluded from detection. Therefore, these foreign object candidates 52 and 53 are determined not to meet the shape conditions. In contrast, foreign object 51 is determined to meet the shape conditions.

[0070] Furthermore, the processor 24 determines whether the foreign object 51 detected by the machine learning model meets the size requirements. The size determination is performed, for example, after binarizing the microscope image PI in which the foreign object 51 is detected. That is, the processor 24 measures the size of the detected foreign object 51 from the binarized microscope image PI and determines whether the measured size meets the requirements. By binarizing, for example, the external shape of the foreign object 51 becomes clearer, thus improving the accuracy of the size measurement. In this example, the size determination is performed separately from the morphological determination, but the size determination may also be performed using a machine learning model, similar to the morphological determination.

[0071] In this way, foreign object candidates that are present in the flow path 31 in the Z direction and that satisfy the shape and size conditions are determined to be foreign objects 51 that should ultimately be evaluated as defects. When step ST1700 is completed, the processor 24 proceeds to step ST1800.

[0072] In step ST1800, the processor 24 counts the foreign object candidate that satisfies the conditions of step ST1700 as a foreign object 51 that should be evaluated as a defect.

[0073] In step ST1900, if the processor 24 finds another potential foreign object detected in the macro inspection (Y in step ST1900), it returns to step ST1500 for that other potential foreign object and repeats the process up to step ST1800. This ensures that a micro inspection is performed on all potential foreign objects detected in the macro inspection.

[0074] In step ST2000, if the processor 24 finds even one foreign object 51 in the flow path 31 (Y in step ST2000), it proceeds to step ST2100, determines that there is a defect, and fails the defect inspection. On the other hand, if there are no foreign objects 51 in the flow path 31 (N in step ST2000), it proceeds to step ST2200, determines that there is no defect, and passes the defect inspection.

[0075] As described above, the defect inspection apparatus 10 is a defect inspection apparatus for inspecting defects in a flow channel device 30 having a flow channel 31, and comprises a processor 24 and, as an imaging apparatus for imaging the flow channel device 30, a first imaging apparatus 11 that captures an overall image WI (an example of a first image), and a second imaging apparatus 12 that has a second optical system 12A (an example of an optical system) with a shallower depth of field than the first imaging apparatus 11 and captures a microscope image PI (an example of a second image) with a higher magnification than the overall image WI. The processor 24 performs a macro inspection (an example of a first determination) to detect candidate foreign objects 51 located in a position overlapping with the flow channel 31 based on the overall image WI, which has a wider imaging range than the microscope image PI, and performs a micro inspection (an example of a second determination) to determine whether or not a candidate foreign object exists in the flow channel 31 in the thickness direction of the flow channel device 30 based on the microscope image PI which is focused on the flow channel 31.

[0076] Therefore, in defect inspection of the flow channel device 30, it becomes possible to perform accurate inspection of foreign matter in the flow channel in a short time. Specifically, in foreign matter inspection in the flow channel, by performing a macro inspection (an example of a first determination), a candidate foreign matter present in the flow channel 31 is detected in the XY plane when the flow channel device 30 is viewed from above, and a micro inspection (an example of a second determination) is performed on the candidate foreign matter to determine whether or not the candidate foreign matter is present in the flow channel in the Z direction, which is the thickness direction of the flow channel device 30 that is perpendicular to the XY plane. Micro inspection allows for accurate determination, but the imaging range in the XY plane is narrow. Therefore, performing a micro inspection over the entire range in the XY plane is very time-consuming. The defect inspection apparatus 10 of this disclosure detects candidate foreign matter with a macro inspection and performs a micro inspection on the location of the candidate foreign matter detected by the macro inspection. Since the range in which the micro inspection is performed is narrowed down by the macro inspection, it becomes possible to perform accurate inspection of foreign matter in the flow channel in a short time compared to when the micro inspection is performed over the entire range in the XY plane.

[0077] In addition, while the second determination is performed based on a microscope image PI focused on the channel 31 during microscopic inspection, the second determination may also be performed based on a microscope image PI that is slightly out of focus from the channel 31. When focusing during microscopic inspection, the accuracy of focusing can often be improved by moving the focus back and forth. In the microscope image PI obtained in this way, the foreign object candidate will appear somewhat blurred, but it is also possible to perform the second determination using that microscope image PI.

[0078] Furthermore, the thickness of the channel 31 formed in the channel device 30 to be inspected for defects is on the order of micrometers. A size on the order of micrometers requires micrometer precision, specifically less than 1 mm (less than 1000 μm). When the thickness is on the order of micrometers, precision is required in determining whether or not a candidate foreign object exists in the channel 31 in the thickness direction, so the technology of this disclosure, which performs defect inspection by combining the first imaging device 11 and the second imaging device 12, is effective. Moreover, as mentioned above, the thickness of the channel 31 is approximately 30 μm as an example. Thus, the technology of this disclosure is particularly effective when the thickness of the channel 31 is 100 μm or less.

[0079] Furthermore, the flow channel device 30 includes a transparent substrate 38 (an example of a plate-like member) having a groove-forming surface on which grooves forming a flow channel 31 are formed, and a transparent film 39 that seals the grooves. Because the film 39 is used to seal the grooves that constitute the flow channel 31, the flow channel 31 can be formed relatively easily compared to the case where the film 39 is not used.

[0080] Furthermore, the thickness of the film 39 is on the order of micrometers. When the thickness of the film 39 is on the order of micrometers, the distance between the film 39 and the channel 31 decreases in the thickness direction of the channel device 30. Therefore, as shown in Figures 11 and 12, it is difficult to distinguish between foreign matter 51 adhering to the surface of the main body 33 and foreign matter 51 present in the channel 31. The technology of this disclosure is particularly effective when performing inspection of foreign matter in the channel of such a channel device 30.

[0081] Furthermore, as mentioned above, the thickness TF of the film 39 is approximately 130 μm as an example. There are several reasons for using such a relatively thin film 39. When performing sample testing such as immunoassays optically, as mentioned above, it is necessary to detect light entering and / or leaving the channel 31 through the film 39 covering the channel 31. In this case, the thinner the film 39, the less light attenuation there is, and the less noise is introduced. Therefore, it is necessary to make the film 39 thin. To satisfy these requirements, for example, the film 39 is preferably 200 μm or less. Thus, the technology of this disclosure is particularly effective when the thickness of the channel 31 is 200 μm or less.

[0082] Furthermore, the film 39 is bonded to the substrate 38 (an example of a plate-shaped member) by welding. In the above embodiment, heat welding was shown as an example, but the reason why a method that does not use adhesive, such as welding, is adopted is to suppress the concern that if adhesive is mixed into the flow channel 31 during measurements such as immunoassay, it may have adverse effects. When the film 39 is provided by welding, the thickness of the film 39 is often thinner compared to when adhesive is used. For example, when heat welding is performed, it is preferable to make the film 39 thin from the viewpoint of ensuring thermal conductivity. Thus, the flow channel device 30 on which the film 39 is bonded by welding tends to have a thin film 39. For this reason, the technology of this disclosure is particularly effective for flow channel devices 30 having a film 39 bonded by welding. In addition to heat welding, ultrasonic welding or the like may be used as the welding method.

[0083] Furthermore, the flow channel device 30 is a flow channel device that moves the liquid in the flow channel 31 by electrophoresis. The size of the flow channel 31 through which the liquid is moved by electrophoresis is often on the order of micrometers. In addition, when moving the liquid by electrophoresis, there is a greater concern about the impact on the measurement of foreign matter 51 mixed in the flow channel 31 compared to when moving the liquid by pressure. For this reason, the technology of this disclosure is particularly effective.

[0084] Furthermore, as mentioned above, if an adhesive is used for the film 39, there is a concern that the adhesive may enter the flow channel 31. Therefore, it is preferable that the film 39 be welded without using an adhesive, which tends to result in a thinner film 39. The need for the technology of this disclosure is particularly high when welding the film 39 in a flow channel device 30 that flows liquid by electrophoresis.

[0085] Of course, even in the case of electrophoresis, the film 39 may be bonded with an adhesive. In this case as well, the technology of this disclosure, which can accurately detect foreign matter 51 other than the adhesive that enters the channel 31, is effective.

[0086] Furthermore, in the above embodiment, the second imaging device 12 is a digital microscope. A digital microscope is suitable for the technology of this disclosure because it has a depth of field on the order of micrometers.

[0087] Furthermore, the second imaging device 12 does not necessarily have to be a digital microscope; it just needs to have a shallower depth of field than the first imaging device 11. Depending on the thickness of the channel 31, it may be possible to inspect for foreign objects in the channel without using a digital microscope.

[0088] Furthermore, in the above embodiment, the processor 24 performs a morphological determination based on the microscope image PI (an example of the second image) to determine whether or not a candidate foreign object has a shape that should be evaluated as a defect. By performing such a morphological determination, it is possible to suppress, for example, the false detection of a candidate foreign object with a shape that should not be evaluated as a defect as a foreign object 51 that should be evaluated as a defect.

[0089] Furthermore, in the above embodiment, the processor 24 performs morphological determination only on foreign object candidates that have been determined to be inside the channel 31 in the micro-inspection (an example of the second determination). This allows for narrowing down the target of morphological determination, which is effective in shortening the time required for defect inspection. Of course, morphological determination may also be performed on foreign object candidates before determining whether they are inside the channel 31. If there are advantages other than time that take precedence, this method may be adopted.

[0090] Furthermore, in the above embodiment, the processor 24 performs morphological determination using a machine learning model. In image morphological determination, the accuracy of the machine learning model is improved, so the accuracy of morphological determination can be improved compared to when a machine learning model is not used. Of course, depending on the need for accuracy in morphological determination, morphological determination may be performed using rule-based algorithms such as pattern matching instead of a machine learning model.

[0091] Furthermore, in the above embodiment, the processor 24 performs macro inspection (an example of a first determination) based on the overall image WI (an example of a first image) based on a binarized image obtained by binarizing the pixel values. A method of detection using pattern matching without using a binarized image is also conceivable, but in that case, if the shape of the channel 31 changes, a new pattern image would need to be prepared. Therefore, by performing macro inspection based on a binarized image, it is possible to handle channel devices 30 with various channel 31 shapes without having to prepare a new pattern image.

[0092] Furthermore, the processing procedure shown in the above embodiment is merely an example and can be modified as appropriate.

[0093] (Modification 1: Determination of appropriateness of flow path detection) Modification 1, shown in Figures 14 and 15, is a modified example of determining the appropriateness of flow path detection in macro inspection based on the overall image WI. In Figure 14, the procedure is the same as shown in Figure 7 up to step ST1230. Modification 1 is characterized by step ST1240. In step ST1240, the processor 24 first executes step ST1241, separating the flow path 31 into a linear region microflow channel 31A and a circular region well 31B, as shown in Figure 15.

[0094] The separation of the microchannel 31A and the well 31B is performed, for example, as follows. First, the processor 24 performs a reduction process on the binarized image of the overall image WI in which the channel 31 is detected, reducing its size. Since the microchannel 31A is narrower than the well 31B, it disappears before the well 31B when its size is reduced. By extracting only the well 31B from the image in which the microchannel 31A has disappeared, and then enlarging only the well 31B back to its original size, an overall image WIB showing only the well 31B is generated, as shown in Figure 15. Then, by subtracting this overall image WIB showing only the well 31B from the overall image WI showing both the microchannel 31A and the well 31B, a difference image is obtained, and an overall image WIA showing only the microchannel 31A can be obtained.

[0095] Then, the processor 24 moves to step ST1242 and measures the circumference of the microchannels 31A of the channel 31 based on the overall image WIA. Furthermore, it counts the number of wells 31B in the circular region based on the overall image WIB. Then, in step ST1243, the processor 24 compares the circumference of the microchannels 31A measured in this way and the number of wells 31B counted with the circumference and number according to the specifications of the channel device 30 and determines whether the channel detection was performed correctly. In step ST1240 shown in Figure 7, an example of determining correctness based on the total area of ​​the channel 31 was explained. However, by performing the correctness determination as shown in step ST1240 in Figure 14, a more accurate correctness determination can be achieved.

[0096] (Modification 2: Processing of circular areas) Modification 2, shown in Figures 16 and 17, is an example in which, in macro inspection, when there is a circular region C such as a well 31B in the channel 31 of the flow channel device 30, preprocessing is performed using polar coordinate transformation when detecting a candidate foreign object from the circular region C. Specifically, in the processing of step ST1250 in Figure 14, the processor 24 separates the channel 31 into a microchannel 31A and a well 31B, and then uses polar coordinate transformation for the well 31B.

[0097] Figure 16 is a flowchart showing the processing procedure for step ST1250. Figure 17 is a conceptual diagram illustrating the preprocessing using polar coordinate transformation. As a prerequisite for the processing in Figure 16, as shown in Figures 14 and 15, the processor 24 separates the channel 31 into a microchannel 31A and a well 31B.

[0098] Then, in step ST1251, the processor 24 reads the circular region C of well 31B. In step ST1252, the processor 24 performs a polar coordinate transformation on the circular region C, converting it into a strip-shaped region CP in Cartesian coordinates where the circumferential direction extends linearly. As a result, as shown in Figure 17, the circumferential inner wall IW in the circular region C extends linearly along the longitudinal direction in the strip-shaped region CP in polar coordinates.

[0099] In step ST1253, the processor 24 performs preprocessing on the band-shaped region CP. In the preprocessing, the processor 24 first smooths the band-shaped region CP in the longitudinal direction. By smoothing in the longitudinal direction, a smoothed image CPA is obtained in which the inner wall IW extending in the longitudinal direction is retained while foreign object candidates (such as symbols 51 and 52) contained in the band-shaped region CP are removed. The processor 24 subtracts this smoothed image CPA from the band-shaped region CP before smoothing to obtain a difference image CPB. In the difference image CPB, the inner wall IW is removed, and only foreign object candidates remain.

[0100] After performing such preprocessing, the processor 24 proceeds to step ST1254, where it detects potential foreign objects from the band-shaped region obtained as a difference image CPB through preprocessing. Proceeding to step ST1255, the processor 24 performs this foreign object detection on the other well 31B if one exists.

[0101] After detecting potential foreign objects in well 31B is complete, the processor 24 proceeds to step ST1256 and performs potential foreign object detection on the microchannel 31A. Furthermore, in step ST1257, the processor 24 measures the size of the potential foreign objects detected from the microchannel 31A and well 31B and extracts the potential foreign objects that meet the size criteria. After this, the processor 24 proceeds to step ST1300 shown in Figure 6.

[0102] The effects of using such polar coordinate transformations are as follows. As shown in Figure 13 as candidate foreign object 53, if there is an inner wall IW, looseness of the inner wall IW may be detected as a candidate foreign object, and the detected candidate inner wall IW may become the target of micro-inspection. Since looseness of the inner wall IW is not evaluated as a defect, it is preferable from the viewpoint of shortening inspection time to exclude the inner wall IW from the target of micro-inspection. When excluding the inner wall IW from the image, the processing becomes complicated if the circular region C remains as is. Therefore, polar coordinate transformation is used to convert the circular region C into a band-shaped region CP and erase the inner wall IW.

[0103] Furthermore, as shown in Figure 17, using a difference image CPB of a band-shaped region offers the following advantages compared to detecting foreign object candidates while keeping the circular region C intact. For example, it is easier to change the criteria for detecting foreign object candidates on the side closer to the inner wall IW and the side further away from the inner wall IW (i.e., the side closer to the center of the circular region C in Cartesian coordinates). In the example shown in Figure 17, the criteria are set to be stricter on the side closer to the inner wall IW, while becoming looser as the distance from the inner wall IW increases. For example, on the side closer to the inner wall IW, there is a concern about false detection of foreign object candidates that should not be evaluated as defects, so the criteria are set in a direction that reduces the number of foreign object candidates. Conversely, the further away from the inner wall IW, the looser the criteria are set in a direction that leads to over-detection of foreign object candidates. When it is necessary to change the criteria within the circular region C in this way, using polar coordinate transformation simplifies the process. Foreign object candidates that are over-detected due to the loosened criteria are strictly judged by micro-inspection, so ultimately, over-detection is suppressed.

[0104] (Extreme modification 3: Full range scan in the thickness direction) Modification 3 shown in Figure 18 is an example in which, when performing a micro-inspection of the location of a candidate foreign object detected in a macro-inspection, the second imaging device 12 scans the entire range in the thickness direction (Z direction) of the flow channel device 30 and acquires a microscopic image PI. In the micro-inspection of the above embodiment, as shown in Figures 11 and 12, the scan was limited to the range of the flow channel 31, and only the microscopic image PI of the range of the flow channel 31 was acquired. In contrast, in Modification 3 shown in Figure 18, the second imaging device 12 acquires a microscopic image PI of the entire range in the thickness direction of the flow channel device 30. This makes it possible to detect foreign objects 51 adhering to the surface of the main body 33 as well. In the above embodiment, inspection of foreign objects in the flow channel was described, but the microscopic image PI of the entire range in the thickness direction of the flow channel device 30 can be used in other inspection items of defect inspection.

[0105] In the above embodiment, an immunoassay using electrophoresis was described as an example of an application for the flow channel device 30. However, any test using the flow channel 31 may, of course, be used for purposes other than immunoassay. Of course, as mentioned above, if the size of the flow channel 31 is on the order of micrometers, or if electrophoresis is used to move the liquid within the flow channel 31, the influence of foreign matter within the flow channel 31 on the measurement is significant, so the technology of this disclosure is particularly effective.

[0106] In each of the above embodiments, for example, the hardware structure of the Processing Unit that performs various processes, such as the processor 24, can be the following types of processors. As mentioned above, the types of processors include a CPU, which is a general-purpose processor that executes software (program 28) and functions as various processing units, as well as programmable logic devices (PLDs), such as FPGAs (Field Programmable Gate Arrays), which are processors whose circuit configuration can be changed after manufacturing, and dedicated electrical circuits, such as ASICs (Application Specific Integrated Circuits), which are processors with circuit configurations specifically designed to perform specific processes.

[0107] In terms of hardware structure, the processor 24 may be composed of the functions of the entire system on a single IC (Integrated Circuit) chip, as exemplified by a System On Chip (SoC), or it may be composed of multiple chips.

[0108] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits, which are combinations of circuit elements such as semiconductor devices.

[0109] From the above description, the technology described in the following supplementary information can be understood.

[0110] [Additional note 1] A defect inspection device for inspecting defects in a flow channel device having a flow channel, Processor and The imaging device for imaging a fluidic device comprises a first imaging device that captures a first image, and a second imaging device that has an optical system with a shallower depth of field than the first imaging device and captures a second image with a higher magnification than the first image. The processor is, Based on the first image, which has a wider imaging range than the second image, a first determination is performed to detect candidate foreign objects located in a position overlapping with the flow path. Based on the second image, which is focused on the flow path, a second determination is performed to determine whether or not a candidate exists within the flow path in the thickness direction of the flow path device. Defect inspection device. [Additional note 2] The thickness of the channels formed in the flow channel device is on the order of micrometers. The defect inspection device described in Appendix 1. [Additional note 3] The thickness of the channels formed in the flow channel device is 100 μm or less. A defect inspection device as described in Appendix 2 or Appendix 3. [Additional note 4] The flow channel device comprises a transparent plate-shaped member having a groove-forming surface in which grooves are formed as flow channels, and a transparent film that seals the grooves. A defect inspection device as described in any one of the appendices 1 to 3. [Additional note 5] The film thickness is on the order of micrometers. Defect inspection device as described in Appendix 4. [Additional note 6] The film thickness is 200 μm or less. Defect inspection device as described in Appendix 5. [Additional note 7] The film is bonded to the plate-shaped member by welding. Defect inspection device as described in Appendix 4. [Additional note 8] A fluid channel device is a device that moves liquid within a channel by electrophoresis. A defect inspection device as described in any one of the appendices 1 to 7. [Additional note 9] The second imaging device is a digital microscope. A defect inspection device as described in any one of the appendices 1 to 8. [Additional Note 10] The processor performs a morphological determination based on the second image to determine whether the candidate shape should be evaluated as a defect. A defect inspection device as described in any one of the appendices 1 to 9. [Additional Note 11] The processor performs morphological determination only on candidates that were determined to be within the flow path in the second determination. Defect inspection device as described in Appendix 10. [Additional Note 12] The processor performs morphological determination using a machine learning model. A defect inspection device as described in Appendix 10 or Appendix 11. [Additional Note 13] The processor performs a first decision based on the first image, using the binarized image obtained by binarizing the pixel values. A defect inspection device as described in any one of the appendices 1 to 12. [Additional Note 14] In the flow path of a flow channel device, when there is a circular region, The processor transforms the circular region of the first image into a linearly extending band-shaped region using polar coordinate transformation, and then detects candidates from the band-shaped region. A defect inspection device as described in any one of the appendices 1 to 13. [Additional Note 15] A method for operating a defect inspection device for inspecting defects in a flow channel device having a flow channel, The processor in the defect inspection device is The first imaging device acquires a first image of the flow channel device, A second image of a flow channel device captured by a second imaging device having an optical system with a shallower depth of field than the first imaging device, wherein the second image is acquired with a higher magnification than the first image. Based on the first image, which has a wider imaging range than the second image, a first determination is performed to detect candidate foreign objects located in a position overlapping with the flow path. Based on the second image, which is focused on the flow path, a second determination is performed to determine whether or not a candidate exists within the flow path in the thickness direction of the flow path device. How to operate a defect inspection device. [Additional Note 16] An operating program for a defect inspection device that inspects defects in a flow channel device having a flow channel, To acquire a first image of the flow channel device captured by the first imaging device, A second image of a flow channel device, captured by a second imaging device having an optical system with a shallower depth of field than the first imaging device, is obtained, and the second image has a higher magnification than the first image. Based on the first image, which has a wider imaging range than the second image, a first determination is performed to detect candidate foreign objects located in a position overlapping with the flow path. An operating program for a defect inspection device that causes the processor of the defect inspection device to perform a process including: performing a second determination based on a second image focused on the flow path, determining whether or not a candidate exists within the flow path in the thickness direction of the flow path device.

[0111] The technology of this disclosure can be appropriately combined with the various embodiments and / or modifications described above. Furthermore, it is understood that various configurations can be adopted without departing from the spirit of the invention, and the invention is not limited to the embodiments described above. In addition, the technology of this disclosure extends to storage media for storing programs non-temporarily, as well as programs. Programs also include program products.

[0112] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0113] In this specification, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0114] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference. [Explanation of Symbols]

[0115] 10. Defect Inspection Equipment 11. First Imaging Device 11A 1st optical system 11B First Sensor Unit 12. Second Imaging Device 12A 2nd optical system 12B Second Sensor Unit 12C Focus Control Mechanism 13 Displacement gauge 14 Pedestal 14A Marker 16 Control Unit 21 Moving mechanism 24 processors 25 memory 26 storage 27 displays 28 Programs 30 Flow Channel Devices 31 Flow channels 31A Microchannel 31B Well 33 Main body 34 Cylindrical section 36 Measurement area 38 Base material 39 film 51 Foreign matter 52, 53 Potential foreign objects BB identification frame C circular area CP band region CPA smoothed image CPB difference image DF1 depth of field DF2 depth of field Depth of the DP channel IW inner wall PI Microscope Image SA1 imaging range SA2 imaging range TF film thickness TB substrate thickness WFP width WI, WIA, WIB overall image

Claims

1. A defect inspection device for inspecting defects in a flow channel device having a flow channel, Processor and The imaging apparatus for imaging the aforementioned flow channel device comprises a first imaging apparatus for capturing a first image, and a second imaging apparatus having an optical system with a shallower depth of field than the first imaging apparatus and capturing a second image with a higher magnification than the first image. The aforementioned processor, Based on the first image, which has a wider imaging range than the second image, a first determination is performed to detect candidate foreign objects located in a position overlapping with the flow path. Based on the second image focused on the flow path, a second determination is performed to determine whether the candidate exists within the flow path in the thickness direction of the flow path device. Defect inspection device.

2. The size of the channel formed in the aforementioned channel device in the thickness direction is on the order of micrometers. The defect inspection apparatus according to claim 1.

3. The thickness of the channel formed in the aforementioned channel device is 100 μm or less. The defect inspection apparatus according to claim 2.

4. The flow channel device comprises a transparent plate-shaped member having a groove-forming surface on which grooves serving as flow channels are formed, and a transparent film that covers the grooves. The defect inspection apparatus according to claim 1.

5. The thickness of the aforementioned film is on the order of micrometers. The defect inspection apparatus according to claim 4.

6. The thickness of the aforementioned film is 200 μm or less. The defect inspection apparatus according to claim 5.

7. The film is bonded to the plate-shaped member by welding. The defect inspection apparatus according to claim 4.

8. The aforementioned flow channel device is a flow channel device that moves the liquid in the flow channel by electrophoresis. The defect inspection apparatus according to claim 1.

9. The second imaging device is a digital microscope. The defect inspection apparatus according to claim 1.

10. The processor performs a shape determination based on the second image to determine whether the candidate shape should be evaluated as a defect. The defect inspection apparatus according to claim 1.

11. The processor performs the shape determination only on the candidates that were determined to be within the flow path in the second determination. The defect inspection apparatus according to claim 10.

12. The processor performs the morphological determination using a machine learning model. The defect inspection apparatus according to claim 10.

13. The processor performs the first determination based on the first image based on the binarized image obtained by binarizing the pixel values. The defect inspection apparatus according to claim 1.

14. In the flow path of the aforementioned flow channel device, if there is a circular region, The processor transforms the circular region of the first image into a linearly extending band-shaped region using polar coordinate transformation, and detects the candidate from the band-shaped region. The defect inspection apparatus according to claim 1.

15. A method for operating a defect inspection device for inspecting defects in a flow channel device having a flow channel, The processor included in the aforementioned defect inspection device is The first imaging device acquires a first image of the flow channel device, A second image of the flow channel device is captured by a second imaging device having an optical system with a shallower depth of field than the first imaging device, wherein the second image has a higher magnification than the first image. Based on the first image, which has a wider imaging range than the second image, a first determination is performed to detect candidate foreign objects located in a position overlapping with the flow path. Based on the second image focused on the flow path, a second determination is performed to determine whether the candidate exists within the flow path in the thickness direction of the flow path device. How to operate a defect inspection device.

16. An operating program for a defect inspection device that inspects defects in a flow channel device having a flow channel, To acquire a first image of the flow channel device captured by the first imaging device, A second image of the flow channel device captured by a second imaging device having an optical system with a shallower depth of field than the first imaging device, wherein the second image has a higher magnification than the first image. Based on the first image, which has a wider imaging range than the second image, a first determination is performed to detect a candidate foreign object located in a position overlapping with the flow path. An operating program for a defect inspection device that causes the processor of the defect inspection device to perform a process including: performing a second determination based on the second image in focus on the flow path, determining whether or not the candidate exists in the flow path in the thickness direction of the flow path device.

Images