Microfluidic device observation device and microfluidic device observation method

Abstract

The present invention relates to a microfluidic device observation device (1) for observing an object to be detected existing inside a plurality of flow paths (103a) to (103e) arranged in a microfluidic device (100), comprising: a photographing unit (4) for acquiring an image of the microfluidic device (100); an extraction pixel selection unit (21) for selecting an extraction pixel from the plurality of pixels according to a specified reference based on the brightness values ​​of the plurality of pixels constituting the acquired image; a flow path determination unit (22) for determining the positions of the plurality of flow paths (103a) to (103e) included in the image; an object characteristic pixel group detection unit (24) for detecting a specified number or more of continuous extraction pixels existing inside the determined plurality of flow paths (103a) to (103e) as an object characteristic pixel group; and a statistical feature quantity calculation unit (26) for obtaining a statistical feature quantity of the object to be detected based on the number of object characteristic pixel groups and / or the number of extraction pixels constituting each object characteristic pixel group.

Description

Technical Field

[0001] The present invention relates to a microfluidic device observation apparatus and a microfluidic device observation method for observing a test object existing inside a microfluidic device provided with one or more flow channels. Background Art

[0002] A microfluidic device is a device that forms one or more flow paths on a substrate by using micromachining technology such as a microelectromechanical system (MEMS) (e.g., Patent Document 1). A microfluidic device is manufactured, for example, by forming a concave portion that becomes a flow path on the lower surface side, and joining an upper substrate and a lower substrate having a through hole formed therein for introducing a test fluid, etc., at a position corresponding to the end of the flow path. A microfluidic device is used, for example, as described in Patent Document 2, to examine the sensitivity of bacteria and fungi (target bacteria) to antibiotics.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2013-3011

[0006] Patent Document 2: Japanese Patent Application Publication No. 2015-177806

[0007] Patent Document 3: Japanese Patent Application Laid-Open No. 11-97512 Summary of the invention

[0008] Technical problem to be solved by the invention

[0009] In order to specifically explain the technical problem to be solved by the present invention, an example of a microfluidic device used in the above-mentioned inspection is described (refer to Figure 1 ). The upper surface of the microfluidic device 100 is provided with an inlet 101 and five air ports 102a to 102e. A bacterial liquid inlet flow path 103 having one end directly below the inlet 101 is provided inside the microfluidic device 100. The bacterial liquid inlet flow path 103 is branched into five branch flow paths 103a to 103e in the middle, and these branch flow paths extend directly below the air ports 102a to 102e respectively. The five branch flow paths 103a to 103e are arranged parallel to each other near the air ports 102a to 102e.

[0010] In the inspection of the target bacteria using the microfluidic device 100, first, antibacterial liquids a to d containing the same type of antibacterial drugs at different concentrations are introduced into the branch flow paths 103a to 103d from the air ports 102a to 102d. The amount of antibacterial liquids a to d introduced here is a level that is retained inside the branch flow paths 103a to 103d (not reaching the bacterial liquid introduction flow path 103). No antibacterial liquid is introduced into the branch flow path 103e. Then, the antibacterial liquids a to d introduced into each branch flow path 103a to 103d are dried so that the antibacterial drugs are fixed to the wall surfaces of each branch flow path 103a to 103d. Next, the bacterial liquid containing the target bacteria is introduced into the bacterial liquid introduction flow path 103 from the introduction port 101. Since air ports 102a to 102e for ventilation are formed at the ends of the branch flow paths 103a to 103e, the air in each branch flow path 103a to 103e is discharged from the air ports as the bacterial solution flows into each branch flow path 103a to 103e, causing the bacterial solution to flow into each branch flow path 103a to 103e. In the branch flow paths 103a to 103d, the inflowing bacterial solution contacts the antimicrobial agent fixed on the wall surface of each flow path.

[0011] After a predetermined time has passed since the bacterial solution was introduced, the microfluidic device 100 is placed at a predetermined position of a microscope, and an image of a region (observation region) 104 captured in the field of view of the microscope is acquired. Then, the portion corresponding to each branch flow path 103a to 103e is determined from the image of the observation region 104, and the state of the target bacteria present in the branch flow paths 103a to 103e is confirmed.

[0012] As described above, since antibacterial liquid is not introduced into one of the branch flow paths (branch flow path 103e in the above example), by comparing the number and shape of bacteria confirmed based on the image of the branch flow path 103e with the number and shape of bacteria confirmed based on the images of the other branch flow paths 103a to 103d, it is possible to know at what concentration of antibacterial liquid containing the antibacterial drug to show an effect on the target bacteria.

[0013] In the above inspection, it is generally necessary to repeatedly allow the antibacterial drug to act on the target bacteria for a specified time and observe the state of the target bacteria to confirm the change in the state of the target bacteria over time. Therefore, every time the inspector observes the microfluidic device through a microscope, he must repeatedly count the number of target bacteria present in the flow path or confirm the shape of the target bacteria, which is time-consuming. In addition, in most cases, since a large number of microfluidic devices are used for inspection, the following operations must be performed on each of such a large number of microfluidic devices: photographing the microfluidic device, counting the number of target bacteria present in the flow path or confirming the shape of the target bacteria, which becomes a huge burden for the inspector.

[0014] The technical problem to be solved by the present invention is to provide a microfluidic device observation apparatus and a microfluidic device observation method that can easily grasp the state of a test object existing inside a flow path provided in a microfluidic device.

[0015] Solutions for solving the above technical problems

[0016] A first aspect of the present invention made to solve the above-mentioned technical problems is a microfluidic device observation apparatus for observing an object to be tested existing inside a plurality of flow paths disposed in a microfluidic device, characterized by comprising:

[0017] A photographing unit, for acquiring an image of the microfluidic device;

[0018] an extraction pixel selection unit that selects an extraction pixel from a plurality of pixels constituting the acquired image according to a predetermined criterion based on brightness values ​​of the plurality of pixels;

[0019] a flow path determining unit that determines positions of the plurality of flow paths included in the image;

[0020] a detection unit for detecting a characteristic pixel group of an object, detecting a predetermined number or more of the extraction pixels that are continuous and exist inside the plurality of determined flow paths as a characteristic pixel group of an object;

[0021] The statistical feature value calculation unit obtains the statistical feature value of the object characteristic pixel group based on the number and / or shape of the object characteristic pixel group.

[0022] The imaging unit is not limited to any type as long as it can obtain an image showing contrast at the boundary of a flow path formed in the microfluidic device or at a location where the object to be tested exists. The imaging unit can be, for example, an optical microscope, an optical camera, a phase contrast microscope, or a differential interference microscope.

[0023] In the microfluidic device observation device of the first scheme of the present invention, first, the microfluidic device is set at a predetermined position of the shooting unit so that the area equipped with multiple flow paths is captured in the field of view, and the image of the area is obtained. Then, the extraction pixel selection unit selects the extraction pixel from the multiple pixels according to the predetermined reference based on the brightness values ​​of the multiple pixels constituting the acquired image. This is, for example, the brightness value (background brightness value) assumed by the pixels at the position where the boundary of the flow path and the object to be detected do not exist is set as a threshold, and the selection is performed by determining the pixels with higher (or lower) brightness than the threshold. Alternatively, the average value of the brightness of all pixels or the brightness value of the pixels at the position where the boundary of the flow path and the object to be detected do not exist, such as the four corners of the image, can also be set as a threshold, and the pixels with higher (or lower) brightness than the threshold can be determined. In addition, it can also be that instead of using the brightness value itself, the edge intensity (variation in brightness) is obtained from the image data using a differential filter, and the extraction pixel is selected based on the result of comparing the value with the predetermined value. By using such a differential filter, the background can be removed and the extraction pixels corresponding to the boundary of the flow path and the object to be detected in the flow path can be easily selected. In addition, whether to select pixels with high brightness or pixels with low brightness may be determined according to the characteristics of the imaging unit used (whether the boundary of the flow channel or the object to be tested appears bright or dark).

[0024] Next, the flow path determination unit determines the positions of the multiple flow paths included in the image of the microfluidic device acquired by the imaging unit. For example, the determination can be performed by template matching as described below: a template image of the microfluidic device is prepared in advance, and the image acquired by the imaging unit is compared with the template image. In this process, the position of the flow path can be determined by comparing the image of the microfluidic device itself with the template image, or the position of the extracted pixel selected by the extraction pixel selection unit with the template image can be compared to determine the position of the flow path. In addition, in the case where the microfluidic device is provided with multiple flow paths having the same width in parallel in at least a portion of the region, the positions of the multiple flow paths can be determined by image analysis processing such as in the embodiments described later.

[0025] Inside the flow path determined by the flow path determination unit, sometimes not only the contrast generated by structures such as the object to be tested (such as bacteria) but also the contrast generated by noise is shown. Therefore, the pixels corresponding to the positions where these are present may be mixed in the extracted pixels selected by the extraction pixel selection unit. Then, a continuous specified number of extracted pixels or more existing inside the determined multiple flow paths is detected as a characteristic pixel group of the object to be tested. The specified number can be set to be equivalent to the number of pixels of the smallest size assumed to be the object to be tested. Since it is rare for noise data to exist continuously in multiple pixels, the noise can be removed by this processing.

[0026] Next, the statistical feature calculation unit calculates the statistical feature of the object characteristic pixel group based on the number and / or shape of the object characteristic pixel group. The statistical feature refers to the quantity (value) associated with the number and shape of the object characteristic pixel group. The statistical feature may also include the quantity (value) of the shape of the object characteristic pixel group that is not directly dependent on the area and length of the object characteristic pixel group. The number of the object characteristic pixel groups corresponds to the number of objects present in the flow path. In addition, the shape of the object characteristic pixel group, that is, the sequence of pixels constituting each object characteristic pixel group, corresponds to the shape of an object. If the microfluidic device observation device of the present invention is used, the image obtained by the shooting unit can be processed in the order of the flow path determination unit, the extraction pixel selection unit, the object characteristic pixel group detection unit, and the statistical feature calculation unit to obtain the statistical feature of the object present in the flow path. Therefore, the inspector does not need to personally count the number of objects present in the flow path or confirm the shape, and can easily grasp the state of the object.

[0027] In the microfluidic device observation apparatus of the first embodiment of the present invention, the statistical feature quantity calculation unit can be configured to calculate the representative value of the statistical feature quantity of the plurality of the object characteristic pixel groups when there are a plurality of the object characteristic pixel groups inside a flow path. The representative value of the statistical feature quantity can be appropriately selected from the group consisting of the mean value, the median value, the mode value and their combination. By using the microfluidic device observation apparatus of this embodiment, for example, the average size and shape of the objects present in each flow path can be easily known.

[0028] In the case where the object to be tested is a biological sample such as cells or bacteria, the microfluidic device observation apparatus of the first embodiment of the present invention can adopt a structure that also includes a determination unit, which determines the state of the object to be tested based on the result of comparing the statistical feature quantity obtained for one of the multiple flow paths with the statistical feature quantities obtained for other flow paths.

[0029] When testing the sensitivity of a test object such as cells or bacteria to drugs such as antibiotics, generally, no drug is introduced into one of the multiple flow paths, and the number and shape of the test object present inside the flow path are used as a basis for determining the state of the test object. When the test object is a biological sample, the state can be easily known by using a microfluidic device observation device of this scheme. In addition, when there are multiple test object feature pixel groups in one flow path, the state of the test object can be determined based on the representative values ​​of these statistical feature quantities.

[0030] A second aspect of the present invention, which has been made to solve the above-mentioned technical problems, is a microfluidic device observation apparatus for observing an object to be inspected existing inside a plurality of flow paths disposed in a microfluidic device, characterized in that it comprises:

[0031] A photographing unit, for acquiring an image of the microfluidic device;

[0032] a flow path determining unit that determines positions of the plurality of flow paths included in the image;

[0033] The recognition unit specifies the number and / or shape of the inspection objects existing inside the specified plurality of flow paths when the acquired image is input.

[0034] In the microfluidic device observation apparatus of the second scheme, an identification unit is used to replace the extraction pixel selection unit, the object characteristic pixel group detection unit and the statistical feature quantity calculation unit in the first scheme. In the identification unit, if an image of the microfluidic device acquired by the shooting unit is input, the number and / or shape of the objects to be detected contained in each of the multiple flow paths determined by the flow path determination unit are determined. By using the microfluidic device observation apparatus of this scheme, the state of the object to be detected can be easily grasped as in the first scheme.

[0035] When the test object is a biological sample such as cells or bacteria, the microfluidic device observation apparatus of the second scheme can be configured such that the identification unit further determines the state of the test object existing inside each flow path based on the determined number and / or shape of the test object.

[0036] When the object to be tested is a biological sample, its state can be easily known by using the microfluidic device observation apparatus of this aspect.

[0037] The microfluidic device observation technology of the present invention can be applied to the examination of the sensitivity of target bacteria to drugs.

[0038] That is, the third aspect of the present invention, which has been accomplished to solve the above-mentioned technical problems, is a microfluidic device observation method for observing bacteria introduced together with a drug into a plurality of flow paths disposed in a microfluidic device, characterized by comprising:

[0039] acquiring an image of the microfluidic device;

[0040] Based on brightness values ​​of a plurality of pixels constituting the acquired image, selecting an extraction pixel from the plurality of pixels according to a prescribed criterion;

[0041] determining positions of the plurality of flow paths included in the image;

[0042] detecting a predetermined number or more of the extraction pixels that are continuous and exist inside the determined plurality of flow paths as a characteristic pixel group of the object under test;

[0043] Based on the number and / or shape of the characteristic pixel group of the object to be detected, a statistical feature value of the characteristic pixel group of the object to be detected is obtained.

[0044] In addition, a fourth aspect of the present invention, which has been accomplished to solve the above-mentioned technical problems, is a microfluidic device observation method for observing bacteria introduced together with a drug into a plurality of flow paths disposed in a microfluidic device, characterized by comprising:

[0045] acquiring an image of the microfluidic device;

[0046] determining positions of the plurality of flow paths included in the image;

[0047] Based on the input of the acquired image, the number and / or shape of the bacteria existing inside the specified plurality of flow paths are specified.

[0048] Effects of the Invention

[0049] By using the microfluidic device observation apparatus of the first or second aspect, or the microfluidic device observation method of the third or fourth aspect of the present invention, the state of the test object existing inside the flow path provided in the microfluidic device can be easily grasped. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is an example of a microfluidic device.

[0051] Figure 2 This is a diagram showing the main components of the microfluidic device observation apparatus of Example 1.

[0052] Figure 3 This is a flowchart for explaining the procedure of observing the test object existing inside the flow channel of the microfluidic device in Example 1.

[0053] Figure 4 This is an example of an image obtained by binarizing the brightness value of the observed image data in the first embodiment.

[0054] Figure 5 This is a diagram for explaining the template image used in Example 1.

[0055] Figure 6 This is a flowchart for explaining one of the steps of determining the flow path position in the observation image in the first embodiment.

[0056] Figure 7This is a diagram for explaining a method of obtaining the distance between straight lines in one of the steps of determining the flow path position in the observation image in the first embodiment.

[0057] Figure 8 This is an example of a histogram generated in one of the steps of specifying the flow path position in the observation image in the first embodiment.

[0058] Fig. 9 This is a diagram for explaining a method of specifying a flow path from an observation image in one of the steps of specifying the flow path position in the observation image in the first embodiment.

[0059] Fig.10 This is a diagram showing the main components of the microfluidic device observation apparatus of Example 2. DETAILED DESCRIPTION

[0060] The following is a description of the first and second embodiments of the microfluidic device observation apparatus of the present invention with reference to the accompanying drawings. Figure 1 The microfluidic device 100 described herein tests the sensitivity of target bacteria to antimicrobial solutions a to d containing the same antimicrobial drug at four different concentrations. In this embodiment, four antimicrobial solutions a to d are used, but the number of antimicrobial solutions (types of concentrations) can be appropriately changed. When using antimicrobial solutions of five or more concentrations, a microfluidic device having six or more flow paths can be used.

[0061] Figure 2 1 is a diagram showing the main components of the microfluidic device observation apparatus 1 of Example 1. The microfluidic device observation apparatus 1 of this embodiment is roughly composed of a dispensing unit 2, an incubator 3, a phase contrast microscope 4 (photographing unit), and a control and processing unit 10. Between the dispensing unit 2 and the incubator 3, and between the incubator and the phase contrast microscope 4, a device transport path and a transport mechanism (not shown) are provided for receiving instructions from the control and processing unit 10 and transporting the microfluidic device 100 between the two.

[0062] The control and processing unit 10 has the function of controlling the operation of the dispensing unit 2, the incubator 3 and the phase difference microscope 4, and the function of processing the image obtained by the phase difference microscope 4. The control and processing unit 10 has a storage unit 11, and further has an extraction pixel selection unit 21, a flow path determination unit 22, a test object determination unit 23, a characteristic pixel group detection unit 24, a flow path boundary pixel group selection unit 25, a statistical feature quantity calculation unit 26 and a determination unit 27 as functional modules. The entity of the control and processing unit 10 is a general computer, and these functional modules are visualized by executing the microfluidic device observation program 20 by its processor. In addition, the control and processing unit 10 is connected to an input unit 41 for the inspector to input appropriate information, and a display unit 42 for displaying an image of the flow path of the microfluidic device 100, etc.

[0063] The storage unit 11 stores a template image of the microfluidic device 100. The template image of the microfluidic device 100 is used to identify the position of the flow path from an image of the microfluidic device 100 captured by the phase contrast microscope 4. In this embodiment, only one type of microfluidic device 100 is used, but when multiple types of microfluidic devices are used, a template image is prepared for each type and stored in the storage unit 11.

[0064] An inlet 101 and five air ports 102a to 102e are provided on the upper surface of the microfluidic device 100. A bacterial liquid inlet flow path 103 having one end directly below the inlet 101 is provided inside the microfluidic device 100. The bacterial liquid inlet flow path 103 branches into five branch flow paths 103a to 103e in the middle, and these branch flow paths extend in a straight line to directly below the air ports 102a to 102e. The five branch flow paths 103a to 103e are arranged parallel to each other near the air ports 102a to 102e. In addition, Figure 1 Although the branch channels 103a to 103e are shown in an easily understandable manner, in an actual microfluidic device, the channel widths of the branch channels 103a to 103e are wider than the channel intervals.

[0065] Next, the flow of testing the sensitivity of target bacteria to antibiotics using the microfluidic device observation apparatus 1 of this embodiment will be described.

[0066] The tester prepares a bacterial solution containing target bacteria at a predetermined ratio (for example, determined according to the number of target bacteria and the amount of its culture solution) and antibacterial solutions a to d containing antibacterial drugs at different concentrations used for testing, and sets them in the dispensing unit 2 in advance.

[0067] When the inspector instructs to start the inspection via the input unit 41, a control signal instructing the start of the dispensing operation is sent from the control and processing unit 10 to the dispensing unit 2. When there are multiple types of microfluidic devices that can be used, the inspector can specify the type of microfluidic device 100 used for the inspection.

[0068] In the dispensing unit 2, antibacterial liquids a to d are introduced into each branch flow path 103a to 103d of the microfluidic device 100, and then a bacterial liquid containing target bacteria is introduced into the bacterial liquid introduction flow path 103 from the introduction port 101. In addition, no antibacterial liquid is introduced into the branch flow path 103e, and the number and size of the target bacteria in the branch flow path 103e are used as a reference for determining the sensitivity of the target bacteria to the antibacterial liquids a to d introduced into other flow paths. The amount of bacterial liquid introduced is predetermined to be an amount that can achieve a level that fixes the antibacterial drug at the position of the branch flow paths 103a to 103d. Since air ports 102a to 102e for ventilation are formed at the ends of the branch flow paths 103a to 103e, the air in each branch flow path 103a to 103e is discharged from the air ports as the bacterial liquid flows in, so that the bacterial liquid flows into each branch flow path 103a to 103e. In the branch flow paths 103a to 103d, the bacterial solution that flows in comes into contact with the antimicrobial drug fixed on the wall surface of each flow path.

[0069] After the antibacterial solutions a to d and the bacterial solution are introduced, the microfluidic device 100 is transported to the incubator 3 and heated for a predetermined time and a predetermined temperature to culture the target bacteria. After the predetermined time of culture, the microfluidic device 100 is transported from the incubator 3 to the phase contrast microscope 4. Figure 3 The flowchart of FIG. 1 illustrates a procedure for observing target bacteria existing inside a flow channel of a microfluidic device in the microfluidic device observation apparatus 1 of the present embodiment.

[0070] The microfluidic device 100 transported to the phase contrast microscope 4 is held at a predetermined shooting position in a predetermined orientation. The area of ​​the microfluidic device 100 captured in the field of view is photographed by the phase contrast microscope 4. The area captured in the field of view of the phase contrast microscope 4 in the entire microfluidic device 100 (corresponding to the area of ​​at least a portion of the microfluidic device 100) becomes the observation area 104, and an image of the observation area 104 (hereinafter referred to as "observation image") is acquired in the phase contrast microscope 4 (step S1). The phase contrast microscope 4 sends the acquired image data of the observation area 104 to the control and processing unit 10. The photographing data received in the control and processing unit 10 is stored in the storage unit 11.

[0071] If the image data is stored in the storage unit 11, the pixel selection unit 21 uses a differential filter to calculate the edge intensity (brightness change) in the image (step S2). The image obtained by the phase difference microscope 4 can be directly used for the processing described later, and the pre-processing can be performed by using a differential filter, so that the edge intensity in the image can be used to promote the processing described later with higher accuracy. As a differential filter, in addition to the first differential filter such as the sobel filter, the prewitt filter, and the canny filter, a second differential filter such as the laplacian filter can also be used. In addition, various methods such as morphology operations and detection of brightness changes contained in the image data can be used instead of using a differential filter.

[0072] The extraction pixel selection unit 21 selects pixels whose edge strength is greater than or equal to a predetermined value as extraction pixels for the image data from which the edge strength is extracted using the differential filter (step S3), assigns 1 to the selected pixels and assigns 0 to the non-selected pixels, thereby binarizing the image data (step S4, see Figure 4 ). Here, values ​​such as 0 and 1 are used for binarization, but other values ​​or symbols may be used for binarization. In addition, here, the same specified value is used to select the extracted pixel regardless of the position within the pixel, but it is also possible to use a specified value corresponding to the size of the brightness value of each pixel (for example, a pixel with an edge strength value that is greater than a specified ratio relative to the brightness value of the object pixel is set as an extracted pixel). Thus, even in a case where there are bright (large brightness value) and dark (small brightness value) parts in the image depending on the environment at the time of shooting, the extracted pixel can be appropriately selected. In addition, a threshold value that varies according to the dispersion of brightness values ​​in a specified range (for example, a range of 9 pixels × 9 pixels) centered on the pixel of interest in the entire image may be used. In addition, although the size of the bacteria is very small compared to the width and length of the flow path, in Figure 4 In order to show the existence of the target bacteria in an easily understandable manner, it is shown larger than the actual size.

[0073] There are two methods for processing the image data after binarization. When the inspector specifies the type of microfluidic device 100 used in the inspection at the beginning of the inspection (or when there is only one type of microfluidic device 100 that can be used), the flow path determination unit 22 determines the position of the flow path included in the observation image by template matching. First, this process is described.

[0074] The flow path determination unit 22 reads the template image 110 of the microfluidic device 100 from the storage unit 11. The template image 110 is as follows: Figure 5 (a) shows an image in which a shadow is added to a position other than the flow path.

[0075] Next, the flow path determination unit 22 overlaps the template image 110 with the observation image 110 acquired by the phase contrast microscope 4, gradually shifts the relative positions of the two images, and determines the position where the boundary of the flow path included in the observation image and the boundary of the flow path in the template image 110 have the highest degree of consistency. If the position where the boundary of the flow path included in the observation image and the boundary of the flow path in the template image 110 have the highest degree of consistency is determined, the flow path determination unit 22 determines the position where the boundary of the flow path included in the observation image and the boundary of the flow path in the template image 110 have the highest degree of consistency. Figure 5 As shown in (b), an image 104a (an image in which a shadow is added to the parts other than the branch flow paths 103a to 103e) is generated by superimposing the template image on the observation image, and the image of each branch flow path 103a to 103e is determined (step S5). Then, the image data of each branch flow path 103a to 103e is stored in the storage unit 11, and these images are displayed on the display unit 42.

[0076] In addition, it is also possible to use known structural information related to the flow path of the microfluidic device 100 (for example, five straight flow paths are formed) instead of using the above-mentioned template image 110, and extract the images of each branch flow path 103a~103e based on the consistency between the straight template filter and the image. Alternatively, when the microfluidic device 100 used is predetermined and the position set in the phase difference microscope 4 is positioned with high precision, it is also possible to automatically extract the area existing in the predetermined position of the observation image as the image of each branch flow path 103a~103e without using the template image and the template filter. Another method of determining the flow path will be described below.

[0077] If the positions of the branch flow paths 103a to 103e in the observation image are determined by the above step S5, the object determination unit 23 causes the characteristic pixel group detection unit 24 to operate to determine the target bacteria existing inside each branch flow path 103a to 103e. In this embodiment, the object determination unit 23 and the characteristic pixel group detection unit 24 cooperate to detect the object characteristic pixel group (that is, they function as the object characteristic pixel group detection unit in the present invention).

[0078] When the object to be tested is determined, the characteristic pixel group detection unit 24 detects a continuous specified number of extracted pixels or more from the binarized images of each branch flow path 103a to 103e as the object characteristic pixel group (step S6). The specified number can be any number that can separate the target bacteria from the noise inside the flow path. For example, it can be set to the number of pixels equivalent to the minimum size of the assumed target bacteria (for example, 5 pixels). Since noise data rarely exists continuously in multiple pixels, the noise contained in the binarized flow path image can be removed by this processing.

[0079] If the object characteristic pixel group is detected by the characteristic pixel group detection unit 24, the statistical feature calculation unit 26 determines the number of object characteristic pixel groups (the number of target bacteria) existing inside each branch flow path 103a~103e and the arrangement of pixels constituting each object characteristic pixel group (the shape of the target bacteria). Thus, the statistical feature associated with the number and shape of the object existing inside each branch flow path 103a~103e can be obtained (step S7). In addition, when there are multiple object characteristic pixel groups in one flow path, the statistical feature representing the average shape of the object is obtained by dividing the sum of the statistical feature of the object characteristic pixel groups corresponding to these multiple target bacteria by their number. As the statistical feature associated with the shape of the target bacteria, it can be set to the area (or area ratio) of the target bacteria, the perimeter of the target bacteria, the length of the major axis and the minor axis of the target bacteria, the linearity and roundness of the target bacteria, etc. The area ratio of the target bacteria can be obtained based on the ratio of the number of pixels constituting the object characteristic pixel group to the number of pixels in the flow path. The perimeter of the object bacteria can be calculated based on the length of the outer perimeter surrounding the object characteristic pixel group. The major axis length and minor axis length of the object bacteria can be calculated as the maximum and minimum values ​​of the straight line length of the rectangle surrounding the object characteristic pixel group. The linearity and circularity of the object bacteria can be calculated based on the aspect ratio of the shape of the object characteristic pixel group, a specified formula (for example, 4π×area÷(perimeter) 2 ) is obtained. As the statistical feature quantity, other known statistical quantities can be used appropriately. Here, the average of the statistical feature quantity is obtained, but in addition to the average value, the median value, the mode value, etc. can also be obtained.

[0080] If the statistical characteristic quantity calculation unit 26 obtains the statistical characteristic quantity of the target bacteria in each branch flow path 103a to 103e, the determination unit 27 uses the statistical characteristic quantity of the target bacteria in the branch flow path 103e into which the antibacterial solution is not introduced as a reference, and determines the state of the target bacteria in the branch flow paths 103a to 103d into which the antibacterial solutions a to d of different concentrations are introduced (step S8). The image based on the flow path, the statistical characteristic quantity obtained based on the image, and the determination result of the statistical characteristic quantity are displayed by the display unit 42.

[0081] In addition, information related to the morphological changes of the target bacteria may be output. For example, when the shape (linearity, roundness, etc.) of the target bacteria inside the branch flow paths 103a to 103d is different from the shape of the target bacteria in the branch flow path 103e, the change of the statistical feature quantity corresponding to the shape may be displayed on the display unit 42. Thus, information such as the presence or absence of the denaturation of the target bacteria caused by the action of the antibacterial drug may be obtained.

[0082] Here, refer to Figure 6Another method of determining the position of a flow path included in an observation image is described with reference to the flowchart of FIG.

[0083] There are many kinds of microfluidic devices that can be used for inspection, and when the type of the microfluidic device is not determined by the inspector, the flow path determination unit 22 causes the characteristic pixel group detection unit 24 and the flow path boundary pixel group selection unit 25 to operate in sequence to extract the image of the flow path. In subsequent processing, the flow path characteristic pixel group is detected by the coordinated operation of the flow path determination unit 22 and the characteristic pixel group detection unit 24.

[0084] Among the extracted pixels selected by the extracted pixel selection unit 21, there are mixed pixels located at the boundary of the flow path, pixels where the target bacteria exist, and pixels where noise occurs. Therefore, the characteristic pixel group detection unit 24 detects more than a prescribed number of extracted pixels arranged continuously in a linear shape from the binarized image data as a flow path characteristic pixel group (step S5-1). Here, the prescribed number is pre-set to a number that can infer the shape of the flow path characteristic pixel group based on the positions of this number of pixels. The level to which the prescribed number should be set varies depending on the shape of the flow path. In the present embodiment, in order to obtain a straight line equation representing the flow path characteristic pixel group based on the positions (coordinates) of the pixels constituting the flow path characteristic pixel group, it is sufficient to include at least 2 extracted pixels, but if the prescribed number is too small, the accuracy of the straight line equation is poor. On the other hand, although the more the prescribed number is, the more accurately the linear equation representing the flow path feature pixel group can be obtained, if the prescribed number is too large, some of the pixels located at the flow path boundary do not have contrast and are not selected as extraction pixels (that is, the number of consecutive extraction pixels is small), and the extraction pixels located at the flow path boundary will not be detected as the flow path feature pixel group. Considering these situations, the prescribed number is preferably set to the level of the number of pixels corresponding to the length of one tenth of the image size (the short side in the case of a rectangular image, the diameter in the case of a circular image).

[0085] Next, the flow path boundary pixel group selection unit 25 assigns a number m (m is an integer greater than 1 and less than n) to the n flow path characteristic pixel groups detected by the characteristic pixel group detection unit 24, and obtains a linear equation L representing each flow path characteristic pixel group. m (Step S5-2) The linear equation L representing the flow path feature pixel group m It refers to an approximate straight line based on the coordinates of a plurality of extracted pixels constituting the characteristic pixel group. In this embodiment, in order to obtain the straight line equation L representing each flow path characteristic pixel group, m Instead use the Hough transform.

[0086] Hough transform is one of the methods used in the field of image processing to extract features of an image. In Hough transform, the equation of a line is expressed in terms of the distance r from the origin to the normal line of the line and the angle θ between the normal line and the x-axis on a two-dimensional plane (xy plane). In Hough transform, the equation of a line represented by the following equation (1) is used.

[0087] r=xcosθ+ysinθ…(1)

[0088] First, the equation of the line obtained using formula (1) is obtained for one of the extracted pixels constituting the flow path feature pixel group. Since only one coordinate value is specified at this stage, the equation of the line is not uniquely determined, but is a function of r and θ. For the extracted pixels constituting the flow path feature pixel group, the function of r and θ is obtained using the coordinates of the extracted pixel. Then, the function of r and θ obtained according to the number of extracted pixels is projected into the Hough space set as a double axis, and the values ​​of r and θ with a large number of projections (highest consistency) are obtained. The equation of the line representing the flow path feature pixel group is thus determined. In the case where the extracted pixels constituting the flow path feature pixel group are located on a straight line, the functions of r and θ respectively obtained for these extracted pixels intersect at one point, and the straight line can be strictly determined based on the values ​​of r and θ corresponding to the intersection. However, in actual images, since the flow path feature pixel group spans multiple pixels in its width direction or becomes stepped locally, the extracted pixels are rarely completely located on a straight line. Therefore, the values ​​of r and θ with a large number of projections in the Hough space are used to determine the equation of the line corresponding to the flow path feature pixel group. In this way, the following equation (2) can be obtained as the linear equation L for the mth flow path feature pixel group: m :

[0089] a m x+b m y=1(a m , b m is the coefficient)…(2)

[0090] In this embodiment, the Hough transform is used to find the linear equation L corresponding to each of the flow path feature pixel groups. m , but other methods can also be used to find the linear equation L m For example, the least square method can be used to find an approximate straight line for the coordinates of each extraction pixel constituting the flow path feature pixel group.

[0091] Next, the flow path boundary pixel group selection unit 25 selects the linear equation L obtained for each flow path characteristic pixel group. m The slope of the line is determined. The linear equation L represented by the above formula (2) is m The unit vector v of the slope m , as expressed by the following formula (3).

[0092] vm =(-b m / l m ,a m / l m )(where l m =(a m 2 +b m 2 ) 1 / 2 )…(3)

[0093] For all straight line equations L m Get the slope v m Then, find the median value v of the slope ctr Then, a flow path characteristic pixel group (outlier) corresponding to a straight line having a slope that does not satisfy the following equation (4) is excluded from the processing target (step S5-3).

[0094] | <v m ·v ctr >|≧α (α is a value greater than or equal to 0 and less than 1)…(4)

[0095] Formula (4) is the linear equation L corresponding to each flow path feature pixel group m The slope v m With all slopes v m The median value v ctr In addition, α is a real number greater than 0 and less than 1. That is, formula (4) means relative to the average value v ctr The parallelism of the straight line with a slope of is greater than the specified value (straight line L m The slope v m The slope of the equation is close to the slope of other straight line equations). In the microfluidic device 100 of this embodiment, since a plurality of branch flow paths 103a to 103e are formed in parallel in the observation area 104, the straight lines corresponding to the flow path characteristic pixel groups located at the boundaries of each flow path should be parallel to each other. Therefore, by excluding the flow path characteristic pixel groups corresponding to the straight lines having slopes that do not satisfy the above equation (4) from the processing objects, the flow path characteristic pixel groups that are considered not to be the boundaries of the flow path (such as the flow path characteristic pixel groups corresponding to the positions of the target bacteria) can be excluded.

[0096] Here, the linear equation L corresponding to each flow path feature pixel group is used. m The median value v ctr , but it does not necessarily have to be the median value, and the following appropriate value can also be used: Find the linear equation L corresponding to each feature pixel group m The slope v m The average value of the linear equation L corresponding to each feature pixel group m The slope v mThe distribution of the image is obtained by removing the average of the predetermined number of slopes corresponding to the ends of the distribution. In addition, when the approximate slope of the straight line corresponding to the flow path boundary can be estimated (the slope of the flow path in the observation image), the estimated value can also be used.

[0097] After the above processing, the remaining k (k≦n) linear equations L p (p is an integer greater than 1 and less than k), and the distance between the straight line and another straight line adjacent to the straight line in a specific direction (for example, the positive direction of the x-axis) when viewed from the respective straight lines is calculated (step S5-4). p are not strictly parallel to each other, so the spacing distance d between the straight lines is specified as follows p .

[0098] In this embodiment, if Figure 7 As shown in (a), for the linear equations L p and the adjacent straight line equation L p+1 The straight line represented by the image is obtained by finding the straight line passing through the center C of the observation image and parallel to the y-axis (the straight line equation L y ). Then, the distance between these intersection points is set as the distance d between the two straight lines. p In addition, if Figure 7 As shown in (b), for the linear equations L p and the adjacent straight line equation L p+1 The straight line represented by the image is obtained by finding the straight line that passes through the center C of the observation image and is parallel to the x-axis (the straight line equation L x ), the distance between these intersection points can also be set as the distance d between the two straight lines. p Furthermore, if Figure 7 As shown in (c), for the straight lines L p and the adjacent straight line L p+1 The straight line represented by the image is obtained by calculating the equation L of the straight line passing through the center C of the observed image. p The distance between these intersection points can also be set as the distance d between the two straight lines. p .

[0099] Next, the distance d between the k-1 straight lines is calculated. p , a histogram is generated with a predetermined layer width (step S5-5). The channel width and channel spacing of the branch channels 103a to 103e formed in the microfluidic device 100 of this embodiment are fixed. Therefore, in the generated histogram, the interval distance d should be equivalent to the channel width. p The largest, followed by the interval distance d which is equivalent to the flow path interval pTherefore, the interval distance d related to the k-1 straight lines is determined respectively. p Whether to be included in the maximum hierarchy.

[0100] In the microfluidic device 100 of this embodiment, five branch flow paths 103a to 103e are formed in parallel. Figure 8 As shown in (a), there are 4 data points corresponding to the distance between the flow paths, 5 data points corresponding to the flow path width, and no data points at other positions. If such a histogram is obtained, it is determined that the interval distance d obtained for the five straight lines is p For the interval distance d that is judged to be included in the maximum hierarchy, p , by extracting the corresponding straight line L p , the straight line L adjacent to this straight line p+1 By selecting a group of pixels constituting the boundary of the flow path (flow path boundary pixel group), it is possible to select a pixel group constituting the boundary of the flow path.

[0101] However, in reality, even after the above-mentioned processing, there may be a case where straight lines other than the straight lines constituting the flow path boundary (straight lines originating from the target bacteria or from noise) are included. In this case, it becomes, for example Figure 8 (b) and Figure 8 (c) shows the histogram. Even in this case, the straight lines constituting the boundaries of the flow paths are placed at fixed intervals corresponding to the flow path width and the distance between the flow paths, while the other straight lines are placed randomly. Therefore, if the largest layer in the histogram is selected, the flow path width ( Figure 8 (b)) or the distance between flow paths ( Figure 8 (c)) is either one of them. Thus, it is possible to extract the corresponding straight line L p , the straight line L adjacent to this straight line p+1 The pixel group constituting the boundary of the flow path (flow path boundary pixel group) is selected from the group (step S5-6) (step S5-7).

[0102] If the flow path boundary pixel group is selected through the above process, then Fig. 9 (a) or Fig. 9 As shown in (b), the position of the boundary of the flow path in the image can be clearly observed. Fig. 9 (a) The state shown (according to Figure 8 (a) or by Figure 8 (b)), two straight lines from the end of the image are grouped as a group and the area between them (the shaded area) is used as the flow path. Fig. 9 (b) The state shown (according to Figure 8In the state obtained by the histogram of (c), when two straight lines are grouped as one group starting from the end of the image, the area sandwiched between them (the shaded area) is not a flow path.

[0103] Therefore, after determining the position of the boundary of the flow path in the observation image, the flow path width is compared between the following two cases: the flow path width when two lines are selected from the end as a group and the area sandwiched between them is regarded as the flow path; and the flow path width when two lines are selected in sequence from the second straight line from the end as a group and the area sandwiched between them is regarded as the flow path. Figure 1 In order to show the flow path in an easy-to-understand manner, the flow path width is shown as narrow, but in the microfluidic devices actually used, the flow path width is wider than the flow path interval. Therefore, the side judged to be wider as a result of the above comparison is set as the flow path. In the case where the flow path width is judged to be wider in the latter case (the case where 2 lines are selected in sequence as a group starting from the second straight line at the end), the state is achieved that one line of the flow path boundary pixel group located in a straight line is left at each end of the image. Determine whether a flow path of the same width as other flow paths can be added to the outside of each of the straight lines left at the two ends (that is, whether there is a space equivalent to the flow path width left outside the straight line), and if possible, add a straight line equivalent to the boundary of the flow path (in the case of Fig. 9 (c) is indicated by a single-dot chain line). This adds a flow path.

[0104] If the position of the flow path in the observed image is determined through the above processing, the flow path image extraction unit 22 determines the position of each flow path 103a to 103e included in the observed image (step S5, step S5-8). Then, the image data of each flow path 103a to 103e is saved in the storage unit 11, and these images are displayed on the display unit 42. At this time, the image of the determined flow path position can be extracted as it is, but the contrast of the flow path boundary will affect the image of the flow path. In order to prevent the contrast from being mistaken for the target bacteria, the area inside the determined flow path position can also be extracted as the image of the flow path, and the inner area is separated from the determined flow path position by a distance corresponding to a predetermined number of pixels.

[0105] Next, the microfluidic device observation apparatus 1 a according to Example 2 will be described. Fig.10 2 is a diagram showing the main components of the microfluidic device observation apparatus 1a of the embodiment. The same components as those of the microfluidic device observation apparatus 1 of the first embodiment are given the same reference numerals and their description is omitted.

[0106] The control and processing unit 1a of the microfluidic device observation device of Example 2 is partially different from the microfluidic device observation device 1 of Example 1. The control and processing unit 10a has the function of controlling the operation of the dispensing unit 2, the incubator 3 and the phase contrast microscope 4, and the function of processing the image obtained by the phase contrast microscope 4, similarly to the control and processing unit 10 of Example 1. The control and processing unit 10a has a storage unit 11a, and has an extraction pixel selection unit 21 and a flow path determination unit 22 as functional modules. Furthermore, a recognition unit 28 constructed by machine learning described later is also installed. The entity of the control and processing unit 10a is a general computer, and the above-mentioned functional modules are visualized by executing the microfluidic device observation program 20a by its processor. In addition, the control and processing unit 10a is connected to an input unit 41 for the inspector to input appropriate information, and a display unit 42 for displaying an image of the flow path of the microfluidic device 100, etc.

[0107] The storage unit 11a stores the template image 110 described in Example 1 and a large number of reference images of the test object. The reference images of the test object are images of the same test object as in this embodiment, i.e., the target bacteria, after being acted upon by a variety of antibiotics at various concentrations. These images have been analyzed, and the statistical feature quantities associated with the number and shape of the target bacteria contained in each image have been predetermined. In addition, in addition to the images of the target bacteria of standard shapes, the reference images of the test object also include images of the target bacteria whose shapes have changed due to denaturation by antibiotics, and the statistical feature quantities of these images are also calculated.

[0108] The microfluidic device observation apparatus 1a of Example 2 is characterized in that it calculates the statistical feature value of the target bacteria existing inside the flow path and determines the state of the target bacteria. In Example 2, the recognition unit 28 is generated by machine learning using a large number of reference images of the test object stored in the storage unit 11 as learning data. As such a machine learning method, methods such as deep learning, support vector machine (SVM) and random forest (RandomForest) are proposed.

[0109] In the microfluidic device observation apparatus 1a of Example 2, before starting the inspection of the target bacteria, a recognition unit 28 for determining the number and shape of the target bacteria based on the input image is pre-constructed by using machine learning as learning data, using an image of the flow path of the microfluidic device 100 and the statistical feature values ​​of the target bacteria corresponding to the image.

[0110] In Example 2, the microfluidic device 100 is photographed with a phase contrast microscope 4 and the image data is processed with a differential filter to obtain edge intensity, and the image data is binarized. The above processing is the same as that of the microfluidic device observation device 1 of Example 1. In addition, in the process of determining the position of the flow path in the observed image, any of the methods described in Example 1 is used.

[0111] When the position of each branch flow path 103a to 103e included in the observation image is determined by the flow path determination unit 22, the image data of each branch flow path 103a to 103d is input to the recognition unit 28. The recognition unit 28 determines the number and shape (e.g., linearity, circularity) of the sample contained in the image of each branch flow path 103a to 103e, and outputs these as the statistical feature value of the target bacteria. When there are multiple target bacteria in the branch flow paths 103a to 103e, the average of each statistical feature value for these multiple target bacteria is output. As the statistical feature value, in addition to linearity and circularity, the area (or area ratio) of the target bacteria, the perimeter of the target bacteria, the major axis length and minor axis length of the target bacteria, or other known statistics can also be used. When using these statistical feature values, the recognition unit 28 can be pre-constructed by machine learning that outputs the corresponding statistical feature value based on the input of the image of the flow path containing the target bacteria. Here, the average of the statistical feature value is calculated, but in addition to the average value, the median value, the mode value, etc. can also be calculated.

[0112] If the recognition unit 28 obtains the average of the statistical feature values ​​indicating the number and shape of the target bacteria, then, similarly to Example 1, the statistical feature values ​​of the target bacteria in the branch flow path 103e into which the antibacterial solution is not introduced are used as a reference, and the states (sensitivities to the antibacterial solutions a to d containing antibacterial drugs at different concentrations) of the target bacteria in the branch flow paths 103a to 103d into which the antibacterial solutions a to d of different concentrations are respectively introduced are determined. The display unit 42 displays an image based on the flow path, the statistical feature values ​​obtained based on the image, and the determination result of the statistical feature values.

[0113] The above embodiments are merely examples and can be modified appropriately according to the spirit of the present invention. In the above embodiments, the case of testing the sensitivity of target bacteria to antibiotics is described, but the same microfluidic device observation apparatus as described above can also be used in tests for other purposes.

[0114] In the above embodiment, the method of detecting the boundary of the flow path and extracting the characteristic pixels of bacteria by the same method is described, but different methods may be used. For example, when the boundary of the flow path is detected by template matching, the shape (including information such as area) can be detected by extracting the characteristic pixels of bacteria by machine learning, for example.

[0115] In the above embodiment, the case where 5 straight branch flow paths 103a~103e are formed in the microfluidic device 100 is used as an example for explanation, but the number of flow paths can also be appropriately changed according to the number of antibacterial liquids (types of concentrations) used during the inspection. In addition, the shape of the flow path is not limited to a straight line, but can also be a curved shape. The curved shape can be, for example, a circular arc (including an elliptical arc) or an S shape (a serpentine shape). Even if the shape of the flow path is a curved shape, the Hough transform described in this specification can be applied. In particular, in the case where the shape of the flow path is a circular arc or an elliptical arc, it is sufficient to use a mathematical formula representing a circle or an ellipse instead of the Hough transform of the above embodiment to obtain the shape of the feature pixel group, and use the parallelism of the curve instead of the parallelism of the straight line in the above embodiment. For example, a parallel curve consideration method can be used, and the parallel curve is defined as a curve with a fixed distance from each point on the curve in the normal direction relative to a certain curve.

[0116] In both of the above-mentioned embodiments 1 and 2, a phase contrast microscope is used as the imaging unit, but the imaging unit is not limited to any type as long as it can obtain an image showing contrast at the boundary of the flow path formed in the microfluidic device. For example, an optical microscope, an optical camera, or a differential interference microscope can be used.

[0117] In the process of determining the position of the flow path described in the first embodiment, after determining the position of the flow path in the observation image, the flow path is determined by comparing the flow path width of the following two cases: the flow path width when two lines are selected from the end of the image as a group and the area sandwiched therebetween is regarded as the flow path; and the flow path width when two lines are selected from the second straight line from the end as a group and the area sandwiched therebetween is regarded as the flow path. However, the image in which the position of the flow path in the observation image is determined may be displayed on the display unit 42, and the inspector may select which one is the flow path by operating the input unit 41. Alternatively, the flow path may be determined without performing the above comparison, and the image of the flow path extracted by the flow path determination unit 22 may be displayed on the display unit 42. If the inspector confirms and determines that it is not the flow path, the inspector may instruct to perform a reversal operation (an operation in which the area sandwiched between two straight lines different from the group of straight lines previously used as the boundary of the flow path is regarded as the flow path and the flow path determination unit 22 extracts the image).

[0118] In the above-mentioned second embodiment, the observation image in which the position of the flow path is determined is input to the recognition unit 28, but the image data of the microfluidic device 100 (image data in which the position of the flow path is not determined) may be input to the recognition unit 28 as it is, and the position of the flow path is determined by the recognition unit 28. In this case, the recognition unit 28 may be configured by machine learning using image data of a microfluidic device in which flow paths of various numbers and shapes are formed and data in which the position of each flow path is determined as learning data.

[0119] Description of Reference Numerals

[0120] 1.1a Microfluidic device observation device

[0121] 2 sub-notes

[0122] 3. Incubator

[0123] 4 Phase contrast microscopy

[0124] 10a Control and processing unit

[0125] 10.10a Control and processing unit

[0126] 11. 11a Storage unit

[0127] 110 Template Image

[0128] 20.20a Microfluidic Device Observation Program

[0129] 21 Extraction pixel selection unit

[0130] 22 Flow path determination unit

[0131] 23. Inspection object identification department

[0132] 24 Feature pixel group detection unit

[0133] 25. Flow path boundary pixel group selection unit

[0134] 26 Statistical feature quantity calculation unit

[0135] 27 Judgment Department

[0136] 28 Identification Department

[0137] 41 Input section

[0138] 42 Display unit

[0139] 100 Microfluidic Devices

[0140] 101 Import port

[0141] 102a~102e Air port

[0142] 103 Bacterial liquid introduction flow path

[0143] 103a~103e Branch flow path

[0144] 104 Observation Area

[0145] 110 Template images.

Claims

1. A microfluidic device observation device for observing an object to be inspected existing inside a plurality of flow paths provided in a microfluidic device, characterized in that: have: A photographing unit, for acquiring an image of the microfluidic device; an extraction pixel selection unit that selects an extraction pixel from a plurality of pixels constituting the acquired image according to a predetermined criterion based on brightness values ​​of the plurality of pixels; a flow path determining unit that determines positions of the plurality of flow paths included in the image; a detection unit for detecting a characteristic pixel group of an object, detecting a predetermined number or more of the extraction pixels that are continuous and exist inside the determined plurality of flow paths as a characteristic pixel group of an object; The statistical feature quantity calculation unit calculates the statistical feature quantity of the object characteristic pixel group based on the number and / or shape of the object characteristic pixel group. The flow path identification unit identifies positions of boundaries between the plurality of flow paths included in the image, and identifies a region sandwiched between adjacent boundaries as the flow path. The flow path is formed in a straight line. The flow path determining unit comprises: a flow path characteristic pixel group detection unit, detecting, as flow path characteristic pixel groups, a plurality of extraction pixel groups composed of a predetermined number or more of the extraction pixels respectively arranged continuously in a linear shape, a plurality of extraction pixel groups whose mutual parallelism is equal to or greater than a predetermined parallelism; a flow path boundary pixel group selection unit, which selects, from a plurality of flow path characteristic pixel groups, a group of characteristic pixel groups whose interval distance corresponds to the width of the flow path as the flow path boundary pixel group; The flow path image extraction unit specifies the position of the boundary of one or more flow paths based on the position of the flow path boundary pixel group in the image and extracts the image of each flow path.

2. The microfluidic device observation device according to claim 1, characterized in that: When a plurality of the object characteristic pixel groups exist in one flow channel, the statistical characteristic quantity calculation unit obtains a representative value of the statistical characteristic quantity of the plurality of the object characteristic pixel groups.

3. The microfluidic device observation device according to claim 1, characterized in that: The method further includes a determination unit that determines the state of the object to be inspected based on a result of comparing the statistical feature amount obtained for one of the plurality of flow paths with the statistical feature amounts obtained for the other flow paths.

4. The microfluidic device observation device according to claim 1, characterized in that: The plurality of flow paths having the same width are arranged in parallel in at least a portion of the microfluidic device.

5. The microfluidic device observation device according to claim 1, characterized in that: The extraction pixel selection unit obtains the edge strength of the image using a differential filter.

6. The microfluidic device observation device according to claim 1, characterized in that: The flow path feature pixel group detection unit obtains an approximate straight line corresponding to the positions of pixels constituting the extracted pixel group for each extracted pixel group, and detects an extracted pixel group whose slope difference with the slope of other approximate curves is less than a predetermined value as a feature pixel group.

7. The microfluidic device observation device according to claim 6, characterized in that: The flow path characteristic pixel group detection unit obtains the approximate straight line through Hough transform.

8. The microfluidic device observation device according to claim 1, characterized in that: The flow paths are arranged in a plurality and are parallel to each other. The flow path boundary pixel group selection unit selects a pixel group having an equal interval between two adjacent flow path feature pixel groups as the flow path boundary pixel group.

9. The microfluidic device observation device according to claim 8, characterized in that: The flow path boundary pixel group selection unit generates a histogram related to the interval distance, and selects a pixel group belonging to the largest level of the histogram as the flow path boundary pixel group.

10. A microfluidic device observation device for observing an object to be inspected existing inside a plurality of flow paths provided in a microfluidic device, characterized in that: have: A photographing unit, for acquiring an image of the microfluidic device; a flow path determining unit that determines positions of the plurality of flow paths included in the image; The recognition unit is constructed by machine learning using an image of a microfluidic device in which a test object exists inside a flow path and the number and / or shape of the test object in the image as learning data, and when the acquired image is input, the number and / or shape of the test object existing inside the determined multiple flow paths is determined, The flow path identification unit identifies positions of boundaries between the plurality of flow paths included in the image, and identifies a region sandwiched between adjacent boundaries as the flow path. The flow path is formed in a straight line. The flow path determining unit comprises: A flow path characteristic pixel group detection unit detects, as flow path characteristic pixel groups, a plurality of extraction pixel groups composed of a predetermined number or more of extraction pixels respectively arranged continuously in a linear shape, and a plurality of extraction pixel groups whose mutual parallelism is equal to or greater than a predetermined parallelism. a flow path boundary pixel group selection unit, which selects, from a plurality of flow path characteristic pixel groups, a group of characteristic pixel groups whose interval distance corresponds to the width of the flow path as the flow path boundary pixel group; The flow path image extraction unit specifies the position of the boundary of one or more flow paths based on the position of the flow path boundary pixel group in the image and extracts the image of each flow path.

11. The microfluidic device observation device according to claim 10, characterized in that: When a plurality of the samples exist in one flow channel, the identification unit obtains a representative value of a statistical feature quantity associated with the number and / or shape of the plurality of samples.

12. The microfluidic device observation device according to claim 10, characterized in that: The recognition unit further determines the state of the object to be detected existing inside each flow path based on the determined number and / or shape of the object to be detected.

13. The microfluidic device observation device according to claim 10, characterized in that: The plurality of flow paths having the same width are arranged in parallel in at least a portion of the microfluidic device.

14. The microfluidic device observation device according to claim 10, characterized in that: The flow path feature pixel group detection unit obtains an approximate straight line corresponding to the positions of pixels constituting the extracted pixel group for each extracted pixel group, and detects an extracted pixel group whose slope difference with the slope of other approximate curves is less than a predetermined value as a feature pixel group.

15. The microfluidic device observation device according to claim 14, characterized in that: The flow path characteristic pixel group detection unit obtains the approximate straight line through Hough transform.

16. The microfluidic device observation device according to claim 10, characterized in that: The flow paths are arranged in a plurality and are parallel to each other. The flow path boundary pixel group selection unit selects a pixel group having an equal interval between two adjacent flow path feature pixel groups as the flow path boundary pixel group.

17. The microfluidic device observation device according to claim 16, characterized in that: The flow path boundary pixel group selection unit generates a histogram related to the interval distance, and selects a pixel group belonging to the largest level of the histogram as the flow path boundary pixel group.

18. A method for observing a microfluidic device, the method being used for observing bacteria introduced together with a drug into a plurality of flow paths disposed in the microfluidic device, characterized in that: include: acquiring an image of the microfluidic device; Based on brightness values ​​of a plurality of pixels constituting the acquired image, selecting an extraction pixel from the plurality of pixels according to a prescribed criterion; determining positions of boundaries of the plurality of flow paths included in the image, and determining regions sandwiched between adjacent boundaries as the flow paths; Detecting the determined extraction pixels existing inside the plurality of flow paths and being continuous and having a number greater than a predetermined number as a characteristic pixel group of the object under test; Based on the number and / or shape of the characteristic pixel group of the object under test, a statistical characteristic value of the characteristic pixel group of the object under test is obtained. The flow path is formed in a straight line. In which, when determining the position of the boundary of the flow path, multiple extraction pixel groups composed of more than a specified number of extraction pixels arranged continuously in a linear manner and with a parallelism greater than or equal to a predetermined degree are detected as flow path feature pixel groups, and a group of feature pixel groups having an interval distance corresponding to the width of the flow path is selected from the multiple flow path feature pixel groups as the flow path boundary pixel group, and the position of the boundary of one or more flow paths is determined based on the position of the flow path boundary pixel group in the image, and the image of each flow path is extracted.

19. A method for observing a microfluidic device, the method being used for observing bacteria introduced together with a drug into a plurality of flow paths disposed in the microfluidic device, characterized in that: Include: acquiring an image of the microfluidic device; determining positions of boundaries of the plurality of flow paths included in the image, and determining regions sandwiched between adjacent boundaries as the flow paths; determining the number and / or shape of the bacteria existing inside the plurality of determined flow paths based on the input of the acquired image, The flow path is formed in a straight line. In which, when determining the position of the boundary of the flow path, multiple extraction pixel groups composed of more than a specified number of extraction pixels arranged continuously in a linear manner and with a parallelism greater than or equal to a predetermined degree are detected as flow path feature pixel groups, and a group of feature pixel groups having an interval distance corresponding to the width of the flow path is selected from the multiple flow path feature pixel groups as the flow path boundary pixel group, and the position of the boundary of one or more flow paths is determined based on the position of the flow path boundary pixel group in the image, and the image of each flow path is extracted.

Images