Detection device

Abstract

To provide a detection device capable of improving detection accuracy.SOLUTION: The detection device includes an optical sensor including a plurality of light detection elements arranged in a planar shape, a translucent detection target installation unit for installing a plurality of detection targets, and a control circuit that controls the optical sensor, wherein the optical sensor acquires image data every predetermined period from the start of measurement, the control circuit performs predetermined processing on each of the plurality of pieces of image data, extracts a second contour of at least one region exceeding a predetermined threshold value with respect to second image data acquired in a second period in which a predetermined period elapses from a first period, when there is a second contour not including the first coordinates among the second contours, the second coordinates corresponding to the second contour not including the first coordinates are calculated, the second identification information corresponding to the second contour not including the first coordinates is newly assigned, and the total number of the first identification information labeled with the first image data and the second identification information newly labeled with the second image data is calculated.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a detection device. [Background technology]

[0002] Patent Document 1 discloses an image acquisition device that includes an optical sensor, a container containing microorganisms and a culture medium, and a light source, and that acquires images over time that show the growth of the microorganisms in the container. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-033430 Summary of the Invention [Problem to be solved by the invention]

[0004] In such a detection device, there is a demand for improving the detection accuracy of the object to be detected.

[0005] An object of the present invention is to provide a detection device that can improve detection accuracy. [Means for solving the problem]

[0006] A detection device according to one embodiment of the present disclosure includes an optical sensor including a plurality of light detection elements arranged in a planar manner, a translucent object mounting portion for mounting a plurality of objects to be detected, and a control circuit for controlling the optical sensor, wherein the optical sensor acquires image data at predetermined intervals from the start of measurement, and the control circuit performs predetermined processing on each of the plurality of image data, extracting a first contour of at least one region of first image data acquired during a first period that exceeds a predetermined threshold, calculating first coordinates corresponding to the first contour, and labeling the first identification information corresponding to the first coordinates, extracting a second contour of at least one region of second image data acquired during a second period that is a predetermined period after the first period, and when there is a second contour among the second contours that does not include the first coordinates, calculating second coordinates corresponding to the second contour that does not include the first coordinates, and newly assigning second identification information corresponding to the second contour that does not include the first coordinates, and calculating the total number of the first identification information labeled in the first image data and the second identification information newly labeled in the second image data. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a cross-sectional view schematically showing a detection device according to an embodiment. [Figure 2] FIG. 2 is a block diagram illustrating an example of the configuration of the detection device according to the embodiment. [Figure 3] FIG. 3 is a circuit diagram showing an optical sensor of the detection device according to the embodiment. [Figure 4] FIG. 4 is an explanatory diagram for explaining a method for detecting a detection target by the detection device according to the embodiment. [Figure 5] FIG. 5 is a block diagram showing an example of the configuration of the sensor control circuit according to the embodiment. [Figure 6] FIG. 6 is a flowchart for explaining a method for extracting a contour corresponding to an object to be detected based on image data. [Figure 7]FIG. 7 is a flowchart for explaining various processes performed on the contour extracted in FIG. 6 and image output. [Figure 8] FIG. 8 is an explanatory diagram for explaining a method of calculating differential image data. [Figure 9] FIG. 9 is an explanatory diagram for explaining a method for extracting a contour. [Figure 10] FIG. 10 is an explanatory diagram for explaining a method of processing coordinates and labeling for contours extracted from image data. [Figure 11] FIG. 11 is a schematic diagram showing an example of an output image. [Figure 12] FIG. 12 is a schematic diagram illustrating an example of the configuration of a detection system. [Figure 13] FIG. 13 is a cross-sectional view schematically showing an inspection unit included in the detection system. DETAILED DESCRIPTION OF THE INVENTION

[0008] Modes (embodiments) for carrying out the present disclosure will be described in detail with reference to the drawings. The present disclosure is not limited to the contents described in the following embodiments. Furthermore, the components described below include those that can be easily imagined by a person skilled in the art and those that are substantially identical. Furthermore, the components described below can be combined as appropriate. Note that the disclosure is merely an example, and any appropriate modifications that a person skilled in the art can easily conceive while maintaining the gist of the present disclosure are naturally included within the scope of the present disclosure. Furthermore, for clarity of explanation, the drawings may schematically depict the width, thickness, shape, etc. of each part compared to the actual embodiment. However, these are merely examples and are not intended to limit the interpretation of the present disclosure. Furthermore, in this disclosure and each figure, elements similar to those described above with reference to the previous figures may be designated by the same reference numerals, and detailed descriptions may be omitted as appropriate.

[0009] In the present disclosure, when expressing an aspect of placing another structure on top of a certain structure, the term "on top" is used, unless otherwise specified, to include both a case in which another structure is placed directly on top of a certain structure so as to be in contact with the certain structure, and a case in which another structure is placed above a certain structure via yet another structure.

[0010] (Embodiment) Fig. 1 is a cross-sectional view schematically illustrating a detection device according to an embodiment. As shown in Fig. 1, the detection device 1 includes an optical sensor 10, an optical filter layer 50, a container 110 for accommodating a detection target 100, and a light source 80. The container 110 (detection target 100) is disposed between the optical sensor 10 and the light source 80. In this embodiment, the detection device 1 is arranged in the following order: optical sensor 10, optical filter layer 50, container 110 (detection target 100), and light source 80. However, the present invention is not limited to this, and the detection device 1 may be arranged in the following order: light source 80, container 110 (detection target 100), optical filter layer 50, and optical sensor 10.

[0011] The detection object 100 is a minute object such as bacteria. When bacteria or the like are cultured on a culture medium 102 and grow into a visible mass, this mass may be referred to as a colony. The detection device 1 is a biosensor that detects minute objects such as bacteria. Note that the detection object 100 is not limited to bacteria, and may be other minute objects such as cells.

[0012] The container 110 includes a container body 111 and a cover member 112. The container 110 is, for example, a petri dish. The container 110 is translucent. A culture medium 102 is accommodated in the container body 111, and the detectable substances 100 are cultured in the culture medium 102. In other words, the container 110 (at least the container body 111 of the container body 111 and the cover member 112) is a translucent detectable substance installation portion for mounting multiple detectable substances 100 thereon.

[0013] In this embodiment, the container 110 is arranged with the container body 111 on the lower side and the cover member 112 on the upper side. This is not limiting, and the container 110 may be arranged upside down. That is, the container 110 may be arranged with the container body 111 on the upper side and the cover member 112 on the lower side. In this case, the detectable substance 100, such as bacteria, is placed on the upper side of the culture medium 102 and cultured, and when imaging the detectable substance 100, the container 110 is arranged upside down so that the detectable substance 100 is arranged below the culture medium 102. The detectable substance 100 and the culture medium 102 to be detected are contained in the container 110 and arranged between the optical sensor 10 and the light source 80.

[0014] The optical sensor 10 is a detection device including a plurality of photodiodes 30 arranged in a plane. Each photodiode 30 is a light detection element that outputs an electrical signal in response to light irradiated thereon. More specifically, the photodiodes 30 are PIN (Positive Intrinsic Negative) photodiodes using inorganic semiconductors or OPD (Organic Photodiodes) using organic semiconductors.

[0015] The optical filter layer 50 is a light directivity control element disposed between a plurality of light-emitting elements 82 (light source 80) and a plurality of photodiodes 30 (optical sensors 10). More specifically, the optical filter layer 50 is provided between the plurality of photodiodes 30 of the optical sensor 10 and the container 110. The optical filter layer 50 is disposed opposite the plurality of photodiodes 30 of the optical sensor 10. The optical filter layer 50 is an optical element that transmits, toward the photodiodes 30, components of the light emitted from the plurality of light-emitting elements 82 that travel in a direction perpendicular to the optical sensor 10. The optical filter layer 50 is also called a collimating aperture or a collimator. Alternatively, the optical filter layer 50 may be a louver or a microlens.

[0016] The light source 80 includes a light source substrate 81 and a plurality of light-emitting elements 82. The plurality of light-emitting elements 82 are point light sources provided corresponding to the plurality of photodiodes 30 of the optical sensor 10. The plurality of light-emitting elements 82 are provided on the light source substrate 81 and arranged opposite the plurality of photodiodes 30 of the optical sensor 10. Each of the plurality of light-emitting elements 82 is formed, for example, by a light-emitting diode (LED: Light Emitting Diode).

[0017] Light emitted from the light emitting element 82 passes through the cover member 112, the culture medium 102, the container body 111, and the optical filter layer 50, and is irradiated onto the multiple photodiodes 30 of the optical sensor 10. The amount of light irradiated onto the multiple photodiodes 30 differs between the area overlapping with the object to be detected 100 and the area not overlapping with the object to be detected 100. This allows the optical sensor 10 to capture an image of the object to be detected 100.

[0018] Fig. 2 is a block diagram showing an example of the configuration of a detection device according to an embodiment. As shown in Fig. 2, the detection device 1 further includes a control circuit 70 that controls the optical sensor 10 and the light source 80. The control circuit 70 controls the detection operation of the object 100 by the optical sensor 10 in synchronization (or asynchronously) with the lighting operation of the light emitting element 82 by the light source 80. The control circuit 70 is configured with, for example, an MCU (Micro Control Unit), RAM, EEPROM, ROM, etc.

[0019] The optical sensor 10 includes an array substrate 2, a plurality of sensor pixels 3 (photodiodes 30) formed on the array substrate 2, a first gate line driving circuit 15A, a second gate line driving circuit 15B, a signal line driving circuit 16A, and a detection circuit 11.

[0020] The array substrate 2 is formed using a substrate 21 as a base. Each of the sensor pixels 3 includes a photodiode 30, a plurality of transistors, and various wirings. The array substrate 2 on which the photodiodes 30 are formed is a drive circuit substrate that drives the sensors for each predetermined detection area, and is also called a backplane or active matrix substrate.

[0021] The substrate 21 has a detection area AA and a peripheral area GA. A plurality of sensor pixels 3 (a plurality of photodiodes 30) are arranged in a matrix in the detection area AA. That is, the plurality of photodiodes 30 are arranged in a first direction Dx and a second direction Dy intersecting the first direction Dx. Furthermore, the first gate line driving circuit 15A, the second gate line driving circuit 15B, the signal line driving circuit 16A, and the detection circuit 11 are provided in the peripheral area GA.

[0022] In the following description, the first direction Dx is a direction in a plane parallel to the substrate 21. The second direction Dy is a direction in a plane parallel to the substrate 21, and is a direction perpendicular to the first direction Dx. The second direction Dy may intersect the first direction Dx without being perpendicular thereto. The third direction Dz is a direction perpendicular to the first direction Dx and the second direction Dy, and is a normal direction to the main surface of the substrate 21. Furthermore, "plan view" refers to the positional relationship when viewed from a direction perpendicular to the substrate 21.

[0023] The detection circuit 11 supplies control signals Sa, Sb, and Sc to the first gate line drive circuit 15A, the second gate line drive circuit 15B, and the signal line drive circuit 16A, respectively, and controls their operations. Specifically, the first gate line drive circuit 15A outputs a gate drive signal (e.g., a reset control signal RST) to a reset control scan line GLrst (see FIG. 3) based on the control signal Sa. The second gate line drive circuit 15B outputs a gate drive signal (e.g., a read control signal RD) to a read control scan line GLrd (see FIG. 3) based on the control signal Sb. The signal line drive circuit 16A electrically connects the detection circuit 11 to a signal line SL (see FIG. 3) selected based on the control signal Sc.

[0024] The detection circuit 11 also includes a signal processing circuit that processes the detection signals Vdet from the multiple photodiodes 30. The detection circuit 11 includes a Readout Integrated Circuit (ROIC). The detection circuit 11 may be provided in the peripheral area GA of the array substrate 2, or may be provided on a wiring substrate electrically connected to the array substrate 2.

[0025] The photodiodes 30 included in the plurality of sensor pixels 3 perform detection in accordance with gate drive signals supplied from the first gate line drive circuit 15A and the second gate line drive circuit 15B. The plurality of photodiodes 30 output electrical signals corresponding to the light irradiated thereon as detection signals Vdet to the signal line drive circuit 16A. The detection circuit 11 is electrically connected to the plurality of photodiodes 30 via the signal line drive circuit 16A. The detection circuit 11 processes the detection signals Vdet from the plurality of photodiodes 30 and outputs a sensor value So based on the detection signals Vdet to the control circuit 70. In this way, the detection device 1 detects information related to the object 100 to be detected.

[0026] The light source 80 has a light source drive circuit 12 that drives a plurality of light emitting elements 82 mounted on a light source substrate 81. The plurality of light emitting elements 82 are arranged in a matrix in an area that overlaps with the detection area AA of the light source substrate 81. The light source drive circuit 12 supplies power supply voltages (anode power supply potential, cathode power supply potential) to the plurality of light emitting elements 82 based on a control signal Sd from the control circuit 70 (light source control circuit 72). This switches the plurality of light emitting elements 82 between on (lighted state) and off (non-lighted state).

[0027] The number and arrangement of the multiple light-emitting elements 82 can be changed as appropriate. The multiple light-emitting elements 82 may emit monochromatic light, or may be configured to emit light of multiple different wavelengths. The lighting pattern of the multiple light-emitting elements 82 can also be changed as appropriate depending on the state of the detection target 100 that is the detection object. The multiple light-emitting elements 82 may be lit simultaneously, or may be lit in a time-division manner for each predetermined region.

[0028] The control circuit 70 includes a sensor control circuit 71 that controls the optical sensor 10, a light source control circuit 72 that controls the light source 80, and a communication circuit 73. The sensor control circuit 71 and the light source control circuit 72 control the optical sensor 10 and the light source 80, respectively, so that the detection operation of the optical sensor 10 and the lighting operation of the light source 80 are performed in synchronization.

[0029] The communication circuit 73 connects the control circuit 70 and the external circuit 85 by wire or wirelessly. The external circuit 85 is, for example, a personal computer (PC). However, the external circuit 85 is not limited to this and may be a mobile terminal such as a tablet or smartphone. As a result, information about the object 100 detected by the detection device 1 is output to the external circuit 85 through the communication circuit 73. Furthermore, the user operates the external circuit 85 to input various conditions such as the timing of starting and ending detection by the detection device 1, or a threshold value for determining the presence or absence of the object 100.

[0030] Next, a circuit configuration and an example of operation of the optical sensor 10 will be described. Fig. 3 is a circuit diagram showing an optical sensor of a detection device according to an embodiment. As shown in Fig. 3, the sensor pixel 3 has a photodiode 30, a reset transistor Mrst, a readout transistor Mrd, and a source follower transistor Msf. In addition, the sensor pixel 3 is provided with a reset control scanning line GLrst and a readout control scanning line GLrd as detection drive lines (gate lines), and a signal line SL as wiring for signal readout.

[0031] The reset control scanning line GLrst, the readout control scanning line GLrd, and the signal line SL are each connected to a plurality of sensor pixels 3. Specifically, the reset control scanning line GLrst and the readout control scanning line GLrd extend in a first direction Dx and are connected to a plurality of sensor pixels 3 arranged in the first direction Dx. The signal line SL extends in a second direction Dy and is connected to a plurality of sensor pixels 3 arranged in the second direction Dy. The signal line SL is a wiring through which signals from a plurality of transistors (readout transistors Mrd and source follower transistors Msf) are output.

[0032] The reset transistor Mrst, the readout transistor Mrd, and the source follower transistor Msf are provided corresponding to one photodiode 30. Each of the multiple transistors included in the sensor pixel 3 is configured as an n-type TFT (Thin Film Transistor). However, without being limited to this, each transistor may be configured as a p-type TFT.

[0033] A common voltage VCOM is applied to the anode of the photodiode 30. The cathode of the photodiode 30 is connected to a node N1. The node N1 is connected to one of the source or drain of the reset transistor Mrst and the gate of the source follower transistor Msf. When light is irradiated onto the photodiode 30, a signal (charge) output from the photodiode 30 is accumulated in a capacitance element Cs formed at the node N1.

[0034] The gate of the reset transistor Mrst is connected to a reset control scanning line GLrst. A reset voltage VPP1 is supplied to the other of the source or drain of the reset transistor Mrst. When the reset transistor Mrst is turned on (conductive) in response to a reset control signal RST supplied from the first gate line driving circuit 15A, the voltage of the node N1 is reset to the reset voltage VPP1. The common voltage VCOM has a voltage lower than the reset voltage VPP1, and the photodiode 30 is reverse-bias driven.

[0035] The source follower transistor Msf is connected between a terminal to which a power supply voltage VPP2 is supplied and the readout transistor Mrd (node ​​N2). The gate of the source follower transistor Msf is connected to the node N1. A signal (voltage) corresponding to the signal (charge) generated in the photodiode 30 is supplied to the gate of the source follower transistor Msf. As a result, the source follower transistor Msf outputs a voltage corresponding to the signal (charge) generated in the photodiode 30 to the readout transistor Mrd.

[0036] The readout transistor Mrd is connected between the source (node ​​N2) of the source follower transistor Msf and the signal line SL. The gate of the readout transistor Mrd is connected to the readout control scanning line GLrd. When the readout transistor Mrd is turned on in response to a readout control signal RD supplied from the second gate line driving circuit 15B, a signal output from the source follower transistor Msf, i.e., a signal (voltage) corresponding to the signal (charge) generated in the photodiode 30, is output to the signal line SL as a detection signal Vdet. Each of the multiple signal lines SL is connected to the detection circuit 11.

[0037] 3, the reset transistor Mrst and the readout transistor Mrd each have a single-gate structure, but the reset transistor Mrst and the readout transistor Mrd may each have a so-called double-gate structure in which two transistors are connected in series, or may each have a structure in which three or more transistors are connected in series. Furthermore, the circuit of one sensor pixel 3 is not limited to a structure having three transistors: the reset transistor Mrst, the source follower transistor Msf, and the readout transistor Mrd. The sensor pixel 3 may have two transistors, or may have four or more transistors.

[0038] Next, a method for detecting a detectable object 100 by the detection device 1 will be described with reference to Fig. 4 to Fig. 11. Fig. 4 is an explanatory diagram for explaining a method for detecting a detectable object by the detection device according to the embodiment. Fig. 5 is a block diagram showing an example of the configuration of a sensor control circuit according to the embodiment.

[0039] 4, the optical sensor 10 acquires a plurality of image data I(n) at predetermined intervals. This allows the detection device 1 to acquire data on changes over time in the growth of the detection target 100 (colony). The predetermined period for acquiring the plurality of image data I(n) is, for example, about 5 to 10 minutes, but is not limited to this and can be changed as appropriate depending on the type of detection target 100 (colony), culture conditions, etc.

[0040] As shown in FIG. 5, the sensor control circuit 71 includes an image processing circuit 74, a contour processing circuit 75, a determination circuit 76A, a calculation circuit 76B, a labeling circuit 76C, an image output circuit 77, and a memory circuit 78.

[0041] 4 and 5, the image processing circuit 74 performs predetermined processing on each of the multiple image data I(n) acquired by the optical sensor 10 at predetermined intervals (where n is a natural number). Specifically, the image processing circuit 74 calculates the difference between the image data I(n) acquired at the predetermined interval and the initial image data Ib, and generates differential image data Id (see FIG. 8). The image processing circuit 74 also generates a binary image Ibi based on a predetermined threshold and the differential image data Id.

[0042] The contour processing circuit 75 extracts a contour OL of an area exceeding a predetermined threshold (ie, an area corresponding to the object 100) based on the binarized image Ibi.

[0043] The determination circuit 76A compares the extracted contour OL with the coordinates (X, Y) calculated using the previous image data I(n-1) to determine whether the previous coordinates (X, Y) are included in the area surrounded by the contour OL.

[0044] The calculation circuit 76B calculates coordinates (X, Y) corresponding to the extracted contour OL. The coordinates (X, Y) are, for example, the center coordinates when the contour OL is approximated to a circle. However, the coordinates (X, Y) are not limited to this and may be, for example, the geometric center of the area surrounded by the contour OL, or may be defined by other methods.

[0045] The labeling circuit 76C performs labeling by assigning identification information CN corresponding to the calculated coordinates (X, Y). The labeling circuit 76C then associates the coordinates (X, Y) and the identification information CN corresponding to the coordinates (X, Y) with the extracted contour OL, and stores the resulting data in the storage circuit 78 as a set of data. The identification information CN is information such as a number for distinguishing between a plurality of coordinates (X, Y) and the corresponding contours OL. However, the identification information CN is not limited to a number, and may include other information related to the object 100 to be detected, as necessary.

[0046] The image output circuit 77 generates an output image Io by superimposing information such as the contour OL, coordinates (X, Y), and identification information CN on the image data I(n) acquired by the optical sensor 10, and outputs the output image Io to a display device 86 of the external circuit 85.

[0047] The memory circuitry 78 stores various information such as a predetermined threshold value, multiple image data I(n) acquired at predetermined intervals, differential image data Id, binarized image Ibi, and corresponding contours OL, coordinates (X, Y), identification information CN, etc.

[0048] For ease of understanding, FIG. 4 illustrates a method for processing one image data I(n) acquired over a predetermined period. However, in this embodiment, multiple image data I(n) are acquired over a predetermined period to detect the growth of the detectable object 100 (colony) over time. Therefore, the shape of the outline OL may change over time, or a new detectable object 100 (colony) may be detected at a predetermined timing. This may make it difficult to associate the detected detectable object 100 with the outline OL, coordinates (X, Y), identification information CN, etc. Furthermore, if foreign matter such as printed marking 110A or moisture is attached to the container 110, false detection may occur.

[0049] Fig. 6 is a flowchart for explaining a method for extracting a contour corresponding to a detectable object based on image data, and Fig. 7 is a flowchart for explaining various processes for the contour extracted in Fig. 6 and image output.

[0050] 6, the optical sensor 10 acquires initial image data Ib (step ST11). The initial image data Ib is image data based on the sensor value So obtained by scanning the multiple photodiodes 30 of the optical sensor 10 in an initial state (for example, the time after power is turned on). The acquired initial image data Ib is stored in the memory circuit 78 of the sensor control circuit 71.

[0051] After acquiring the initial image data Ib, or after completing the processes of steps ST13 to ST28 described below, the optical sensor 10 waits for a predetermined period of time (step ST12). The predetermined period of time can be set or changed by operating the external circuit 85, and information about the set predetermined period of time is stored in the memory circuit 78.

[0052] After a predetermined period has elapsed since the previous detection, the optical sensor 10 scans the multiple photodiodes 30 to acquire image data I(n) (step ST13). The image data I(n) acquired at each predetermined period is stored in the memory circuit 78.

[0053] The acquired image data I(n), the various differential image data Id generated based on the image data I(n), the binarized image Ibi, and various pieces of information are stored in the memory circuit 78 of the sensor control circuit 71 as appropriate. However, in the following explanation, the storage of these various pieces of image data and information in the memory circuit 78 will be omitted.

[0054] The image processing circuit 74 calculates the difference between the initial image data Ib acquired in step ST11 and the image data I(n) acquired in step ST13, and generates differential image data Id (step ST14).

[0055] FIG. 8 is an explanatory diagram for explaining a method for calculating differential image data. As shown in FIG. 8, the image processing circuit 74 generates differential image data Id by calculating the difference between the sensor value So of the image data I(n) and the initial image data Ib for each sensor pixel 3. As a result, even if there is a print 110A on the container 110, or if there is moisture or dirt on the container 110, information such as the print 110A, which does not change over time from the initial state, is removed from the image data I(n), and the detectable object 100 is accurately extracted. As a result, even if a foreign object such as the print 110A is present, the influence of the print 110A is eliminated in various processes from step ST15 onwards, and the number, area, etc. of the detectable object 100 can be accurately calculated.

[0056] 6, the image processing circuit 74 compares the generated differential image data Id with a predetermined threshold value (step ST15). More specifically, the differential data of the sensor values ​​So for each sensor pixel 3 in the differential image data Id is compared with the predetermined threshold value, and if the differential data is equal to or less than the predetermined threshold value (step ST15, No), it is determined that the detectable object 100 (colony) is not growing, and the process returns to step ST12.

[0057] If the differential image data Id is greater than the predetermined threshold value (Yes in step ST15), it is determined that the detectable substance 100 (colony) has grown, and the differential image data Id is processed.

[0058] The image processing circuit 74 generates a binary image Ibi based on the differential image data Id (step ST16). The image processing circuit 74 compares the differential image data Id with a predetermined threshold, and binarizes an area greater than the predetermined threshold (an area corresponding to the detectable object 100) and an area equal to or less than the predetermined threshold (an area corresponding to the background). For example, in the example shown in FIG. 4, the area corresponding to the detectable object 100 is displayed in white, and the background is displayed in black, to generate a binary image Ibi.

[0059] The contour processing circuit 75 extracts contours of an area larger than a predetermined threshold based on the binary image Ibi (step ST17). FIG. 9 is an explanatory diagram for explaining a contour extraction method. As shown in FIG. 9, the contour processing circuit 75 raster scans the binary image Ibi for each row of sensor pixels 3 (step ST17-1). As indicated by the arrow in step ST17-1 in FIG. 9, the contour processing circuit 75 performs raster scanning, for example, from the first row of sensor pixels 3. When the sensor pixels 3 change from displaying black to displaying white, the contour processing circuit 75 detects a part of the contour OL of the object 100.

[0060] The contour processing circuit 75 searches the surrounding sensor pixels 3 in a counterclockwise direction starting from the sensor pixel 3-1 corresponding to the first detected part of the contour OL (step ST17-2). As indicated by the arrows in step ST17-2 in Fig. 9, the search is performed counterclockwise around the sensor pixel 3-1, and a sensor pixel 3 (e.g., sensor pixel 3-2) that has changed from black display to white display is detected as part of the contour OL of the detection object 100. Next, a part of the contour OL of the detection object 100 is similarly detected around the sensor pixel 3-2, and so on.

[0061] The contour processing circuit 75 repeatedly executes step ST17-2, and when it returns to the first sensor pixel 3-1, it can connect all contours OL to extract the contour OL of the region corresponding to the object 100 (step ST17-3).

[0062] Although Figure 9 describes the extraction of one contour OL, if the binary image Ibi includes multiple detectable objects 100, the contour processing circuit 75 executes Figure 9 multiple times to extract contours OL corresponding to each of the multiple detectable objects 100.

[0063] Next, as shown in FIG. 7, the sensor control circuit 71 executes the processes of steps ST18 to ST25 for all contours OL.

[0064] The decision circuit 76A decides whether the coordinates (X, Y) labeled with the previous image data I(n-1) are within the range of one contour OL selected from the contours OL extracted in step ST17 (step ST19).

[0065] If the coordinates (X, Y) labeled in the previous image data I(n-1) are not within the range of the selected contour OL (step ST19, No), that is, if a new contour OL is extracted in the image data I(n), the calculation circuit 76B calculates the coordinates (X, Y) of the new contour OL (step ST20).

[0066] The labeling circuit 76C labels the newly extracted contour OL by assigning identification information CN to the coordinates (X, Y) corresponding to the contour OL (step ST21).

[0067] The labeling circuit 76C associates the coordinates (X, Y) and identification information CN (label) with the contour OL newly extracted from the image data I(n) (step ST22).

[0068] The arithmetic circuit 76B calculates the area of ​​the region surrounded by the newly extracted contour OL (step ST23).

[0069] The storage circuitry 78 stores the newly extracted contour OL in association with information such as coordinates (X, Y) and identification information CN (label) (step ST24).

[0070] If the coordinates (X, Y) labeled in the previous image data I(n-1) are within the range of the selected contour OL (step ST19, Yes), that is, if the selected contour OL among the multiple contours OL extracted in the image data I(n) corresponds to a contour OL already extracted in the previous image data I(n-1), the coordinates (X, Y) calculated in the previous image data I(n-1) and the identification information CN labeled in the previous image data I(n-1) are associated with the contour OL selected in the image data I(n) (step ST22).

[0071] In other words, the information on the contour OL, coordinates (X, Y) and identification information CN (label) that were associated with each other in the previous image data I(n-1) is updated and associated with the information on the contour OL acquired in the image data I(n) while maintaining the information on the coordinates (X, Y) and identification information CN (label).

[0072] Next, the arithmetic circuit 76B calculates the area of ​​the region surrounded by the selected contour OL in the image data I(n) (step ST23).

[0073] The storage circuitry 78 associates the contour OL extracted from the image data I(n) with information such as coordinates (X, Y) and identification information CN (label), and stores the result as a set of data (step ST24).

[0074] When step ST19 is executed for the first time, that is, when processing the initial image data I(1) after obtaining the preliminary image data Ib, there is no previously labeled identification information CN, so the process proceeds to step ST20.

[0075] A specific example of the processing from step ST18 to step ST25 will now be described with reference to Fig. 10. Fig. 10 is an explanatory diagram for explaining a method of processing coordinates and labeling for contours extracted from image data. Fig. 10 explains the processing of image data I(n-1) acquired in the (n-1)th period, image data I(n) acquired in the nth period, and image data I(n+1) acquired in the (n+1)th period.

[0076] 10, three contours OL1-1, OL1-2, and OL1-3 have been extracted up to the past image data I(n-1). Based on the image data I(n-1), coordinates (X1, Y1) and identification information CN(1) are associated with contour OL1-1. Similarly, coordinates (X2, Y2) and identification information CN(2) are associated with contour OL1-2, and coordinates (X3, Y3) and identification information CN(3) are associated with contour OL1-3.

[0077] In the image data I(n), in addition to the three contours OL2-1, OL2-2, and OL2-3, a new contour OL2-4 is extracted. First, the processing of the newly extracted contour OL2-4 among the four contours OL will be described. The determination circuit 76A determines whether the coordinates (X1, Y1), (X2, Y2), and (X3, Y3) labeled in the previous image data I(n-1) are within the range of one contour OL2-4 selected from the four extracted contours OL (corresponding to step ST19).

[0078] Since the previous coordinates (X1, Y1), (X2, Y2), and (X3, Y3) are not within the range of contour OL2-4 (corresponding to step ST19, No), the calculation circuit 76B calculates the coordinates (X4, Y4) of the new contour OL2-4 (corresponding to step ST20).

[0079] The labeling circuit 76C assigns identification information CN(4) to the coordinates (X4, Y4) corresponding to the newly extracted contour OL2-4 to label it (corresponding to step ST21). Thereafter, the newly extracted contour OL2-4 is subjected to the processes of steps ST22 to ST24 described above.

[0080] Next, we will explain the processing of a contour OL (e.g., contour OL2-1) extracted up to the previous image data I(n-1) out of the four contours OL. The decision circuit 76A decides whether the coordinates (X1, Y1), (X2, Y2), and (X3, Y3) labeled in the previous image data I(n-1) are within the range of one contour OL2-1 selected from the four extracted contours OL (corresponding to step ST19).

[0081] Since the past coordinates (X1, Y1) are within the range of the contour OL2-1 (corresponding to step ST19, Yes), the labeling circuit 76C associates the coordinates (X1, Y1) calculated in the previous image data I(n-1) and the identification information CN(1) with the contour OL2-1 extracted in the image data I(n) (corresponding to step ST22).

[0082] In other words, for the information of the contour OL1-1, coordinates (X1, Y1) and identification information CN(1) that were associated with each other in the previous image data I(n-1), the information of the coordinates (X1, Y1) and identification information CN(1) is maintained, while the information of the previous contour OL1-1 is updated and associated with the information of the contour OL2-1 newly acquired in the image data I(n).

[0083] The arithmetic circuit 76B calculates the area of ​​the region surrounded by the contour OL2-1 extracted from the image data I(n) (corresponding to step ST23).

[0084] The storage circuitry 78 stores information such as coordinates (X1, Y1) and identification information CN(1) in association with the contour OL2-1 acquired from the image data I(n) (corresponding to step ST24).

[0085] The sensor control circuit 71 performs the same processing as that for the above-described contour OL2-1 (or contour OL2-4) on all contours OL acquired in the image data I(n). That is, information such as coordinates (X2, Y2) and identification information CN(2) is associated with contour OL2-2 and stored in the memory circuit 78. Also, information such as coordinates (X3, Y3) and identification information CN(3) is associated with contour OL2-3 and stored in the memory circuit 78.

[0086] In the image data I(n+1) acquired in the next (n+1) period, similarly, the contour OL3-1 extracted in the image data I(n+1) is associated with information such as the image coordinates (X1, Y1) and identification information CN(1) carried over from the previous image data I(n) and stored in the memory circuit 78. Similarly, the contour OL3-2 extracted in the image data I(n+1) is associated with information such as the coordinates (X2, Y2) and identification information CN(2) carried over from the previous image data I(n) and stored in the memory circuit 78. The contour OL3-3 extracted in the image data I(n+1) is associated with information such as the coordinates (X3, Y3) and identification information CN(3) carried over from the previous image data I(n) and stored in the memory circuit 78. The contour OL3-4 extracted in the image data I(n+1) is associated with information such as the coordinates (X4, Y4) and identification information CN(4) carried over from the previous image data I(n) and stored in the memory circuit 78. In the (n+1)th period, due to the growth of the detectable object 100 (colony), the two adjacent contours OL3-2 and OL3-3 are detected as connected, but even in this case, it is possible to detect that the contours OL3-2 and OL3-3 are each individual detectable object 100 (colony) based on the previous coordinates (X, Y) and identification information CN, etc.

[0087] As described above, the coordinates (X, Y) calculated for the specified image data I(n) and the identification information CN labeled corresponding to the coordinates (X, Y) are carried over to the image data I(n+1) acquired in the subsequent period.

[0088] 7, after executing the processes of steps ST19 to ST24 for all contours OL extracted from the image data I(n), the arithmetic circuit 76B counts the number of pieces of identification information CN labeled at the coordinates (X, Y) (step ST26). In step ST26, the arithmetic circuit 76B counts the total number of pieces of identification information CN labeled in the previous image data I(n-1) and carried over to the image data I(n), and the total number of pieces of identification information CN newly labeled in the image data I(n).

[0089] 11 is a schematic diagram showing an example of an output image. As shown in FIG. 11, the image output circuit 77 generates an output image Io by superimposing, on the image data I(n), identification information CN labeled for each contour OL and information such as coordinates (X, Y) corresponding to the contour OL, and outputs the output image Io to the external circuit 85 (step ST27). The output image Io is displayed on the display device 86 of the external circuit 85. Furthermore, the image output circuit 77 displays the contour OL in the output image Io using a line of a different color or darkness from that of the area surrounded by the contour OL.

[0090] 11 is merely an example, and the output image Io may have any form. For example, the labeled identification information CN, coordinates (X, Y), and other information may be displayed in an area separate from the output image Io.

[0091] The sensor control circuit 71 determines whether to end the measurement by the optical sensor 10 (step ST28). The measurement by the optical sensor 10 is determined based on a preset number of measurements (or measurement period). Alternatively, the end of the measurement by the optical sensor 10 may be determined based on an input from the external circuit 85. If the measurement is to be ended (step ST28, Yes), the sensor control circuit 71 stops driving the optical sensor 10. If the measurement is to be continued (step ST28, No), the sensor control circuit 71 returns to step ST12 and executes the measurement by the optical sensor 10 and processes the image data I(n).

[0092] 6 and 7 are merely examples and can be modified as appropriate. For example, in the acquired image data I(n), some of the processing for the detected object 100 (extracted contour OL) may be omitted, or other processing may be added as necessary.

[0093] As described above, the detection device 1 of this embodiment extracts first contours (e.g., contours OL1-1, OL1-2, OL1-3) of at least one area that exceeds a predetermined threshold for first image data (e.g., image data I(n-1)) acquired in a first period (e.g., the (n-1)th period), calculates first coordinates (e.g., coordinates (X1, Y1), coordinates (X2, Y2), coordinates (X3, Y3)) corresponding to the first contours, and labels first identification information (e.g., identification information CN(1), CN(2), CN(3)) corresponding to the first coordinates. For second image data (e.g., image data I(n)) acquired during a second period (e.g., the (n) period) a predetermined time after the first period, second contours (e.g., contours OL2-1, OL2-2, OL2-3, OL2-4) of at least one region exceeding a predetermined threshold are extracted, and if there is a second contour (e.g., contour OL2-4) among the second contours that does not include the first coordinate, second coordinates (e.g., coordinates (X4, Y4)) corresponding to the second contour that does not include the first coordinate are calculated, and new second identification information (e.g., identification information CN(4)) corresponding to the second contour that does not include the first coordinate is assigned. The sensor control circuit 71 calculates the total number of first identification information (e.g., identification information CN(1), CN(2), CN(3)) labeled in the first image data and second identification information (e.g., identification information CN(4)) newly labeled in the second image data.

[0094] As a result, even if the shape or area of ​​the contour OL changes due to the growth of the detectable object 100 (colony), the detection device 1 of this embodiment can determine whether the detectable object 100 (contour OL) is a previously detected detectable object 100 (contour OL) or a newly grown detectable object 100 (contour OL). Furthermore, even if two adjacent contours OL3-2 and OL3-3 are detected as connected due to the growth of the detectable object 100 (colony) as shown in image data I(n+1), it can be determined that the contour OL does not represent a single detectable object 100 but represents two detectable objects 100 (contours OL) based on information such as the previous coordinates (X, Y) and identification information CN. In other words, the detection device 1 can accurately detect the number of detectable objects 100. As described above, the detection device 1 can improve the detection accuracy of the detectable object 100 by processing the contour OL and labeling various information for each of the multiple image data I(n) acquired at predetermined intervals.

[0095] Next, a description will be given of an example of a detection system including the above-described detection device 1. Fig. 12 is a schematic diagram showing an example of the configuration of the detection system. Fig. 13 is a cross-sectional view showing an inspection unit included in the detection system.

[0096] 12, the detection system 5 includes a plurality of detection units 121 (detection devices 1), a control circuit 70, and a connection circuit 125 that connects the plurality of detection units 121 (detection devices 1) and the control circuit 70. The plurality of detection units 121 (detection devices 1) are electrically connected to the common control circuit 70 via the connection circuit 125.

[0097] 13, the detection unit 121 has a detection device 1 (optical sensor 10, optical filter layer 50, light source 80, and control board 13) and a housing 122. The detection device 1 (optical sensor 10, optical filter layer 50, light source 80, and control board 13) is disposed in the housing 122. A container 110 containing a detection target 100 and a culture medium 102 is disposed between the optical sensor 10 (optical filter layer 50) and the light source 80. The detection device 1 is stacked in the housing 122 in the order of the light source 80, the container 110, and the optical sensor 10 (optical filter layer 50). However, this is not limiting, and the stacking order of the detection device 1 may be reversed.

[0098] The control board 13 is mounted with a detection circuit 11 (see FIG. 2) that processes the detection signal Vdet from the optical sensor 10. The control board 13 may also be mounted with a light source control circuit 72 (see FIG. 2) that controls the light source 80.

[0099] 12, with its door closed, the incubator 120 is maintained in an environment (temperature, humidity, etc.) suitable for culturing the test object 100. A plurality of detection units 121 (detection devices 1) are arranged inside the incubator 120, and detect the test object 100 multiple times at preset timings (times). A connection circuit 125, not shown, is connected to the plurality of detection units 121 (detection devices 1) by wire or wirelessly. The plurality of detection units 121 (detection devices 1) transmit image data I(n) calculated by the respective detection circuits 121 to the control circuit 70 via the connection circuit 125.

[0100] The detection system 5 has a plurality of detection units 121 (detection devices 1) and a control circuit 70, and therefore can easily detect (image) a plurality of different objects to be detected 100 in parallel. In this embodiment, some of the processing performed by the sensor control circuit 71 shown in FIGS. 6 and 7 may be distributed among the plurality of detection units 121 (detection devices 1). In this case, the calculation processing load on the control circuit 70 can be reduced in the detection system 5 having a plurality of detection units 121 (detection devices 1).

[0101] Although preferred embodiments of the present disclosure have been described above, the present disclosure is not limited to such embodiments. The contents disclosed in the embodiments are merely examples, and various modifications are possible within the scope of the present disclosure. Appropriate modifications made within the scope of the present disclosure naturally fall within the technical scope of the present disclosure. At least one of various omissions, substitutions, and modifications of components can be made within the scope of the gist of each of the above-described embodiments and modifications. [Explanation of symbols]

[0102] 1. Detection device 2 Array board 3 sensor pixels 5. Detection System 10. Optical Sensor 11 Detection circuit 30 Photodiode 50 optical filter layers 70 Control circuit 71 Sensor control circuit 74 Image processing circuit 75 Contour processing circuit 76A judgment circuit 76B Arithmetic circuit 76C Labeling Circuit 77 Image output circuit 78 Memory circuit 80 light source 81 Light source board 82 Light-emitting element 100 Object to be detected 102 Culture medium 110 Container CN Identification Information Office Lady Outline I(n) image data

Claims

1. an optical sensor including a plurality of photodetection elements arranged in a plane; a light-transmitting detection target installation portion disposed over the optical sensor and configured to install a plurality of detection targets; a control circuit for controlling the optical sensor; The optical sensor acquires image data at predetermined intervals from the start of measurement, the control circuit performs a predetermined process on each of the plurality of image data; extracting a first contour of at least one region exceeding a predetermined threshold from first image data acquired during a first period, calculating first coordinates corresponding to the first contour, and labeling the first identification information corresponding to the first coordinates; extracting second contours of at least one region exceeding a predetermined threshold value from second image data acquired during a second period that is a predetermined period after the first period; if there is a second contour among the second contours that does not include the first coordinates, calculating second coordinates corresponding to the second contour that does not include the first coordinates; and assigning new second identification information corresponding to the second contour that does not include the first coordinates; Calculating the total number of the first identification information labeled in the first image data and the second identification information newly labeled in the second image data Detection device.

2. When there is a second contour including the first coordinates corresponding to the first contour extracted from the first image data among at least one of the second contours extracted from the second image data, the first coordinates and the first identification information corresponding to the first coordinates are associated with the second contour including the first coordinates. The detection device according to claim 1 .

3. The control circuit generates an output image by superimposing the extracted second contour and the second coordinates or the first coordinates corresponding to the second contour on the second image data, and displays the output image on a display device. The detection device according to claim 1 .

4. In the output image, the second contour is displayed in a color or a line of a different density from that of the area surrounded by the second contour. The detection device according to claim 3 .

5. The control circuit calculates the area of ​​the region surrounded by the second contour. The detection device according to claim 1 .

6. a light directivity control element disposed between the plurality of light detection elements and the detection target placement portion; The detection device according to claim 1 .

7. The light directivity control element is a louver, a collimator, or a microlens. The detection device according to claim 6.

8. a light source that irradiates the plurality of light detection elements with light; The detection device according to claim 1 .

Images