Inspection device and inspection method for aggregate of foreign matter and particles

Abstract

The invention provides an inspection device capable of detecting foreign matters and aggregate of fine particles and accurately detecting the reproduction degree of the fine particles. An aggregate inspection device (500) is provided with an image acquisition unit (11) that acquires an image in a container (511) that holds a liquid containing fine particles, and detects the presence or absence of an aggregate containing fine particles. An inspection device (500) is provided with: a foreign matter presence / absence determination unit (24) that binarizes an image, generates a first binarized image, and determines the presence / absence of a foreign matter; a microparticle region detection unit (25) that extracts the contour of the image as a contour extraction image, binarizes a composite image of the extracted contour extraction image and the image, generates a second binarized image, and detects a microparticle region; an integrated signal difference processing unit (16) that calculates a difference value between a first integrated signal in which the first binarized image is integrated and a second integrated signal in which the second binarized image is integrated; and an aggregation presence / absence determination unit (17) that determines the presence / absence of aggregates by performing numerical analysis on the difference waveform of the difference value, and the inspection device detects the presence / absence of at least two different types of objects included in the image.

Description

Technical Field

[0001] The present disclosure relates to an inspection device and an inspection method for detecting foreign matter and aggregates of fine particles and accurately detecting the reproduction rate of fine particles. Background Art

[0002] The international spread of infectious diseases is a major concern for human health worldwide. In recent years, the prevalence of various emerging infectious diseases has continued to expand, prompting the development of new testing methods to identify and detect the causative viruses and bacteria, as well as new vaccines.

[0003] Let's take bacterial infections as an example. The emergence of multidrug-resistant bacteria is a major global problem. The US CDC identifies carbapenem-resistant Acinetobacter, carbapenem-resistant Enterobacteriaceae, Clostridioides difficile, and the fungus Candida auris as major threats.

[0004] If such a cluster of drug-resistant bacteria occurs, the risk of death for patients is high. Rapid identification of the pathogens and their sensitivity testing are crucial to detect drug-resistant bacteria and promptly implement appropriate measures. Mass spectrometry has become the mainstream method for identifying pathogens, enabling testing within one hour from positive samples such as colonies or blood cultures.

[0005] Turbidimetric testing is the mainstream method for evaluating bacterial sensitivity to drugs in susceptibility testing (AST). However, in order to speed up the testing process, a method has been put into practical use that uses a microscope to photograph bacteria and detect bacterial growth with higher sensitivity than conventional methods.

[0006] Among image-based AST, one method involves incubating fluorescently labeled bacteria with an antimicrobial drug, then imaging the bacteria using a fluorescence microscope. Sensitivity is determined by measuring growth based on changes in the fluorescence image. Alternatively, a method that does not use fluorescent labeling but processes bacterial images acquired with bright-field microscopy to measure changes in bacterial shape and growth to determine sensitivity is also available. This latter technique, which uses bright-field microscopy, is a label-free method and therefore offers the advantage of reducing reagent costs.

[0007] However, label-free detection methods have the disadvantage of being prone to misidentifying non-bacteria objects such as dust, dirt, or air bubbles contained in samples or test containers as bacteria. For example, if dust, dirt, or air bubbles are misidentified as bacteria, bacteria may be mistakenly identified as multiplying even when they are not.

[0008] Here, when considering objects included in the sample used for sensitivity testing and potentially imaged, there are bacteria to be tested, cells in the biological sample from which the bacteria are isolated, such as red blood cells, white blood cells, or platelets, as well as other particles such as dust, dirt, and air bubbles. Furthermore, depending on the bacterial species, bacterial aggregation may result in the formation of aggregates that may be imaged, as they produce viscous substances such as mucus through their own metabolism or exhibit coagulation through the action of extracellular enzymes such as coagulase.

[0009] In areas where dust, dirt, or bubbles are present, incident light is scattered and absorbed, resulting in a lower pixel brightness and a darker image. Similarly, when bacteria aggregate, the pixel brightness in the aggregated area decreases, resulting in a darker image and difficulty distinguishing between the two. On the other hand, aggregated bacteria are less susceptible to external antimicrobial agents, sometimes exhibiting drug resistance, making them an important indicator in inspections. If dust, dirt, or bubbles can be accurately distinguished from aggregated bacteria for detection, they could become effective indicators for growth in short-term cultures.

[0010] Patent Document 1 discloses a method for preventing erroneous recognition and accurately detecting regions of microparticles corresponding to bacteria by performing a logical AND operation on images before and after contour extraction and then performing binarization.

[0011] Furthermore, Patent Document 2 discloses a method of extracting noise from the frequency of appearance of shape feature values ​​of individual cells. Prior art literature Patent Literature

[0012] Patent Document 1: Japanese Patent No. 7130109 Patent Document 2: Japanese Patent Application Laid-Open No. 2011-243188 Summary of the Invention Technical problem to be solved by the invention

[0013] The technology described in Patent Document 1 detects both isolated areas of illuminated microparticles and areas of densely illuminated, darker microparticles. Therefore, while microparticle detection sensitivity is high, it is difficult to distinguish and detect isolated bacteria, agglomerated bacteria, and foreign matter other than bacteria, such as dust, dirt, and air bubbles.

[0014] Furthermore, in the technology described in Patent Document 2, since noise is extracted from information such as the size of the object, it is difficult to distinguish and detect foreign matter and bacterial aggregates from shape information.

[0015] Therefore, to date, no method has been disclosed for distinguishing and detecting bacteria, bacterial aggregates, and foreign matter other than bacteria, such as dust, dirt, and bubbles, while maintaining the sensitivity of bacteria detection in an image containing particulate measurement objects such as growing bacteria.

[0016] The present disclosure aims to provide an inspection device and an inspection method that can detect foreign matter and aggregates of fine particles and accurately detect the reproduction rate of fine particles. Technical solutions to technical problems

[0017] In order to achieve the above-mentioned object, the present disclosure is constructed as follows.

[0018] A coagulation detection device is provided with an image acquisition unit for acquiring an image of a container holding a liquid containing microparticles and detecting the presence or absence of coagulation of microparticles, including: a foreign matter presence or absence determination unit, which binarizes the image to generate a first binarized image and determines the presence or absence of foreign matter; a microparticle area detection unit, which extracts the contour of the image as a contour extraction image, binarizes the extracted contour extraction image and a composite image of the image to generate a second binarized image and detects the microparticle area; a cumulative signal difference processing unit, which calculates the difference between a first cumulative signal accumulated by the first binarized image and a second cumulative signal accumulated by the second binarized image; and an coagulation presence or absence determination unit, which determines the presence or absence of coagulation by performing numerical analysis on the differential waveform of the differential value, wherein the coagulation detection device detects the presence or absence of at least two different types of objects contained in the image.

[0019] In addition, a method for detecting agglomerates is provided, which acquires an image of a container holding a liquid containing microparticles and detects the presence or absence of agglomerates of the microparticles. In this method for detecting agglomerates, the image is binarized to generate a first binarized image, the presence or absence of foreign matter is determined, the contour of the image is extracted as a contour extraction image, the extracted contour extraction image and a composite image of the image are binarized to generate a second binarized image, the microparticle area is detected, the difference between a first cumulative signal accumulated by the first binarized image and a second cumulative signal accumulated by the second binarized image is calculated, the presence or absence of agglomerates is determined by numerically analyzing the differential waveform of the differential value, and the presence or absence of at least two different types of objects contained in the image is detected. Effects of the Invention

[0020] According to the present disclosure, it is possible to provide an inspection device and an inspection method that can detect foreign matter and aggregates of fine particles and accurately detect the reproduction degree of fine particles. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1A This is a diagram showing an example of an image including bubbles as an example of foreign matter. Figure 1B This figure shows an example of an image containing bacteria. Figure 1C This figure shows an example of an image of bacteria contained in a limited area after the bacteria have grown and agglomerated. Figure 2 This is a flowchart illustrating the details of image processing in the first embodiment. Figure 3A This is a graph showing the results when an image contains round foreign objects that are considered to be bubbles. Figure 3B This figure shows the results when no foreign matter is contained but uniformly dispersed bacteria are contained. Figure 3C This figure shows the results when no foreign matter is included but aggregated bacteria are included. Figure 4 It is a diagram showing a method of correcting a signal. Figure 5 This is a diagram showing a schematic configuration example of a bacteria inspection device according to Example 1. Figure 6 This is a functional block diagram of the determination processing unit included in the control unit of Example 1. Figure 7 This is an operational flow chart for determining the bacterial proliferation degree in Example 2. Figure 8 This is a graph showing the dependence of the area value of the bacterial region on the culture time as an example of a feature value calculated by removing the influence of foreign matter. Figure 9 This is a functional block diagram of a determination processing unit included in the control unit of the second embodiment. Figure 10 This is a flowchart of shooting and image processing in Example 3. Figure 11 This is a diagram showing a configuration example of a GUI for outputting results. Figure 12 This is a functional block diagram of a determination processing unit included in a control unit of the third embodiment. DETAILED DESCRIPTION

[0022] For example, this embodiment discloses a technology that, for example, takes an image of microparticles captured through a microscope in a well that holds a liquid containing microparticles, detects the area of ​​the microparticles, detects the area where the microparticles are aggregated and the area where foreign matter other than the microparticles is present, determines the presence or absence of aggregates and foreign matter and their size, and uses information on the foreign matter area to correct the characteristic quantity of the microparticles, thereby accurately determining the degree of microparticle reproduction based on this characteristic quantity. Below, embodiments of the present disclosure will be described with reference to the accompanying drawings. While the accompanying drawings illustrate specific embodiments based on the principles of the present technology, these drawings are intended to facilitate understanding of the present technology and are not intended to limit the present disclosure. In the embodiments and all drawings used to illustrate the embodiments, components having the same function may be given the same reference numerals, and their repeated descriptions may be omitted.

[0023] Examples of microparticles include bacteria, archaea, fungi, and biological cells. Here, bacteria will be described as one of the microparticles. Example

[0024] <Image Description> Figure 1A 、 Figure 1B 、 Figure 1C This figure shows an example of an image of a sample solution containing bacteria or foreign matter other than bacteria as fine particles. Here, as an example, an 8-bit grayscale image with 256 gray levels is assumed for description.

[0025] Figure 1A This figure shows an example of an image containing bubbles as an example of foreign matter 100. In the area where foreign matter 100 exists, the amount of light received by the imaging unit decreases due to absorption, refraction, and scattering, resulting in a lower brightness than the background 101 area, and appearing darker in the image.

[0026] Figure 1B The figure shows an example of an image containing bacteria 102. Bacteria 102 appear circular or elliptical in the image. When properly focused and captured, a difference in brightness occurs between the outline of bacteria 102 and the center of bacteria 102. For example, the brightness of the outline of bacteria 102 is lower than that of background 101, while the brightness of the center of bacteria 102 is higher than that of background 101.

[0027] Generally, bacteria 102 can move freely in the sample solution, so they usually exist in a uniformly dispersed manner, but depending on the state of the sample or the external environment, they can also gather in one place and Figure 1B Consider the state of being aggregated into a circle. When the growth of bacteria is not progressing sufficiently, the background 101 region exists between the bacteria 102 regions, and thus the two are spatially separated.

[0028] Figure 1C 1 is a diagram showing an example of an image of bacteria 103 after bacterial growth and aggregation in a limited area. When bacteria 103 are densely packed, the outline of bacteria 103 is common to the outline of adjacent bacteria 103, so it is not possible to Figure 1BIn this way, a clear difference in lightness and darkness is obtained, and areas with brightness values ​​higher than the background and areas with brightness values ​​lower than the background exist locally.

[0029] The method disclosed in Patent Document 1 can detect both bacteria 102 and aggregated bacteria 103. Furthermore, the area of ​​foreign matter 100 is also detected. Bacteria typically have a diameter of approximately 0.5 μm to several μm, while bubbles, which are foreign matter 100, are larger than several tens of μm. Therefore, bacteria 102 and foreign matter 100 can be distinguished based on their size.

[0030] However, the size of aggregated bacteria 103 ranges widely, from a few μm to several hundred μm, and considering that dust particles of various sizes can also be foreign matter, it is difficult to distinguish aggregated bacteria 103 from foreign matter 100. Even if other shape information is considered, aggregated bacteria 103 gather in random shapes, making it difficult to distinguish. In view of the above situation, a method for distinguishing and detecting agglutinated bacteria and foreign matter and determining the presence or absence of agglutination and foreign matter will be described.

[0031] (Example 1) <Details of the Image Processing Method of the Present Disclosure> Figure 2 This is a flowchart explaining the details of image processing.

[0032] (1) Step S200 In step S200, an image containing bacteria is acquired. In Example 1, the image is an 8-bit grayscale image, where a pixel value of 0 represents black and a pixel value of 255 represents white. The processing of this embodiment is also effective for grayscale images other than 8-bit images or images that have been inverted from black to white. Furthermore, similar processing can be performed by converting color images to a color space such as R, G, B or H, S, V.

[0033] (2) Step S201 In step S201, the first image acquired in step S200 is binarized. In this binarization process, to detect dimly illuminated foreign objects, a luminance value lower than the background is preferably used as the binarization threshold. To detect brighter foreign objects, a luminance value higher than the background can be used as the binarization threshold.

[0034] This threshold can be a fixed value set in advance, or it can be determined dynamically for each image by calculating the degree of inter-class separation in the histogram, as in Otsu binarization. Alternatively, if the brightness within the image plane is uneven due to, for example, variations in illumination intensity, the binarization threshold can be changed dynamically for each small area of ​​the image.

[0035] (3) Step S202 In step S202, the pixels detected by the binarization process in step S201 are accumulated in at least one of the vertical and horizontal directions (generating a first binarized image). The brightness value of the binarized image is 0 or 255. For example, when a dimly illuminated foreign object is detected and accumulated in the vertical direction, the image is scanned in the vertical direction, and pixels with a brightness value of 255 are ignored. If the pixel has a brightness value of 0, 1 is added to the pixel to obtain a cumulative signal in the vertical direction.

[0036] For example, assuming that the image is 100 pixels in the vertical direction and 200 pixels in the horizontal direction, a vertical cumulative signal with a width of 100 and a maximum amplitude of 200 pixels and a horizontal cumulative signal with a width of 200 and a maximum amplitude of 100 pixels can be obtained.

[0037] When a foreign object is present in an image, a peak in the accumulated signal is generated. Therefore, foreign objects can be detected by numerically analyzing the accumulated signal. Specifically, smoothing is first performed to remove noise from the accumulated signal. For example, a moving average or low-pass filtering process is preferably performed. Afterwards, a peak in the peak is detected. Simply calculate the signal's standard deviation and coefficient of variation and determine whether the value exceeds a reference value.

[0038] More specifically, the first and second derivatives of the signal are calculated, along with the amplitude of the signal's extreme value and peak, to determine whether the value exceeds a reference value. If a peak value is present, it is determined that a foreign object is present.

[0039] The above-mentioned peak value detection may be performed on only one of the vertical or horizontal integrated signals, or on both. In a more preferred method for highly sensitive foreign matter detection, peak values ​​are detected on both the vertical and horizontal integrated signals. If a peak value is detected in either signal, a foreign matter is determined to be present.

[0040] In addition to the above method, it is also possible to identify particles that are clustered together in the binarized image generated in step S201. The size and shape information of the identified particles, such as true roundness, circularity, aspect ratio, and circumference, can be compared with a threshold value to determine the presence of foreign matter. This method increases the computational load when a large number of particles are detected. Therefore, from the perspective of high-speed processing, a detection method that performs numerical analysis on the accumulated signal is preferred.

[0041] (4) Step S203 In step S203, the first image acquired in step S200 is copied to memory and contour extraction processing is performed. For example, a Sobel filter or Laplacian filter is used to extract edges (contours), or a variance filter is used to replace the brightness value of the pixel with the variance of the brightness of the surrounding N×N pixels. A logical operation, specifically a logical AND operation, is then performed on the brightness values ​​of each pixel in the first image copied to memory and the image after edge extraction (contour-extracted image), resulting in a second image (a composite image of the extracted contour and the first image (second binarized image)).

[0042] (5) Step S204 In step S204, the second image acquired in step S203 is binarized. The binarization process is performed in the same manner as in step S201. By binarizing the second image, an image of all areas where bacteria, agglutinated bacteria, and foreign matter are detected is obtained.

[0043] (6) Step S205 In step S205, a first cumulative signal of the binarized first image and a second cumulative signal of the binarized second image are calculated, and the difference between the first and second cumulative signals is calculated. Since the cumulative signal of the binarized first image was acquired in step S202, in this step S205, the cumulative signal of the binarized second image is calculated, and the difference between the two signals is calculated. This process can calculate the difference between the cumulative signals in either the vertical or horizontal direction of the image, but more preferably, calculates the difference between the two cumulative signals.

[0044] (7) Step S206 In step S206, the presence of aggregates is determined by numerically analyzing the differential values ​​of the accumulated signals acquired in step S205. Since the first image detects areas of foreign matter, and the second image detects areas of bacteria, aggregated bacteria, and foreign matter, a peak in the differential values ​​of the accumulated signals indicates the presence of aggregated bacteria. Similar to step S202, peak detection is performed, and the smoothed deviation value is compared with a threshold. Alternatively, extreme value detection based on the first and second derivatives of the signal and subsequent threshold comparison are performed.

[0045] Here, since the purpose of calculating the difference value in step S205 and step S206 is to extract the difference between the two signals, it is not necessarily necessary to calculate the difference value, and the same effect can be obtained even if the ratio of the two signals is calculated.

[0046] <Effects of the Present Disclosure on Detection of Foreign Matter and Aggregates> Figure 3A 、 Figure 3B and Figure 3C This figure shows the waveform of the accumulated signal calculated according to the above workflow.

[0047] Figure 3A This is the result when a circular foreign object, thought to be a bubble, is included in the image. Here, as an example, when integrating vertically in the image, the binarized integrated signal 300 of the first image is represented by a solid line, and the difference signal 301 of the integrated signal is represented by a dotted line. When a foreign object is included, only the binarized integrated signal 300 of the first image contains a peak with large fluctuations.

[0048] Figure 3B This is the result when there is no foreign matter and bacteria are evenly dispersed. In this case, there are no clear peaks in any signal.

[0049] Figure 3C This is the result when there is no foreign matter but aggregated bacteria. In this case, only the differential signal 301 of the integrated signal includes a peak with a large fluctuation.

[0050] By performing the numerical analysis of the above signals, the presence or absence of foreign matter and agglomerated bacteria can be determined. Figure 3B The differential signal 301 of the accumulated signal is tilted, and it is preferable to perform analysis after correcting this waveform.

[0051] Figure 4 Indicates the method of correcting the signal.

[0052] First, an approximate straight line 400 is calculated for the differential signal 301. For example, the least squares method is preferably used to approximate the straight line 400. By calculating the slope of the approximate straight line 400 and subtracting the value corresponding to the slope, the corrected differential signal 401 can be obtained. After performing this correction, the standard deviation of the waveform is calculated as follows.

[0053] Regarding the determination of foreign matter, there is foreign matter Figure 3A The standard deviation of the cumulative signal after binarization of the first image is 166, and there is no foreign matter Figure 3B The standard deviation of the cumulative signal after the first image is binarized is 4. By comparing the value of the standard deviation with, for example, a threshold value (80), the presence or absence of foreign matter can be detected.

[0054] Regarding the determination of agglomerates, there is no agglomerate Figure 3A The standard deviation of the differential signal of the cumulative signal is 30. In addition, Figure 3B The standard deviation of the differential signal of the cumulative signal is 27, and there are aggregates Figure 3CThe standard deviation of the difference signal of the cumulative signal is 121. Here, too, the presence or absence of aggregates can be detected by comparing the value of the standard deviation with, for example, the determination threshold value (80).

[0055] Although not shown, by calculating an average value of the corrected differential signal 401 and performing differentiation on the average value, the influence of bacteria uniformly distributed in space can be eliminated, and aggregates can be detected more accurately.

[0056] In this way, by performing correction processing on the integrated signal, analyzing the waveform and numerical values, calculating the standard deviation of the waveform, and comparing it with the determination threshold, the presence or absence of foreign matter and aggregates can be detected.

[0057] While this method shows a simplified result, similar results can be achieved even if the method is configured to detect the presence of foreign matter or aggregates by calculating the derivative of the signal, performing peak detection, and comparing the peaks with a determination threshold, as described in step 202. Furthermore, although the results are not shown, peaks of both foreign matter and aggregated bacteria can be detected, allowing for discrimination of images containing both foreign matter and aggregated bacteria.

[0058] <Structure example of bacteria inspection device> Figure 5 The diagram shows a schematic configuration example of a bacteria inspection apparatus (aggregate inspection apparatus) 500 according to Example 1 of the present disclosure. The bacteria inspection apparatus 500 includes an illumination unit 501 , an inspection plate 502 , a stage 503 , an objective lens 504 , an imaging unit 505 , an image processing unit 506 , and a control unit 507 .

[0059] The test plate 502 has one or more containers (wells) 511, preferably 96 or 384 wells, each of which holds a sample solution 510. The sample solution 510 contains a bacterial sample and may also contain a culture medium for bacterial growth and one or more antimicrobial drugs at pre-adjusted concentrations.

[0060] Because the inspection plate 502 varies depending on the type of bacteria or drug being inspected, it is preferable to introduce multiple different inspection plates 502 into the bacteria inspection apparatus 500 for each inspection. The inspection plates 502 are preferably identified by numbers that distinguish the patient ID of the inspection target, the type of bacteria being inspected, or the type of drug being inspected. For example, identification is preferably achieved using a barcode label, a QR code, or an RFID.

[0061] The lighting unit 501 irradiates light in the direction of the inspection plate 502. The lighting unit 501 can also use a light source such as a lamp, white light, or an LED containing light in a specific wavelength region. The light that passes through the holes in the inspection plate 502 and the sample solution 510 is focused by the objective lens 504 and imaged and photographed by the shooting unit 505. In addition, for example, the platform 503 can be moved under the control of the control unit 507 to change the relative position of the holes in the inspection plate 502 and the shooting unit 505, so that different holes can be photographed. The shooting action is also controlled by the control unit 507 and is performed at predetermined time intervals, such as 30 minutes. The image acquired by the shooting unit 505 is processed by the image processing unit 506 and sent to the control unit 507.

[0062] Here, the focus of the objective lens 504 is preferably aligned with the bottom surface of the hole of the inspection plate 502, but it can also be aligned with the interior of the sample solution 510 away from the bottom surface. In addition, it is possible to capture images of multiple locations within the hole, or to capture multiple images of the interior of the sample solution 510 away from the bottom surface of the hole of the inspection plate 502.

[0063] The control unit 507 is, for example, a general computer, and in addition to controlling the imaging unit 505 and the image processing unit 506, includes an input / output unit 508. The input / output unit 508 includes a display, a mouse, a keyboard, and a touch panel, and displays measurement conditions and outputs.

[0064] Furthermore, the bacteria inspection apparatus 500 includes a humidification and temperature control unit 509. If the entire imaging system for acquiring images is controlled at, for example, 35°C and a humidity of approximately 30%, bacteria can be efficiently grown, which is preferable.

[0065] Figure 6 This is a functional block diagram of the determination processing unit 1 included in the control unit 507. The determination processing unit 1 executes Figure 2 action flow.

[0066] exist Figure 6 In the figure, the determination processing unit 1 includes an image acquisition unit 11, a first image binarization processing unit 12, a contour extraction processing logic and operation processing unit 13, a foreign matter presence determination unit 14, a second image binarization processing unit 15, a cumulative signal difference processing unit 16, and an agglutination presence determination unit 17.

[0067] The first image binarization processing unit 12 and the foreign matter presence / absence determination unit 14 constitute a foreign matter presence / absence determination unit 24. The contour extraction processing logical AND operation processing unit 13 and the second image binarization processing unit 15 correct the microparticle region detection unit 25.

[0068] The image acquisition unit 11 executes Figure 2Following the processing of step S200, the first image binarization processing unit 12 performs the processing of step S201. Furthermore, the contour extraction processing logical AND operation processing unit 13 performs the processing of step S203, and the foreign matter presence determination unit 14 performs the processing of step S202. Furthermore, the second image binarization processing unit 15 performs the processing of step S204, and the accumulated signal difference processing unit 16 performs the processing of step S205. Then, the agglutination presence determination unit 17 performs the processing of step S206.

[0069] Since the first embodiment is configured as described above, it is possible to provide an inspection apparatus and an inspection method capable of detecting foreign matter and aggregates of fine particles and accurately detecting the proliferation rate of fine particles.

[0070] (Example 2) Next, Example 2 will be described.

[0071] Example 2 is an example in which Example 1 is applied to eliminate the influence of foreign matter, calculate the characteristic amount of bacteria contained in the image, and determine the bacterial proliferation rate.

[0072] The overall structure of the bacteria inspection device in Example 2 is similar to Figure 5 The structures shown are the same, so the illustration and detailed description are omitted.

[0073] <Detection of foreign matter and aggregates and determination of bacterial growth rate based on characteristic quantities> Figure 7 This is an operational flow chart for determining the bacterial proliferation degree in Example 2. Figure 7 Explain the determination of bacterial proliferation rate.

[0074] (1) Step S600 In step S600, the image processing unit 506 receives the image captured by the imaging unit 505, stores it in the internal memory (not shown) of the image processing unit 506 or the memory (not shown) of the control unit 507, and then performs the processing in the subsequent steps. It is preferable to pre-determine the brightness value of the input image (input value determination). For example, if the brightness values ​​of all pixels are predicted to be 0 or 255, indicating an abnormality in the lighting unit 501 or an abnormality in the positional relationship between the inspection plate 502 and the platform 503, it is preferable to skip the subsequent processing and set it as an error.

[0075] Furthermore, if the incubation time is sufficient and the bacteria are overgrown, the entire image may be filled with agglomerated bacteria, making it difficult to distinguish between foreign matter and agglomerates. In such cases, the condition can be determined by abnormally low or high values ​​in the mean or standard deviation of the image brightness histogram. For example, only steps S203 to S204 may be executed, omitting steps S201 to S603.

[0076] (2) Steps S201 to S202, and Steps S203 to S204 Steps S201 to S202 and steps S203 to S204 are Figure 2 Steps S201 to S202 and steps S203 to S204 are the same, and their details are omitted.

[0077] (3) Step S601 Step S601 is executed only when a foreign object is detected in step S202 , and detects the area of ​​the foreign object in the image.

[0078] For example, using the binarized image obtained in step S201, a connected area can be regarded as a foreign body area. In addition, the image obtained in step S200 can also be re-binarized. In the binarization process, in order to detect foreign bodies that are darker, it is preferred to use a brightness value lower than the background as the binarization threshold for binarization, but in order to detect foreign bodies that are brighter than the background, a brightness value higher than the background can also be used as the binarization threshold, or the result of overlapping the two can be used. The threshold can be a pre-set fixed value, or it can be calculated, for example, like Otsu binarization, to calculate the inter-class separation in the histogram and dynamically determine the threshold for each image. Or, for example, in the case where the brightness in the image plane is uneven due to a deviation in the amount of illumination, the binarization threshold can be dynamically changed for each small area of ​​the image.

[0079] In addition, in order to distinguish agglomerated bacteria from foreign matter, additional processing can be performed on the binarized image. Figure 1C The agglomerated bacteria 103 shown here have localized areas of reduced brightness and areas of high brightness. This can hinder discrimination between agglomerated and foreign matter areas during binarization at a certain threshold. In such cases, it is effective to remove the agglomerated bacterial areas by repeatedly performing erosion or dilation processing using, for example, the Minkowski sum or Minkowski difference, leaving only the foreign matter areas.

[0080] (4) Step S602 In step S602, the foreign matter region is corrected and the bacterial (microparticle) region is calculated. Specifically, the binarized region calculated in step S204 includes both the bacterial and foreign matter regions. By performing a differential process with the foreign matter region calculated in step S601, the bacterial region is calculated by removing the foreign matter region.

[0081] (5) Step S603 In step S603, execute Figure 2In the steps S205 and S206 shown in FIG, after determining whether there are aggregates, the size and number of aggregates are calculated if there are aggregates. Figure 3C As shown, a certain determination threshold 302 is set in advance. Then, when it is determined in step S206 that there are aggregates, the cumulative signal is compared with the determination threshold 302, and the area where the signal exceeds the threshold is regarded as the area of ​​aggregates. Figure 3C In the example of , by approximating aggregates extending in the areas of 630 to 1030 pixels and 2300 to 2450 pixels in the horizontal direction to circles, it is determined that there are aggregates with a diameter of 400 pixels and a diameter of 150 pixels.

[0082] in addition, Figure 3C While the differential waveforms are integrated vertically across the image, they are also integrated horizontally. Similarly, by comparing the calculated vertical and horizontal aggregate information with the judgment threshold, the two-dimensional position and size of the aggregates can be calculated. While processing time increases, the number and area of ​​aggregates can be more accurately calculated, even if the image is not approximately circular.

[0083] (6) Step S604 In step S604, feature quantities are calculated based on the corrected bacterial regions calculated in step S602. Specifically, the number of small regions determined to be bacteria, the average area and total area of ​​each small region, and numerical values ​​representing the shape of each small region, such as roundness, circularity, aspect ratio, perimeter, and convexity, are calculated. The numerical values ​​representing the shape of each small region are preferably averaged, and preferably, the value obtained by averaging the number of small regions detected as bacteria is output.

[0084] Since the foreign matter region is corrected in step S602 , the influence of the foreign matter is removed from the feature amount calculated in step S604 .

[0085] For example, when calculating the number of small areas identified as bacteria and the total area of ​​each small area, simply subtracting the foreign matter area will not take into account the influence of bacteria that should be present in the foreign matter area, resulting in an overly low estimated value. In this case, by performing a calculation such as: feature quantity after subtraction × (area of ​​the entire image) / {(area of ​​the entire image) - (area of ​​the foreign matter)}, a value that estimates the influence of bacteria contained in the foreign matter area can be calculated.

[0086] Typically, image acquisition in step S600 is performed multiple times as the culture time elapses. Therefore, each step from S600 to S604 may be performed only once, or the steps from S600 to S604 may be repeated for multiple images captured at a certain interval, with image feature quantities calculated for each number of images. Furthermore, image feature quantities can be calculated for images captured with different concentrations of antibiotics or samples containing different concentrations of antibiotics.

[0087] (7) Step S605 In step S605, the bacterial proliferation level is determined based on the image feature information calculated in step S604. In the simplest approach, the number of small areas identified as bacteria or the total area of ​​these small areas can be used as an indicator of bacterial proliferation. Therefore, by comparing this feature with a pre-set threshold, a determination of bacterial proliferation can be made.

[0088] As the characteristic amount of the fine particles, the total area value of the fine particles, the total number of fine particles, the number of aggregates, or the area value of aggregates may be calculated to determine the proliferation degree of the fine particles.

[0089] Alternatively, a discriminant analysis model may be generated based on separately prepared data, and the image features may be input to the model to determine whether or not there is proliferation. The number of aggregates and the total area of ​​aggregates calculated in step S603 may also be used to determine whether or not there is proliferation. By combining the numerical information calculated in step S603 with the numerical information calculated in step S604, a more accurate determination can be made.

[0090] Generally, when the area occupied by aggregates in an image increases, bacteria tend to multiply, so this feature value serves as information indicating growth. Since bacterial agglutination ability varies depending on the species and strain, better results can be achieved by varying the judgment threshold or discriminant analysis model based on the species and strain.

[0091] Figure 8 This is a graph showing the dependence of the area value of the bacterial region on the culture time as an example of a feature value calculated by removing the influence of foreign matter. Figure 8 The vertical axis represents the area of ​​the bacterial region, and the horizontal axis represents the culture time.

[0092] The calculation of the dependence of the bacterial area value on the incubation time is performed in step S604. For example, if a shadow is present and the area captured by the shadow changes constantly, the feature value will repeatedly increase and decrease, as shown in feature value 700 before correction. This increase and decrease is caused by the foreign matter and may affect the accurate determination of the proliferation rate. Corrected feature value 701 shows a monotonically increasing trend, allowing calculation of the feature value after eliminating the influence of the foreign matter.

[0093] Figure 9 FIG. 1 is a functional block diagram of the determination processing unit 1A included in the control unit 507. The determination processing unit 1A executes Figure 7 action flow.

[0094] exist Figure 9 In the figure, the determination processing unit 1A includes an image acquisition unit 11, a first image binarization processing unit 12, a contour extraction processing logic and operation processing unit 13, a foreign matter presence determination unit 14, a foreign matter area detection unit 18, a second image binarization processing unit 15, an aggregate area determination unit 19, and a proliferation degree determination unit 19.

[0095] The image acquisition unit 11 executes Figure 7 The first image binarization processing unit 12 performs step S201. Furthermore, the contour extraction processing logical AND operation processing unit 13 performs step S203, the foreign matter presence determination unit 14 performs step S202, and the foreign matter region detection unit 18 performs step S601. Furthermore, the second image binarization processing unit 15 performs step S204, and the aggregate region determination unit 19 performs steps S602, S603, and S604. Then, the proliferation degree determination unit 20 performs step S605.

[0096] Since the second embodiment is configured as described above, similarly to the first embodiment, an inspection device and an inspection method can be provided that are capable of detecting foreign matter and aggregates of fine particles and accurately detecting the proliferation rate of fine particles.

[0097] (Example 3) Next, Example 3 will be described.

[0098] Example 3 is an example of applying Example 1 and Example 2 to obtain an image without foreign matter and calculate the characteristic value of bacteria. The overall structure of the bacteria inspection device in Example 3 is the same as that in Figure 5 The structures shown are the same, so the illustration and detailed description are omitted.

[0099] Figure 10 This is a flowchart of shooting and image processing in Example 3.

[0100] (1) Step S800 In step S800, according to Figure 2 The flow chart shown detects the presence of foreign matter and aggregates.

[0101] (2) Step S801 In step S801, the current number of shots is compared with a predetermined threshold. If the number is above the threshold, the process proceeds to step 804. If the number is below the threshold, the process proceeds to step S802. The number of shots is preferably 2 or 3, for example. Alternatively, an appropriate number of shots may be selected based on the number of holes.

[0102] (3) Step S802 In step S802, a determination is made as to whether foreign matter and aggregates were detected in step 800. If neither was detected, the process proceeds to step S804. If foreign matter was detected, the process proceeds to step S803. Even if aggregates were detected, the process proceeds to step S803. However, if the process requires more aggressive detection of aggregates, the process may proceed to step S804. For example, if the target bacteria are susceptible to agglutination and more sensitive detection of aggregates is desired, the process may proceed to step S803 only if foreign matter was detected.

[0103] (4) Step S803 In step S803, the relative positions of the inspection plate 502, the objective lens 504, and the imaging unit 505 are changed, and imaging is performed again. Since an estimated value of the location where foreign matter or aggregates are present can be calculated based on the results of the numerical analysis in steps 202 and 206, it is preferable to perform imaging again by, for example, moving the inspection plate 502 in a direction in which no foreign matter or aggregates are expected to be present.

[0104] (5) Step S804 In step S804, according to Figure 7 For example, if it is determined that there is no foreign matter or aggregate in step S800, even if the calculation is skipped, Figure 7 There is no problem in implementing the processing by steps S201 to S202 and steps S601 to S603.

[0105] <Example of result output> An example is shown in which the image processing result of the present disclosure is delivered to the user via the input / output unit 508 .

[0106] Figure 11 This is a diagram showing an example of the structure of a GUI (Graphical User Interface) for outputting results. Figure 11In the embodiment, the GUI is composed of, for example, a measurement condition display unit 900, a measurement result display unit 901, and an apparatus status display unit 902. These display units are preferably displayed on monitors, and the display units are switched by inputting, for example, a label representing the display unit or a button indicating switching.

[0107] The measurement condition display unit 900 displays conditions for performing bacterial testing and allows input and output. For example, in addition to displaying measurement conditions related to the test target bacterial species or drugs, test time, image capture cycle, etc., it can also accept input from the user.

[0108] The device status display unit 902 displays information indicating the device status and allows input and output. For example, it displays device error and warning information, temperature and humidity information, device door opening and closing information, information related to device maintenance, and various device settings, and can also receive input from the user.

[0109] The measurement result display unit 901 outputs the image processing results calculated according to the present disclosure. For example, it displays the ID of the specimen being inspected, the current measurement status of the specimen, the measurement results, and the image processing output 903. The measurement status column for the specimen preferably displays the measurement time up to now, the time until the measurement ends, and instructions for the user to remove the plate.

[0110] The measurement results section preferably displays errors and warnings during measurement, error information determined through image processing, and information such as the presence of foreign matter or aggregates. Furthermore, a plot showing the temporal changes in numerical information calculated through image processing is preferred, useful for visually displaying the extent of bacterial growth to the user. Specifically, each point within the quadrilateral in this plot represents the point at which measurement was performed. The presence of foreign matter or aggregates can be clearly indicated by a different color or shape. Selecting this plot preferably displays detailed information about the error or an image of the error to the user.

[0111] Figure 12 This is a functional block diagram of the determination processing unit 1B included in the control unit 507. The determination processing unit 1B performs Figure 10 The determination processing unit 1B is Figure 6 The functional blocks shown in the figure have been supplemented with a capture count determination unit 21, a recapture processing unit 22, and an image processing feature quantity calculation unit 23. The capture count determination unit 21 performs the operation of step S801. The recapture processing unit 22 performs steps S802 and S803, and the image processing feature quantity calculation unit 23 performs the operation of step S804.

[0112] Since the third embodiment is configured as described above, similarly to the first embodiment, an inspection device and an inspection method can be provided that are capable of detecting foreign matter and aggregates of fine particles and accurately detecting the proliferation rate of fine particles.

[0113] The function of above-mentioned each embodiment can also be realized by the program code of software.In this case, the storage medium having recorded program code is provided to system or device, and the computer (or CPU or MPU) of this system or device reads the program code stored in the storage medium.

[0114] In this case, the program code read from the storage medium itself implements the functions of the above-mentioned embodiments, and the program code itself and the storage medium storing the program code constitute the present disclosure. As the storage medium for providing such program code, for example, a floppy disk, CD-ROM, DVD-ROM, hard disk, optical disk, magneto-optical disk, CD-R, magnetic tape, non-volatile memory card, ROM, etc. can be used.

[0115] Furthermore, based on the instructions of the program code, the OS (operating system) running on the computer performs part or all of the actual processing, and the functions of the above-mentioned embodiments can be realized through this processing. Furthermore, after the program code read from the storage medium is written to the memory on the computer, the CPU of the computer can perform part or all of the actual processing based on the instructions of the program code, and the functions of the above-mentioned embodiments can be realized through this processing.

[0116] Moreover, the program code of the software that implements the functions of the embodiment can be distributed via a network and stored in a storage unit such as a hard disk or memory of the system or device or a storage medium such as CD-RW, CD-R, etc. When in use, the computer (or CPU, MPU) of the system or device reads out the program code stored in the storage unit or the storage medium and executes it.

[0117] Finally, it should be understood that the processes and techniques described herein are not inherently associated with any particular apparatus and can be implemented by any suitable combination of components. In addition, various types of general-purpose equipment can be used according to the teachings described herein. It may be found that it is beneficial to build a dedicated apparatus to perform the steps of the methods described herein. In addition, various inventions can be formed by appropriately combining the multiple structural elements disclosed in the embodiments. For example, several structural elements can be deleted from all the structural elements shown in the embodiments. Furthermore, the structural elements involved in different embodiments can be appropriately combined.

[0118] While the present disclosure has been described with reference to specific examples, these are intended to be illustrative in all respects and not restrictive. Those skilled in the art will appreciate that there are many combinations of hardware, software, and firmware suitable for implementing the present disclosure. For example, the software described may be implemented in a wide variety of programming or scripting languages, such as assembler, C / C++, Perl, Shell, PHP, Java (registered trademark), and the like.

[0119] In addition, in the above embodiment, the control lines and information lines necessary for explanation are shown, but this is not limited to showing all the control lines and information lines necessary for the product. All structures can be connected to each other.

[0120] The present disclosure is not limited to the above-mentioned embodiments, but includes various modified examples. The above-mentioned embodiments are detailed descriptions for the purpose of explaining the technology of the present disclosure in an understandable manner, and are not limited to including all the structures described. In addition, a part of the structure of a certain embodiment can be replaced with the structure of another embodiment, and the structure of another embodiment can also be added to the structure of a certain embodiment. In addition, with respect to a part of the structure of each embodiment, other structures can also be added, deleted, or replaced.

[0121] In addition, the present disclosure includes the following agglomerate inspection method.

[0122] (1) A method for detecting aggregates of particles, wherein an image of the inside of a container holding a liquid containing particles is acquired to detect the presence or absence of aggregates of the particles, comprising: binarizing the image to generate a first binarized image, accumulating the first binarized image at least longitudinally or transversely, and performing numerical analysis on the accumulated value to detect foreign matter and determine the presence or absence of the foreign matter; and, if it is determined that the foreign matter is present, changing the relative position of the container and the imaging unit to acquire the image inside the container again.

[0123] (2) In the agglomerate detection method of (1), the steps include: extracting the contour of the image as a contour extraction image, binarizing the extracted contour extraction image and the composite image of the image to generate a second binarized image, and detecting the microparticle area; calculating the difference between the first cumulative signal accumulated by the first binarized image and the second cumulative signal accumulated by the second binarized image; and determining the presence or absence of agglomerates by numerically analyzing the differential waveform of the differential value. Label Description

[0124] 1. 1A, 1B judgment processing unit 11 Image acquisition unit 12 1st image binarization processing unit 13Contour extraction processing logic and operation processing unit 14 Foreign matter presence determination unit 15 Second image binarization processing unit 16 Accumulated signal difference processing unit 17 Agglutination determination unit 18 Foreign matter area detection unit 19 Aggregate Area Determination Unit 20 Reproduction Degree Determination 21 Shooting times determination unit 22 Reshooting Processing Department 23 Image processing feature quantity calculation unit 24 Foreign matter presence determination unit 25. Microparticle area detection unit 100 Foreign Objects 101 Background 102 bacteria 103 Agglutinated bacteria 300 Cumulative signal after binarization of the first image 301 Differential signal of cumulative signal 302 Decision Threshold 400 Approximate straight line 401 Corrected differential signal 500 Bacteria Inspection Device 501 Lighting Department 502 Inspection Board 503 Platform 504 objective lens 505 Filming Department 506 Image Processing Department 507 Control Department 508 Input and Output 509 Humidification and Temperature Control Unit 510 sample solution 511 container (hole) 700 Feature quantity before correction 701 Corrected feature quantity 900 Measurement condition display section 901 Measurement result display unit 902 Device status display unit 903 Output result of image processing.

Claims

1. An agglomerate inspection device comprising an image acquisition unit for acquiring an image of a container holding a liquid containing microparticles, and detecting the presence or absence of agglomerates of the microparticles, the agglomerate inspection device comprising: a foreign matter presence determination unit that binarizes the image to generate a first binarized image and determines the presence of foreign matter; a microparticle region detecting unit that extracts a contour of the image as a contour extraction image, binarizes a composite image of the extracted contour extraction image and the image to generate a second binarized image, and detects a microparticle region; an integrated signal difference processing unit that calculates a difference value between a first integrated signal obtained by integrating the first binarized image and a second integrated signal obtained by integrating the second binarized image; as well as an aggregation determination unit that determines the presence or absence of aggregates by numerically analyzing the differential waveform of the differential value, The agglomerate inspection device detects the presence or absence of at least two different types of objects included in the image.

2. The agglomerate inspection device according to claim 1, wherein The aggregation presence / absence determination unit may determine the aggregate by calculating a standard deviation of the differential waveform as a numerical analysis of the differential value and comparing the standard deviation with a determination threshold.

3. The agglomerate inspection device according to claim 1, wherein The aggregation presence / absence determination unit calculates a derivative of the differential waveform as a numerical analysis of the differential value, compares the derivative with a determination threshold, detects a peak, and determines the aggregate.

4. The agglomerate inspection device according to claim 1, wherein As a numerical analysis of the integrated waveform of the first binarized image, the standard deviation of the integrated waveform is calculated and compared with a determination threshold value to determine the presence or absence of foreign matter.

5. The agglomerate inspection device according to claim 1, wherein As a numerical analysis of the integrated waveform of the first binarized image, the derivative of the differential waveform is calculated and compared with a determination threshold value to detect a peak and determine the presence or absence of foreign matter.

6. The agglomerate inspection device according to any one of claims 1 to 5, wherein: The microparticle region detecting unit acquires the composite image by performing a logical AND operation on the contour-extracted image and the image, binarizes the composite image to generate the second binarized image, and detects the microparticle region.

7. The agglomerate inspection device according to any one of claims 1 to 5, wherein: Also includes: an aggregate region determination unit that calculates a region of the foreign matter determined to contain the foreign matter and a region containing at least the fine particles, determines a region of the fine particles excluding the region of the foreign matter, and calculates a characteristic value of the fine particles contained in the region of the fine particles; as well as A proliferation degree determination unit determines the proliferation degree of the fine particles.

8. The agglomerate inspection device according to claim 7, wherein: The proliferation degree determination unit determines the proliferation degree of the fine particles by calculating at least one of the total area value of the fine particles and the total number of the fine particles, and the number of the aggregates or the area value of the aggregates as characteristic quantities of the fine particles.

9. An agglomerate inspection device comprising an imaging unit for capturing an image of a container containing a liquid containing microparticles and detecting the presence or absence of agglomerates of the microparticles, the agglomerate inspection device comprising: a foreign matter presence determination unit that binarizes the image to generate a first binarized image, accumulates the first binarized image at least in a vertical or horizontal direction, and performs numerical analysis on the accumulated value to detect foreign matter and determine the presence of the foreign matter; as well as A re-imaging processing unit changes the relative position between the container and the imaging unit and acquires an image of the inside of the container again when it is determined that the foreign matter is present.

10. The agglomerate inspection device according to claim 9, wherein include: a microparticle region detecting unit that extracts a contour of the image as a contour extraction image, binarizes a composite image of the extracted contour extraction image and the image to generate a second binarized image, and detects a microparticle region; an integrated signal difference processing unit that calculates a difference value between a first integrated signal obtained by integrating the first binarized image and a second integrated signal obtained by integrating the second binarized image; as well as The aggregation presence / absence determination unit determines the presence / absence of aggregates by numerically analyzing the differential waveform of the differential value.

11. A method for inspecting aggregates, wherein an image of the interior of a container holding a liquid containing microparticles is captured to detect the presence or absence of aggregates of the microparticles, characterized in that: Binarize the image to generate a first binary image, and determine whether there is a foreign object. Extracting the contour of the image as a contour extraction image, binarizing the extracted contour extraction image and a composite image of the image to generate a second binarized image, and detecting the microparticle region. calculating a difference between a first cumulative signal obtained by accumulating the first binarized image and a second cumulative signal obtained by accumulating the second binarized image, The presence or absence of aggregates is determined by numerically analyzing the differential waveform of the differential value. The presence or absence of at least two objects of different types contained in the image is detected.

12. The method for inspecting aggregates according to claim 11, wherein: The aggregate is determined by numerically analyzing the differential waveform of the differential value and calculating a standard deviation of the differential waveform and comparing the standard deviation with a determination threshold.

13. The method for inspecting aggregates according to claim 11, wherein: As the numerical analysis of the differential waveform of the differential value, the derivative of the differential waveform is calculated, and the peak is detected by comparing with a determination threshold value to determine the aggregate.

14. The method for inspecting aggregates according to claim 11, wherein: As a numerical analysis of the integrated waveform of the first binarized image, the standard deviation of the integrated waveform is calculated and compared with a determination threshold value to determine the presence or absence of foreign matter.

15. The agglomerate inspection method according to claim 11, wherein As a numerical analysis of the integrated waveform of the first binarized image, the derivative of the differential waveform is calculated and compared with a determination threshold value to detect a peak and determine the presence or absence of foreign matter.

16. The method for inspecting aggregates according to any one of claims 11 to 15, wherein: The synthesized image is acquired by performing a logical AND operation on the contour-extracted image and the image, and the synthesized image is binarized to generate the second binarized image, thereby detecting the microparticle region.

17. The method for inspecting aggregates according to any one of claims 11 to 15, wherein: The area of ​​the foreign matter judged to contain the foreign matter and the area containing at least the microparticles are calculated respectively, the area of ​​the microparticles after excluding the area of ​​the foreign matter is determined, the characteristic quantity of the microparticles contained in the area of ​​the microparticles is calculated, and the reproduction degree of the microparticles is determined.

18. The agglomerate inspection method according to claim 17, wherein: At least the total area value of the fine particles and the total number of the fine particles, and the number of the aggregates or the area value of the aggregates are calculated as characteristic quantities of the fine particles to determine the proliferation degree of the fine particles.

Images