Inspection apparatus and inspection method

Abstract

The inspection apparatus includes an image capturing unit that captures an image of the inside of a container that holds a liquid containing microparticles, and an image processing unit that calculates a first statistical amount related to an expansion of a brightness value of the image. On the basis of the result of comparison between the brightness value of the image and a predetermined brightness threshold value, a region having an abnormal brightness value in the image is removed, correction processing is performed for sharpening the image from which the region having the abnormal brightness value has been removed, and a second statistic relating to the expansion of the brightness value of the image after the correction processing is calculated. The correctness of the correction process is determined on the basis of the result of comparison between the ratio of the first statistic and the second statistic and a predetermined statistic ratio threshold value, the degree of proliferation of the fine particles is determined on the basis of the feature amount of the image for which the correction process is determined to be proper, and the determination result of the degree of proliferation is output. As a result, in an image including a particulate measurement object such as proliferating bacteria, no matter how different the degree of proliferation is, the bacteria can be extracted more accurately, and a region in which the bacteria having the same degree of proliferation are present can be detected, regardless of whether or not foreign matters other than the bacteria have been captured.

Description

Technical Field

[0001] The present invention relates to an inspection device and an inspection method. Background Art

[0002] The international spread of infectious diseases is a major concern for human health worldwide. In recent years, the continued spread of various emerging infectious diseases has spurred the development of new testing methods and vaccines to identify and detect causative viruses and bacteria. For example, among bacterial infections, the emergence of multidrug-resistant bacteria is a significant global problem. The US CDC identifies carbapenem-resistant Acinetobacter, carbapenem-resistant Enterobacteriaceae, Clostridioides difficile, and the fungus Candida auris as major threats. When infections are collectively caused by these drug-resistant bacteria, the risk of mortality is high. Therefore, it is crucial to rapidly identify and susceptibility-test the causative bacteria, detect drug-resistant bacteria, and promptly implement appropriate measures. For identification (ID) of pathogens, mass spectrometry has become the mainstream method, enabling testing within an hour of positive bacterial colony or blood culture samples.

[0003] Furthermore, turbidimetric testing is the mainstream method for evaluating bacterial sensitivity to drugs (AST). However, to expedite testing, a method has become available that uses a microscope to image bacteria and measure their proliferation with greater sensitivity than previous methods. Generally, bacteria double in number according to a certain growth cycle. Therefore, the number of bacteria in images captured by a microscope also doubles in number, increasing exponentially. Therefore, a wide dynamic range is required to accurately capture changes in proliferation over time.

[0004] As conventional techniques for detecting bacterial proliferation rates based on images, for example, techniques described in Patent Documents 1 to 3 are known.

[0005] Patent Document 1 discloses a method that captures an image of bacteria, extracts bacterial regions through image processing, and then detects bacterial areas. Specifically, the bacterial image is binarized using a threshold value and then combined with the resulting binarized image after contour extraction to detect bacterial areas. This method allows for highly accurate bacterial area extraction when the number of bacteria in the image is small and each bacterium is captured in isolation.

[0006] Patent Document 2 discloses a method for adjusting contrast by expanding a histogram of image brightness values ​​so that the brightness of an output image does not fluctuate even if the brightness of an input image changes.

[0007] Patent Document 3 discloses an image-based foreign object detection method. Specifically, it discloses a method for adjusting contrast by expanding the image's brightness histogram to achieve optimal foreign object detection. This method enhances contrast by correcting the effects of pixel defects on the image. Prior art literature Patent Literature

[0008] Patent Document 1: U.S. Patent No. 10,902,592 Patent Document 2: Japanese Patent Application Laid-Open No. 2001-119683 Patent Document 3: Japanese Patent Application Laid-Open No. 2006-194869 Summary of the Invention

[0009] However, the above-mentioned prior art has the following problems.

[0010] In the prior art described in Patent Document 1, it is difficult to separate each bacterium in an image and accurately detect it when the bacteria are proliferating and not photographed in isolation. When the bacteria are proliferating and not photographed in isolation, the outlines of the bacteria become common, making it difficult to separate the boundaries of close bacteria, and multiple bacteria are identified as a large block. Therefore, it is not easy to separate each bacterium for detection and identification, and to detect the number of bacteria within a sufficiently wide dynamic range. In addition, the amount of light irradiated to the detection unit decreases as the bacteria proliferate, and the difference in image brightness between the background area and the bacterial area in the image becomes smaller. Therefore, it is necessary to appropriately adjust the contrast to extract the bacteria. However, in addition to the case where the number of bacteria in the image is different, it is difficult to accurately detect the bacterial area when foreign matter is present in the image.

[0011] Furthermore, the prior art described in Patent Document 2 attempts to accurately detect foreign matter other than the target object, but does not accurately detect the target object. Furthermore, the prior art is not suitable for detecting objects in images where the number of measurement targets contained in the image varies within a range of bits, such as bacteria.

[0012] Furthermore, in the prior art described in Patent Document 3, when foreign matter other than the target object is captured in the image, correction is performed to enhance contrast, and the enhancement processing is performed while excluding the influence of the foreign matter, thereby enabling more accurate image processing. However, it is unclear whether the degree of contrast enhancement after correction is appropriate, and there is room for improvement.

[0013] The present invention is completed based on the above situation, and its purpose is to provide an inspection device and an inspection method that can more accurately extract bacteria and detect areas where bacteria corresponding to the proliferation degree exist, regardless of the difference in the proliferation degree or the presence or absence of foreign matter other than bacteria in an image of a particulate measurement object such as proliferating bacteria. Technical means for solving technical problems

[0014] The present application includes multiple means for solving the above-mentioned problems. To cite one example, it includes: a shooting unit that shoots an image of a container holding a liquid containing microparticles; and an image processing unit that performs image processing on the image shot by the shooting unit, the image processing unit calculates a first statistic involved in the expansion of the brightness value of the image, removes the area of ​​abnormal brightness value of the image based on the comparison result between the brightness value of the image and a predetermined brightness threshold, performs correction processing to sharpen the image after the area of ​​abnormal brightness value is removed, calculates a second statistic involved in the expansion of the brightness value of the image after the correction processing, determines the appropriateness of the correction processing based on the comparison result between the ratio of the first statistic and the second statistic and a predetermined statistic ratio threshold, determines the proliferation degree of the microparticles based on the feature quantity of the image determined to be appropriate by the correction processing, and outputs the determination result of the proliferation degree. Effects of the Invention

[0015] According to the present invention, in an image containing particulate measurement objects such as growing bacteria, bacteria can be extracted more accurately regardless of differences in the growth degree or the presence or absence of foreign matter other than bacteria, and regions containing bacteria corresponding to the growth degree can be detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a diagram schematically showing the structure of a bacteria inspection device as an example of the inspection device. Figure 2 This is a functional block diagram schematically showing the configuration of the processing functions of the image processing unit. Figure 3 1 and 2 are diagrams for explaining the characteristics of an image corresponding to the bacterial growth rate, and are diagrams showing an image before culture (before bacterial growth). Figure 4 1 and 2 are diagrams for explaining the characteristics of an image corresponding to the bacterial growth rate, and are diagrams showing an image when bacterial growth has progressed to a certain extent. Figure 5 The diagrams illustrate the characteristics of an image corresponding to the bacterial growth rate, and show an image when bacterial growth has significantly progressed. Figure 6: is a diagram showing an example of a histogram of brightness values ​​of an image. Figure 7 It shows the Figure 4 The diagram shows an example of an image of bacteria when an appropriate intensity correction process (contrast enhancement) is performed on the image of bacteria. Figure 8 The following figure shows the difference between the case where the histogram is expanded as a correction process (contrast enhancement) and the case where the histogram is expanded as a correction process (contrast enhancement) Figure 4 and Figure 7 A graph of the histogram corresponding to the bacterial image. Figure 9 Graphs showing brightness histograms before and after histogram enhancement processing when a dark object is present in an image. Figure 10 Graphs showing brightness histograms before and after histogram enhancement processing when almost no bacteria are captured in an image. Figure 11 It shows Figure 10 FIG. 1 is a diagram showing an example of an image after contrast enhancement processing (after correction processing). Figure 12 This is a flowchart showing the contents of the bacteria detection process in the image processing unit of the bacteria inspection apparatus according to the first embodiment. Figure 13 This diagram schematically illustrates a single bacterium in an image. Figure 14 It shows Figure 13 A diagram showing the relationship between the position and brightness value of pixels in . Figure 15 It is a diagram for explaining the effects of image processing in the image processing unit. Figure 16 FIG. 1 is a diagram showing an example of an image processing result output to the input / output unit of the control device. Figure 17 This is a diagram showing an example of a histogram of brightness when many foreign objects are captured in an image. Figure 18 This is a flowchart showing a part of the image processing according to the second embodiment. DETAILED DESCRIPTION

[0017] This embodiment discloses, for example, the following technology: for an image of microparticles captured by a microscope in a hole holding a liquid containing microparticles, contrast enhancement is performed, the appropriateness of the contrast enhancement is determined, and the degree of proliferation of the microparticles is accurately determined. Below, the embodiment of the present disclosure is described with reference to the accompanying drawings. In addition, although the accompanying drawings show specific embodiments based on the principles of the present technology, the accompanying drawings are for the purpose of facilitating the understanding of the present technology and are not used to restrictively interpret the present disclosure. In addition, in this embodiment and the accompanying drawings, the same reference numerals are attached to components having the same function, and repeated descriptions are omitted. In addition, as microparticles, bacteria, archaea, fungi, living cells, etc. can be cited, and here bacteria are described as one of the microparticles.

[0018] <Implementation Method 1> Reference Figures 1 to 16 Embodiment 1 of the present invention will be described.

[0019] (Bacteria Inspection Device 100) Figure 1 This is a diagram schematically showing the structure of a bacteria inspection device as an example of the inspection device according to this embodiment.

[0020] Figure 1 The bacteria inspection apparatus 100 is generally composed of an illumination unit 110, an inspection plate 120, a stage 130, an objective lens 140, an imaging unit 150, an image processing unit 160, and a humidification and temperature control unit 190. Furthermore, the bacteria inspection apparatus 100 includes a control unit 200 that controls the overall operation of the bacteria inspection apparatus 100.

[0021] The test plate 120 has one or more wells, preferably 96 or 384 wells, with each well holding a sample solution 180. The sample solution 180 contains a bacterial sample and may also contain a culture medium for bacterial growth or one or more antimicrobial agents at pre-adjusted concentrations. Because the number of test plates 120 varies depending on the type of bacteria being tested and the type of drug, it is preferable to introduce multiple different test plates 120 into the bacterial testing apparatus 100 for each test. The test plates 120 are preferably distinguished by patient ID, test bacteria, and the type of drug used, for example, using barcode labels, QR codes, RFID tags, or the like for individual identification.

[0022] The lighting unit 110 irradiates light in the direction of the inspection plate 120. As the lighting unit 110, a light source such as a white light such as a lamp or an LED containing light in a specific wavelength region can be used. The light irradiated from the lighting unit 110 to the inspection plate 120 passes through the holes in the inspection plate 120 and the sample solution 180, is focused by the objective lens 140, and is imaged and photographed by the photographing unit 150. For example, the platform 130 is moved under the control of the control device 200, so that the relative position of the hole of the inspection plate 120 and the photographing unit 150 changes, so that different holes can be photographed. In addition, the control device 200 also controls the photographing action, which is performed at a predetermined time interval (for example, every 30 minutes), and the resulting image is processed by the image processing unit 160 and sent to the control device 200.

[0023] The objective lens 140 is preferably focused on the bottom surface of the well of the inspection plate 120, but may also be focused on the interior of the sample solution 180 away from the bottom surface. Furthermore, multiple locations within the well may be photographed, and multiple images of the interior of the sample solution 180 away from the bottom surface of the well of the inspection plate 120 may also be taken.

[0024] The humidification and temperature control unit 190 adjusts the temperature and humidity of the entire imaging system for acquiring images, for example, to approximately 35° C. and approximately 30% humidity, thereby effectively promoting bacterial growth in the inspection plate 120 .

[0025] The control device 200 is, for example, a general computer, and controls the entire bacteria inspection apparatus 100, including the imaging unit 150 and the image processing unit 160. The control device 200 also includes an input / output unit 210, which is responsible for inputting various information related to inspections by the bacteria inspection apparatus 100, such as measurement conditions and set values, and receiving and outputting (displaying) inspection results. The input / output unit 210 is, for example, a display, a mouse, a keyboard, a touch panel, or the like.

[0026] (Image Processing Unit 160) Figure 2 This is a functional block diagram schematically showing the configuration of the processing functions of the image processing unit.

[0027] Figure 2In the embodiment, the image processing unit 160 performs a bacteria detection process for calculating the bacterial proliferation rate from the image obtained by the imaging unit 150 and outputting the calculated bacterial proliferation rate under the control of the control device 200. The image processing unit 160 is generally configured to include: an image acquisition unit 161 for acquiring the image captured by the imaging unit 150; a statistic calculation unit 162 for calculating a statistic related to the expansion of the brightness value of the image, etc. acquired by the image acquisition unit 161; an abnormal brightness area calculation unit 163 for removing an area of ​​abnormal brightness value from the image based on a comparison result between the brightness value of the image and a predetermined brightness threshold; and a correction processing unit 164 for performing a correction process to sharpen the image after removing the area of ​​abnormal brightness value. A validity determination unit 166 determines the validity of the correction processing by the correction processing unit 164 based on a comparison result between the ratio of the statistic of the image before correction processing and the statistic of the image after correction processing and a predetermined statistic ratio threshold value; an adjustment unit 165 adjusts the reference value used for the correction processing in the correction processing unit 164 when the validity determination unit 166 determines that the correction processing by the correction processing unit 164 is inappropriate; and an image processing output unit 167 determines the proliferation degree of the microparticles based on the feature quantity of the image determined by the validity determination unit 166 as the correction processing by the correction processing unit 164 is valid, and outputs the determination result.

[0028] Hereinafter, the bacteria inspection process performed by the bacteria inspection apparatus 100 (control device 200 ) including the image processing unit 160 (bacteria detection process) will be described in detail.

[0029] (Basic Principle) First, the basic principle of the bacteria inspection process performed by the bacteria inspection apparatus according to this embodiment will be described.

[0030] Figures 3 to 5 is a diagram illustrating the characteristics of an image corresponding to the bacterial proliferation rate. Figure 3 The image before culture (before bacterial growth) is shown as an example. Figure 4 This shows an image of bacteria growing to a certain extent. Figure 5 An example of an image showing a case where bacterial growth is progressing significantly is shown. Figures 3 to 5 In FIG, a grayscale image with 256 levels is illustrated using 8 bits.

[0031] like Figures 3 to 5 As shown, the image has as main components the region of the background 300 , the region of the bacteria 301 , and the region of foreign matter other than the background 300 and the bacteria 301 .

[0032] For example, Figure 3 As shown, in the image before culture (before bacterial growth), the background 300 is mostly formed, and the bacteria 301 are minute areas.

[0033] In addition, if Figure 4 As shown, in the image after culturing (when bacterial growth has progressed to a certain extent), the bacteria divide and grow, and the area of ​​bacteria 301 increases. In this case, the area of ​​background 300 can also be confirmed.

[0034] Furthermore, if Figure 5 As shown, when proliferation has progressed significantly, the background 300 region is barely recognizable, and most of the image is filled with the region of bacteria 301 .

[0035] Figure 6 : is a diagram showing an example of a histogram of brightness values ​​for an image, where the horizontal axis shows brightness in the range of 0 to 255, and the vertical axis shows the number of pixels of the corresponding brightness value. Figure 6 , the image before bacterial proliferation is shown ( Figure 3 ) corresponding to the histogram A, and the image when the bacterial proliferation progresses to a certain extent ( Figure 4 ) and the histogram B corresponding to the image when the bacterial proliferation progresses significantly ( Figure 5 ) The corresponding histogram C.

[0036] In histogram A, corresponding to the image before bacterial proliferation, the difference between its minimum and maximum values ​​is small, resulting in a steep shape. This is because the majority of the image consists of the background 300 area. As bacteria begin to proliferate, the area of ​​bacteria 301 increases. The interior of the area of ​​bacteria 301 appears brighter than the background, while the outline of bacteria 301 appears darker. Therefore, as the bacteria proliferate, the width of the histogram increases, as shown in histogram B. As the bacteria proliferate further, scattered light creates areas of localized increased and decreased light intensity, further widening the width, as shown in histogram C. As the bacteria proliferate, the histogram of the image changes, as shown in histograms A, B, and C. Not only does the width of the histogram change, but the peak position also shifts. Furthermore, the histogram changes become more complex depending not only on the bacterial proliferation rate but also on the bacterial species and strain, thus obviating the need for a regular pattern. Furthermore, the presence of foreign matter (such as dust, dirt, air bubbles, or biological cells other than bacteria) in the image further complicates the histogram. Therefore, it is difficult to perform correction processing such as contrast enhancement processing at an appropriate intensity for various images whose histograms change in complex ways depending on various conditions in the conventional technology.

[0037] (Example of Correction Processing with Appropriate Strength) Figure 7 It shows the Figure 4 The diagram shows an example of an image of bacteria when an appropriate intensity correction process (contrast enhancement) is performed on the image of bacteria. Figure 8 The following figure shows the difference between the case where the histogram is expanded as a correction process (contrast enhancement) and the case where the histogram is expanded as a correction process (contrast enhancement) Figure 4 and Figure 7 A graph of the histogram corresponding to the bacterial image.

[0038] like Figure 7 As shown, when Figure 4 The image of bacteria shown here has contrast enhancement performed as a correction process, and when the contrast enhancement works as expected, the Figure 4 In contrast, the difference in lightness between the brighter area inside the bacteria 301 and the lower lightness area of ​​the outline of the bacteria 301 is enhanced. Figure 7 In the bacterial image shown, the brightness difference between the background 300 and the bacteria 301 is sufficiently large, making it easier to distinguish between the two and calculate the bacterial image feature value. Furthermore, as one correction method, histogram expansion as disclosed in Patent Document 2 (Japanese Patent Application Laid-Open No. 2001-119683) can be used.

[0039] like Figure 8 As shown, in Figure 4 When performing histogram expansion on the bacterial image shown in the figure as contrast enhancement, first, when the number of pixels in the pre-correction brightness histogram B is accumulated in descending order of brightness value, the brightness value at which the accumulated value exceeds the reference is set as the lower limit value B1_min of the saturation brightness. Alternatively, the reference value can be selected as the ratio of the target image to all pixels. For example, if a ratio of 1% is set as the reference value for a 100×100 pixel image, the reference value is 100, and the lowest brightness value when the accumulated value of the number of pixels exceeds 100 becomes the lower limit value B1_min of the saturation brightness. In other words, by increasing or decreasing this ratio (reference value), the strength of the correction process (contrast enhancement process) is changed. Similarly, when the number of pixels in the pre-correction brightness histogram B is accumulated in ascending order of brightness value, the brightness value when the accumulated value exceeds the reference is set as the upper limit value B1_Max of the saturation brightness. Finally, the brightness of pixels corresponding to the lower limit value B1_min and the upper limit value B1_Max of the saturated brightness are considered to be the minimum and maximum brightness of the image, respectively (for example, 0 and 255 in the case of an 8-bit, 256-level image), and the corrected histogram B1 is obtained by uniformly expanding the histogram. In addition, by expanding the pre-corrected histogram B, the corrected histogram B1 becomes more discrete.

[0040] (Example of Correction Processing for Inappropriate Strength) Compared to Figure 7 and Figure 8 Although the correction processing is performed with appropriate strength, there are cases where the expected enhancement processing is not performed depending on the image conditions.

[0041] Figure 9Graphs showing brightness histograms before and after histogram enhancement processing when a dark object is present in an image.

[0042] like Figure 9 As shown, when an image contains dust or bubbles, incident light is blocked, resulting in areas of low brightness in the image. Histogram D includes not only the background peak D_P1 but also the foreign matter peak D_P2. When a simple histogram expansion is performed on an image with this histogram, observing the histogram D before and histogram D1 after correction, it can be seen that the shape of the histograms remains virtually unchanged. This is equivalent to not performing histogram enhancement (correction) processing, and image processing such as distinguishing between background and bacteria and extracting features may not be performed with sufficient accuracy.

[0043] also, Figure 10 Graphs showing brightness histograms before and after histogram enhancement processing when almost no bacteria are captured in an image.

[0044] like Figure 10 As shown, the histogram E before correction has a steep peak near the center of the luminance range, but the peak of the histogram E1 after correction becomes gentle. Figure 11 It shows Figure 10 FIG. 1 is a diagram showing an example of an image after contrast enhancement processing (after correction processing). Figure 11 In this example, the contrast is enhanced not only for the outline of the bacteria 301 region but also for the scratch 302 region in the background 300. This is excessive histogram enhancement, and scratches and other objects other than the bacteria 301 being inspected are clearly captured in the image. This is not considered appropriate correction.

[0045] In view of the above situation, this embodiment describes a detection device and a detection method that can perform accurate correction processing even when the density of particles in an image greatly changes due to proliferation, such as bacteria, and contains foreign matter.

[0046] (Bacteria detection and processing) Figure 12 This is a flowchart showing the contents of the bacteria detection process in the image processing unit of the bacteria inspection apparatus according to the present embodiment.

[0047] like Figure 12As shown, in the bacteria detection process, the image acquisition unit 161 of the image processing unit 160 first acquires the image captured by the shooting unit 150. Specifically, the image acquisition unit 161 receives the image captured by the shooting unit 150 and stores it in the internal memory (not shown) of the image processing unit 160 or the memory (not shown) of the control device 200. In addition, in the image captured under preferred conditions, the inside of the bacterial area is often photographed as whiter and the outline is often photographed as blacker, but it is not limited to this. In addition, in the subsequent description, the image is set to an 8-bit grayscale image, with a pixel value of 0 being black and a pixel value of 255 being white. In addition, the processing of this embodiment is effective even for grayscale images other than 8 bits or black and white inverted images. In addition, with respect to color images, the processing of this embodiment can also be applied by converting them into color spaces such as R, G, B or H, S, V.

[0048] When the processing of step S100 is completed, the statistic calculation unit 162 then calculates the statistic (first statistic) related to the expansion of the brightness value of the image acquired by the image acquisition unit 161 (step S110). Here, the statistic is preferably information representing a pixel value histogram, that is, for example, the average value and standard deviation of the brightness of all pixels. Information on the peak position of the histogram can be obtained from the average value of all pixels, and information on the expansion of the peak can be obtained from the standard deviation of the histogram. In addition, as a statistic, it is not necessarily as described above. For example, the mode value can be used instead of the average value of the brightness of all pixels, and the half-value width of the peak can be used instead of the standard deviation.

[0049] Upon completion of step S120, the abnormal brightness region calculation unit 163 calculates the image's abnormal brightness region based on the comparison results between the image's brightness values ​​and a predetermined brightness threshold, and removes the calculated region from subsequent processing (step S120). In other words, the abnormal brightness region caused by foreign matter in the image is calculated. Specifically, the illuminating light is often absorbed, reflected, or scattered by foreign matter, and therefore often falls near the lower or upper brightness limits. Therefore, a lower or upper brightness threshold is set as the brightness threshold, and pixels with brightness values ​​below the lower threshold or exceeding the upper threshold are extracted as pixels in the abnormal brightness region. In other words, the pixels targeted for extraction are excluded from subsequent processing. Furthermore, when extracting pixels in the abnormal brightness region, both the lower and upper brightness thresholds can be set to remove pixels with brightness values ​​outside each threshold, or only one of the lower and upper brightness thresholds can be set to remove pixels with brightness values ​​outside that threshold. Furthermore, the upper and lower thresholds can be adjusted as appropriate depending on the state of the sample.

[0050] After the processing of step S120 is completed, the correction processing unit 164 then performs correction processing to sharpen the image after removing the region with abnormal luminance values ​​(step S130). Specifically, the correction processing is performed by removing the abnormal luminance region extracted in step S120. As an example of the correction processing, for example, contrast enhancement processing based on histogram expansion is performed. In this case, when calculating the lower and upper limits of saturation luminance through cumulative calculation, the abnormal luminance region is removed from the cumulative calculation to reduce the impact of foreign matter on contrast enhancement.

[0051] After the processing of step S130 is completed, the statistic calculation unit 162 next calculates a statistic (second statistic) related to the expansion of the brightness value of the image after the correction processing unit 164 has been subjected to the correction processing (step S140). In other words, the second statistic of the pixel values ​​is calculated for the image after the correction processing of step S130 has been performed. In this case, as with the first statistic, it is preferable to calculate the average value and standard deviation of the brightness of all pixels.

[0052] After the processing of step S140 is completed, the validity determination unit 166 then determines the validity of the correction processing performed by the correction processing unit 164 based on the comparison result between the ratio of the statistic of the image before correction (the first statistic) and the statistic of the image after correction (the second statistic) and a predetermined statistic ratio threshold (step S150). Specifically, the ratio of the pixel averages and the standard deviations before and after correction is calculated. A high standard deviation ratio indicates that the contrast enhancement processing is excessive. For example, the standard deviation ratio is compared with a certain threshold (a statistic ratio threshold: for example, 5). If the value is above the threshold, the contrast enhancement processing is determined to be excessive and inappropriate. The statistic ratio threshold is determined through prior experiments, etc. (described below). Furthermore, if the ratio of the averages and the standard deviations is close to 1, the correction processing has little effect, and the contrast enhancement processing is determined to be too low and inappropriate. In all other cases, the contrast enhancement processing is determined to be appropriate.

[0053] When the process of step S150 is completed, it is next confirmed whether or not the validity determination unit 166 has determined that the correction process performed by the correction processing unit 164 is valid (step S160 ).

[0054] If the determination result in step S160 is negative, that is, if the validity determination unit 166 determines that the correction processing performed by the correction processing unit 164 is inappropriate, the adjustment unit 165 adjusts the reference value used in the correction processing performed by the correction processing unit 164 (step S161), and the process returns to step S120. In other words, the process in step S160 involves performing additional correction processing (step S130) and re-determination (step S161) as needed. Specifically, if the contrast enhancement processing is inappropriate (inappropriate), for example, if the contrast enhancement processing is excessive, the reference value for the cumulative determination is set to a low value, reducing the intensity of the enhancement processing, or the contrast enhancement processing is not performed and the process moves to the next step (step S170). Alternatively, if the contrast enhancement processing is too low, the reference value for the cumulative determination can be set to a high value, increasing the intensity of the enhancement processing. Alternatively, foreign matter that could not be removed in the process of step S120 can be captured in the image, and an abnormality flag can be added to the image. In this embodiment, the processes of steps S120 to S160 and S161 are repeatedly executed until the correction process is determined to be appropriate, and the process is performed while the range of the abnormal brightness area and the strength of contrast enhancement are sequentially changed until the appropriate correction process is performed.

[0055] Furthermore, if the result of the determination in step S160 is yes, that is, if the validity determination unit 166 determines that the correction processing by the correction processing unit 164 is appropriate, the image processing output unit 167 determines the proliferation degree of the microparticles based on the feature quantity of the image determined to be corrected, and outputs the determination result to the control device 200 (step S170). Specifically, the image processing output unit 167 performs a series of image processing on the image after the correction processing, such as edge extraction processing, comparison of brightness values ​​with nearby pixels to detect maximum or minimum areas, binarization, etc., and calculates numerical information based on the image and outputs it. In the case of bacterial inspection, it is important to calculate the proliferation degree of bacteria. For the areas identified as bacteria in the image, the number, area of ​​each area, total area, and feature quantities representing shape information for each area (e.g., circularity, true roundness, circumference, convexity, aspect ratio) are calculated.

[0056] Here, right Figure 12 A method of setting the statistic ratio threshold used by the validity determination unit 166 in the process of step S150 will be described.

[0057] Figure 13 This is a diagram schematically showing a single bacterium in the image. Figure 14 It shows Figure 13 A diagram showing the relationship between the position and brightness value of pixels in .

[0058] Figure 13In the image, the interior 700 of the region of bacteria 301 is captured at a higher brightness value than the region of background 300, and the outline 701 of the region of bacteria 301 is captured at a lower brightness value. In addition, the boundary 702 between the outline of the region of bacteria 301 and the background is captured at a slightly higher brightness value than the background. In this case, the image passing through Figure 13 The distribution of the brightness values ​​in the line of the area of ​​bacteria 301 in the Figure 14 As shown. Figure 14 As shown, the distribution of brightness values ​​is bilaterally symmetrical, with a maximum peak 703 within the region of bacteria 301 as the center, a minimum peak at the bacterial outline, and a maximum peak 704 at the boundary between the bacteria and the background, with the amplitude gradually decreasing. In this embodiment, the bacterial region is determined in step S170, but, for example, a search could also be conducted for the maximum peak 703 within the bacterial region. On the other hand, if the contrast enhancement process is determined to be excessive in step S160, it is considered that the maximum peak 704 at the boundary between the bacteria and the background is also amplified, and therefore, a simple threshold determination may not be able to distinguish between the two maximum peaks. Therefore, the maximum peak 704 at the boundary between the bacteria and the background can be considered noise, and contrast enhancement is preferably performed within a range where this noise does not affect detection.

[0059] In this embodiment, the noise value is approximately 10. Since noise increases when contrast enhancement is performed, for example, if the standard deviation ratio calculated in step S140 is 5, the noise is (initial value 10) × (standard deviation ratio 5) = 50. In other words, if a threshold of 50 is used to determine the bacterial region in step S170, the noise-producing contours of the bacteria and the ring-shaped region at the boundary 702 of the background may be mistakenly identified as bacteria. As described above, it is considered that the threshold used to determine whether the contrast enhancement process is excessive is preferably (the threshold for determining the bacterial region in step S170) ÷ (the noise value). In this embodiment, the maximum peak 704 at the boundary between the bacteria region 301 and the background is the dominant noise. However, if, for example, the background noise in the background region 300 is high, the value of the background noise can be used to determine the optimal threshold.

[0060] (Effect) Figure 15 It is a diagram for explaining the effects of image processing in the image processing unit.

[0061] Figure 15 In the embodiment, the image processing unit 170 performs correction processing and binarization processing on the bacterial image that is obviously growing normally and is acquired by the imaging system of the bacteria inspection apparatus 100 according to the present embodiment, and converts the number of detected bacteria into a time series.

[0062] Figure 15 In FIG. 8 , data 800 is the result when the conventional correction process is used, and data 801 is the result when the correction process determined to be appropriate by this embodiment is used.

[0063] like Figure 15 As shown, at time 0, the output value of data 800 becomes larger, showing a value higher than 10 times. This is because the contrast enhancement process is excessive when the number of bacteria is small, causing scratches with low pores to be mistakenly identified as bacteria.

[0064] After time 7, the output value of conventional data 800 is smaller and gradually decreases. In contrast, data 801 according to the present embodiment reliably shows that the bacteria increase and reach saturation as the bacterial image changes, indicating normal bacterial growth as expected. This is because in conventional data 800, foreign matter floating in the well is captured in the image after time 7, causing the contrast enhancement processing to be too weak. In contrast, in data 801 according to the present embodiment, the contrast enhancement processing is performed at an appropriate intensity.

[0065] As described above, in this embodiment, by appropriately performing contrast enhancement processing, accurate image processing can be performed even when the concentration of an object such as bacteria varies greatly and foreign matter is included.

[0066] (Result output) Figure 16 FIG. 1 is a diagram showing an example of an image processing result output to the input / output unit of the control device.

[0067] Figure 16 In the figure, the result of the bacteria detection processing obtained by the image processing unit 160 is shown, in other words, the result of the bacteria inspection processing performed by the bacteria inspection device 100 (control device 200) is shown, for example, by displaying it in the form of a GUI (Graphical User Interface) in the input and output unit 210 of the control device 200 to show it to the user.

[0068] like Figure 16 As shown, the result of the bacteria inspection process is composed of, for example, a measurement condition display unit 211 , a measurement result display unit 212 , and an apparatus status display unit 213 , which switch display contents in a tab format.

[0069] The measurement result display unit 212 displays, as the result of the bacteria inspection process, for example, the ID of the specimen to be inspected, the current measurement status of the specimen to be inspected, the measurement result, and an output result 212a of the image processing.

[0070] In the measurement status column of the specimen, it is preferable to display the measurement time up to now, the time until the measurement is completed, a plate unloading instruction to the user, and the like.

[0071] The measurement results section preferably displays errors and warnings during measurement, error information determined through image processing, and information such as the presence of foreign matter. Furthermore, a graph showing the temporal changes in numerical information calculated through image processing is preferably provided, useful for visually displaying the degree of bacterial proliferation to the user. Specifically, each point within the quadrilateral on this graph represents the point at which measurement was performed. The presence of foreign matter can also be clearly indicated by a different color or shape. Preferably, selecting this graph allows the user to display detailed information about the error or an image of the error.

[0072] The measurement condition display unit 211 displays conditions for performing bacterial testing and allows input and output. For example, in addition to displaying measurement conditions related to the test target bacteria species or drugs, test time, image capture cycle, etc., it can also accept input from the user.

[0073] The device status display unit 213 displays information indicating the device status and allows for input and output. For example, it displays device error and warning information, temperature and humidity information, device door opening and closing information, information related to device maintenance, and various device settings, and can also receive input from the user.

[0074] As described above, in this embodiment, the apparatus includes: an imaging unit that captures an image of the interior of a container holding a liquid containing microparticles; and an image processing unit that performs image processing on the image captured by the imaging unit. The image processing unit calculates a first statistic related to the expansion of the brightness value of the image, removes regions of the image having abnormal brightness values ​​based on a comparison result between the brightness value of the image and a predetermined brightness threshold value, performs correction processing to sharpen the image after the regions of the abnormal brightness values ​​have been removed, calculates a second statistic related to the expansion of the brightness value of the image after the correction processing, determines the appropriateness of the correction processing based on a comparison result between the ratio of the first statistic and the second statistic and a predetermined statistic ratio threshold value, determines the proliferation degree of the microparticles based on a feature value of the image determined to have been corrected, and outputs a determination result of the proliferation degree. Therefore, in an image of a particulate measurement object such as growing bacteria, bacteria can be more accurately extracted and regions containing bacteria corresponding to the proliferation degree can be detected, regardless of differences in the proliferation degree or the presence or absence of foreign matter other than bacteria in the image.

[0075] <Implementation Method 2> Reference Figure 17 and Figure 18Embodiment 2 of the present invention will be described. In the drawings, the same components as those in Embodiment 1 are denoted by the same reference numerals, and their description will be omitted as appropriate.

[0076] In this embodiment, the threshold value is changed according to whether or not a foreign object is captured in the image, thereby being able to cope with a case where more foreign objects are captured in the image.

[0077] In the first embodiment, when performing contrast enhancement processing, the number of pixels is accumulated in descending order of brightness value, and the portion where the accumulated value exceeds the reference is set as the lower limit value B1_min of the saturated brightness. The number of pixels is accumulated in ascending order of brightness value, and the portion where the accumulated value exceeds the reference is set as the upper limit value B1_Max of the saturated brightness (see Figure 8 This method is convenient because it only needs to determine a certain reference value. It can remove foreign matter by combining it with the calculation of abnormal brightness areas. However, if more foreign matter is captured in the image, it may be impossible to perform appropriate correction processing.

[0078] Figure 17 This is a diagram showing an example of a histogram of brightness when many foreign objects are captured in an image.

[0079] like Figure 17 As shown, in histogram F, the foreign object's peak F_P2 and the second foreign object's peak F_P3 exist to the left of the background's peak F_P1. This indicates that there are regions in the image corresponding to at least two foreign objects of different brightness. Furthermore, the foreign object's peak F_P2 and the second foreign object's peak F_P3 are clearly indistinguishable from each other on histogram F. In this case, if a brightness value intermediate between the foreign object's peak F_P2 and the second foreign object's peak F_P3 is selected as the abnormal brightness threshold TH1, the influence of the second foreign object's pixels cannot be removed. Therefore, in this embodiment, a method is described that can further accurately remove the foreign object region and perform contrast enhancement processing.

[0080] Figure 18 This is a flowchart showing a part of the image processing according to this embodiment.

[0081] Figure 18 In the embodiment 1, Figure 12 The following processing is performed between the processing of step S120 and the processing of step S130 in the flowchart shown in FIG. The following processing can be performed for both the lower limit and the upper limit of the saturation luminance, but the case where it is performed at the lower limit of the saturation luminance is exemplified and described.

[0082] When the process of step S120 (refer to Figure 12) is completed, the image is then determined to contain foreign matter based on the calculation results of the abnormal brightness area (step S121). For example, assuming the initial value of the abnormal brightness threshold is 40, the reference value is 1%, and the image size is 100×100 pixels, if there are 100 or more pixels with brightness less than 40, it is determined that a foreign matter is present; otherwise, it is determined that no foreign matter is present.

[0083] If the result of determination in step S121 is negative, that is, if it is determined that there is no foreign matter, the process proceeds to step S140 (see Figure 12 ) processing.

[0084] If the result of step S121 is yes, i.e., if a foreign object is determined to be present, a search for an appropriate abnormal brightness threshold is performed (step S122). Specifically, the mode of the brightness values ​​on the histogram is first calculated, and then a search is performed in descending order to find the minimum value on the histogram. In this case, the distribution of the histogram is not necessarily smooth, so it is preferable to perform smoothing or low-pass filtering, and then calculate the first and second derivatives of the histogram to search for the minimum brightness value. Alternatively, signal processing such as curve fitting can be performed.

[0085] When the process of step S122 is completed, it is next determined whether a minimum value equal to or greater than the abnormal brightness threshold value is found (step S123 ).

[0086] If the result of the determination in step S123 is yes, that is, if the minimum value is found and the minimum value is greater than the initial value of the abnormal brightness threshold, the found minimum value is replaced with the abnormal brightness threshold (step S124), and the process is transferred to step S130 (see Figure 12 ).

[0087] If the result of the determination in step S122 is negative, that is, if no minimum value is found, or if the minimum value is found but is smaller than the initial value of the abnormal brightness threshold or is another improper value, the abnormal brightness threshold is kept at the initial value (step S125), and the process is transferred to step S130 (see Figure 12 ).

[0088] The other structures are the same as those in the first embodiment.

[0089] In the present embodiment configured as described above, the same effects as those of the first embodiment can be obtained.

[0090] Furthermore, it is possible to set an appropriate abnormal brightness threshold for each image and perform correction processing to remove abnormal areas, so that, for example, even if Figure 17Even when there are a plurality of foreign matter peaks as in the histogram shown, an appropriate abnormal brightness threshold TH2 can be selected.

[0091] <Note> In addition, the function of each embodiment can also be realized by the program code of software. In this case, the storage medium having recorded the program code is provided to the system or device, and the computer (or CPU or MPU) of the system or device reads the program code stored in the storage medium. In this case, the program code itself read out from the storage medium realizes the function of the above-mentioned embodiment, and the program code itself and the storage medium storing the program code constitute the present disclosure. As the storage medium for providing such program code, for example, a floppy disk, CD-ROM, DVD-ROM, hard disk, optical disk, magneto-optical disk, CD-R, magnetic tape, non-volatile memory card, ROM etc. are used.

[0092] Furthermore, based on the instructions of the program code, the OS (operating system) running on the computer performs part or all of the actual processing, and the functions of the above-mentioned embodiments can be realized through this processing. In addition, after the program code read from the storage medium is written to the memory on the computer, the CPU of the computer can perform part or all of the actual processing based on the instructions of the program code, and the functions of the above-mentioned embodiments can be realized through this processing.

[0093] Moreover, the program code of the software that implements the functions of the implementation method can be distributed via a network and stored in a storage unit such as a hard disk or memory of the system or device or a storage medium such as CD-RW, CD-R, etc. When in use, the computer (or CPU or MPU) of the system or device reads out the program code stored in the storage unit or the storage medium and executes it.

[0094] In addition, it should be understood that the processes and techniques described herein are not inherently associated with any particular device and can be implemented by any suitable combination of components. In addition, according to the main points of the present invention, various general-purpose devices can be used. It may be found that it is beneficial to build a dedicated device to perform the steps of the method described herein. In addition, various inventions can be formed by appropriately combining the multiple structural elements disclosed in the embodiments. For example, several structural elements can be deleted from all the structural elements shown in the embodiments. In addition, the structural elements involved in different embodiments can be appropriately combined. Although the present disclosure is described in conjunction with specific examples, they are intended to illustrate and not to limit in all aspects. Those skilled in the art will appreciate that there are many combinations of hardware, software and firmware suitable for implementing the present disclosure. For example, the software described can be implemented in a wide range of programs or scripting languages ​​such as assembler, C / C++, perl, Shell, PHP, Java (registered trademark), etc.

[0095] In addition, in the above embodiment, the control lines and information lines necessary for explanation are shown, but this is not limited to showing all the control lines and information lines necessary for the product. All the structures can be connected to each other.

[0096] In addition, the present disclosure is not limited to the above-mentioned embodiments and examples, and includes various modified examples. The above-mentioned embodiments are detailed descriptions for the purpose of explaining the technology of the present disclosure in an understandable manner, and are not limited to including all the structures described. In addition, a part of the structure of a certain embodiment may be replaced with the structure of another embodiment, and the structure of another embodiment may be added to the structure of a certain embodiment. In addition, other structures may be added to, deleted from, or replaced with a part of the structure of each embodiment. Label Description

[0097] 100 Bacteria inspection device 110 Lighting Department 120 Inspection Board 130 Platform 140 objective lens 150 Filming Department 160 Image Processing Department 161 Image Acquisition Unit 162 Statistical Operation Department 163 Abnormal Brightness Area Calculation Unit 164 Correction Processing Unit 165 Adjustment Department 166 Appropriateness Assessment Department 167 Image processing output unit 180 sample solution 190 Humidification and temperature control unit 200 Control Device 210 Input and Output 211 Measurement condition display section 212 Measurement result display unit 212a Output results 213 Device status display unit 300 Background 301 bacteria 302 scratches.

Claims

1. An inspection device, characterized in that: include: an imaging unit that captures an image of the interior of a container holding a liquid containing fine particles; as well as an image processing unit that performs image processing on the image captured by the imaging unit, The image processing unit calculates a first statistic related to the expansion of the brightness value of the image, removing regions of the image with abnormal brightness values ​​based on a comparison result between the brightness value of the image and a predetermined brightness threshold, performing correction processing to sharpen the image after removing the region with abnormal luminance value, calculating a second statistic related to the expansion of the brightness value of the image after the correction process, The validity of the correction process is determined based on a comparison result between the ratio of the first statistic and the second statistic and a predetermined statistic ratio threshold value. determining the proliferation degree of the microparticles based on the feature amount of the image determined to be appropriate for the correction process; The determination result of the proliferation degree is outputted.

2. The inspection device according to claim 1, wherein The image processing unit calculates a standard deviation of the brightness value of the image as the first statistic and the second statistic.

3. The inspection device according to claim 1, wherein: The image processing unit calculates a half-value width of a histogram of brightness values ​​of the image as the first statistic and the second statistic.

4. The inspection device according to claim 1, wherein: The image processing unit calculates an average value of the brightness value of the image as the first statistic and the second statistic.

5. The inspection device according to claim 1, wherein: The correction processing is processing for enhancing the contrast of light and dark in the image by stretching the histogram of the luminance value of the image in the luminance value direction.

6. The inspection device according to claim 1, wherein: The correction processing is as follows: starting from the peak of the histogram of the brightness value of the image, an extreme value search is performed in descending or ascending order, at least one brightness value with a minimum brightness value on the histogram is determined, and the brightness value is used as a lower limit value or an upper limit value to stretch the histogram of the brightness value of the image in the brightness value direction, thereby enhancing the contrast of the light and dark of the image.

7. The inspection device according to claim 1, wherein: The image processing unit cancels the correction process of the image determined to be inappropriate, or performs additional correction process.

8. The inspection device according to claim 7, wherein: The correction process is as follows: performing an extreme value search in descending or ascending order from the peak of the histogram of the brightness values ​​of the image, determining at least one brightness value with a minimum brightness value on the histogram, and using the brightness value as a lower limit value or an upper limit value to stretch the histogram of the brightness values ​​of the image in the brightness value direction, thereby enhancing the contrast of light and dark in the image. As the additional correction processing, a processing is performed in which at least one of the lower limit value and the upper limit value when stretching the histogram is changed so as to enhance the contrast of light and dark in the image.

9. The inspection device according to claim 1, wherein: The image processing unit performs binarization processing on the image determined to be correct. Detecting the region of the microparticles in the image by the binarization process, Calculating characteristic quantities related to the number and shape of the microparticles, The proliferation degree of the microparticles is determined based on the characteristic amount.

10. An inspection method, characterized in that: Including the following processes: a step of imaging the interior of a container holding a liquid containing fine particles and calculating a first statistic related to expansion of brightness values ​​of the image thus captured; a step of removing an area of ​​the image having an abnormal brightness value based on a comparison result between the brightness value of the image and a predetermined brightness threshold; a step of performing correction processing to sharpen the image after removing the region with abnormal luminance value; a step of calculating a second statistic related to expansion of the brightness value of the image after the correction processing; a step of determining the validity of the correction process based on a comparison result between a ratio of the first statistic and the second statistic and a predetermined statistic ratio threshold; a step of determining a proliferation degree of the microparticles based on a feature value of the image determined to be appropriate for the correction process; and A step of outputting the determination result of the proliferation degree.

Images