Method for detecting formed elements, apparatus for detecting formed elements, and program

Abstract

To provide a method for detecting a tangible component, a tangible component detector, and a program which can accurately extract mulberry bodies more accurately.SOLUTION: A method for detecting a tangible component includes: in Step S1, preparing a sample which includes a urine sample taken from a subject, a first fluorescent dye for staining nucleic acid, an antibody specifically combined with erythrocytes, and a second fluorescent dye combined with an antibody specifically combined with erythrocyte; in Step S3, imaging a plurality of tangible components in the sample to obtain a fluorescent image; in Step S5, extracting a tangible component in which information obtained from the fluorescent image is included in the range corresponding to the mulberry bodies.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a formed element detection method, a formed element detection device, and a program.

Background Art

[0002] Fabry disease is a type of genetic disorder, and develops when a lipid called globotriaosylceramide (Gb3) accumulates in lysosomes, which are sites of intracellular digestion. When Fabry disease develops, lysosomal function is impaired, leading to damage in various organs. Conventionally, although the causal relationship between the onset of Fabry disease and urinary mulberry bodies is known, since the amount of urinary mulberry bodies contained in urine sediment is extremely small, it is difficult to detect mulberry bodies through urine sediment testing.

[0003] Furthermore, urinary mulberry bodies have been visually confirmed using a microscope, but their morphology is similar to that of red blood cells and fungi, so discrimination by this method is highly difficult. For this reason, this discrimination requires advanced skills from the inspector.

[0004] Non-Patent Document 1 below describes that by staining urine sediment, it was clarified that urinary mulberry bodies originate from Gb3-accumulated lysosomes located in the cytoplasm of podocytes.

[0005] In addition, Patent Document 1 below describes a method of applying a urine sediment image to a neural network to determine whether a urine specimen of a subject contains mulberry bodies.

Prior Art Documents

Patent Documents

[0006]

Non-Patent Document 1

[0007] Gb3 is expressed not only in mulberry bodies but also in renal tubular cells. Therefore, although Non-Patent Literature 1 has revealed that urinary mulberry bodies originate from lysomes in which Gb3 has accumulated, it is difficult to accurately differentiate between mulberry bodies and renal tubular cells using Gb3.

[0008] Furthermore, the number of patients with Fabry disease is extremely small, and the number of mulberry bodies in the urine samples of these patients is also extremely small. For this reason, the method described in Patent Document 1 cannot use a sufficient amount of training data to train the neural network, making it difficult to put into practical use.

[0009] In view of these challenges, the present invention aims to provide a method for detecting formed elements, a device for detecting formed elements, and a program that can more accurately extract mulberry bodies. [Means for solving the problem]

[0010] In the method for detecting formed elements of the present invention, Figure 3, 4、 6. 9 As shown in (S1), a sample is prepared containing a urine sample taken from the subject, a first fluorescent dye for staining nucleic acids, an antibody that specifically binds to red blood cells, and a second fluorescent dye that binds to the antibody that specifically binds to red blood cells, and multiple formed elements in the sample are imaged. , the first fluorescence image produced by the first fluorescent dye, the second fluorescence image produced by the second fluorescent dye, and the bright-field image Obtain (S3), Based on the first fluorescence image, the second fluorescence image, and the bright-field image. Corresponding to Mulberry bodies There is Extract the formed components (S4、 S5 、S6 ). The process for extracting formed elements includes: a first extraction step of extracting formed elements in which at least one of a value relating to the size of the formed elements and a value relating to the focus state of the formed elements, based on a bright-field image, meets a first condition; a second extraction step of obtaining a value relating to the first fluorescence amount generated by a first fluorescent dye from the formed elements from a first fluorescence image, obtaining a value relating to the second fluorescence amount generated by a second fluorescent dye from the formed elements from a second fluorescence image, and extracting formed elements in which the value relating to the first fluorescence amount and the value relating to the second fluorescence amount meet a second condition; and a third extraction step of extracting formed elements in which at least one of a value relating to the contrast of the formed elements and a value relating to the luminance variation of the formed elements, based on a bright-field image, meets a third condition.

[0011] According to the formed element detection method of the present invention, formed elements whose information obtained from the fluorescence image falls within the range corresponding to mulberry bodies are extracted, thus enabling more accurate extraction of mulberry bodies.

[0012] The formed element detection device (1) of the present invention is shown in Figure 1. 4、 6 、9 As shown, a sample preparation unit (18) prepares a sample containing a urine sample collected from a subject, a first fluorescent dye for staining nucleic acids, an antibody that specifically binds to red blood cells, and a second fluorescent dye that binds to the antibody that specifically binds to red blood cells, and images multiple formed elements in the sample. , the first fluorescence image produced by the first fluorescent dye, the second fluorescence image produced by the second fluorescent dye, and the bright-field image An imaging unit (19) that acquires the image, Based on the first fluorescence image, the second fluorescence image, and the bright-field image. Corresponding to Mulberry bodies There is The system includes a processing unit (11) for extracting the formed components. The processing unit (11) performs the following steps: extract formed elements in which at least one of a value relating to the size of the formed elements and a value relating to the focus state of the formed elements, based on a bright-field image, meets the first condition; obtain a value relating to the first fluorescence amount generated by a first fluorescent dye from the formed elements from a first fluorescence image, obtain a value relating to the second fluorescence amount generated by a second fluorescent dye from the formed elements from a second fluorescence image, and extract formed elements in which the value relating to the first fluorescence amount and the value relating to the second fluorescence amount meet the second condition; and extract formed elements in which at least one of a value relating to the contrast of the formed elements and a value relating to the luminance variation of the formed elements, based on a bright-field image, meets the third condition.

[0013] According to the formed element detection device of the present invention, formed elements whose information obtained from the fluorescence image falls within the range corresponding to mulberry bodies are extracted, thus enabling more accurate extraction of mulberry bodies.

[0014] The program (12a) of the present invention is shown in Figure 1. 4、 6 、9 As shown, a program that causes a computer to perform a process to detect formed elements in a urine sample collected from a subject, wherein multiple formed elements in the sample, which includes a urine sample, a first fluorescent dye that stains nucleic acids, an antibody that specifically binds to red blood cells, and a second fluorescent dye that binds to the antibody that specifically binds to red blood cells, are imaged. Based on the first fluorescence image of fluorescence produced by the first fluorescent dye, the second fluorescence image of fluorescence produced by the second fluorescent dye, and the bright-field image obtained, , corresponding to Mulberry bodies There is Extract the formed components. The process for extracting formed elements includes: extracting formed elements in which at least one of a value relating to the size of the formed elements and a value relating to the focus state of the formed elements, based on a bright-field image, meets the first condition; obtaining a value relating to the first fluorescence amount produced by a first fluorescent dye from the formed elements from the first fluorescence image, obtaining a value relating to the second fluorescence amount produced by a second fluorescent dye from the formed elements from the second fluorescence image, and extracting formed elements in which the values ​​relating to the first fluorescence amount and the values ​​relating to the second fluorescence amount meet the second condition; and extracting formed elements in which at least one of a value relating to the contrast of the formed elements and a value relating to the luminance variation of the formed elements, based on a bright-field image, meets the third condition.

[0015] According to the program of the present invention, formed elements that fall within the range corresponding to mulberry bodies are extracted from the information obtained from the fluorescence image, thus enabling more accurate extraction of mulberry bodies. [Effects of the Invention]

[0016] According to the present invention, it is possible to extract Malpighian corpuscles more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] [Figure 1] Figure 1 is a block diagram showing the functions of a formed element detection device according to an embodiment. [Figure 2] Figure 2 is a diagram schematically showing the configuration of an imaging unit according to the embodiment. [Figure 3] Figure 3 is a flowchart showing processes related to a formed element detection method performed by a processing unit according to the embodiment. [Figure 4] Figure 4 is a diagram showing a scattergram generated in a first extraction step according to the embodiment. [Figure 5] Figure 5 is a diagram exemplifying bright-field images of formed elements plotted in a range other than the extraction range in the scattergram of the first extraction step according to the embodiment. [Figure 6] Figure 6 is a diagram showing a scattergram generated in a second extraction step according to the embodiment. [Figure 7] Figure 7 is a diagram exemplifying bright-field images, first fluorescence images and second fluorescence images of red blood cells, urothelial cells and renal tubular epithelial cells plotted in a range other than the extraction range in the scattergram of the second extraction step according to the embodiment. [Figure 8] Figure 8 is a diagram exemplifying bright-field images, first fluorescence images and second fluorescence images of white blood cells and fungi plotted near positions other than the extraction range in the scattergram of the second extraction step according to the embodiment. [Figure 9] Figure 9 is a diagram showing a scattergram generated in a third extraction step according to the embodiment. [Figure 10] Figure 10 is a diagram exemplifying bright-field images of formed elements plotted near positions other than the extraction range in the scattergram of the third extraction step according to the embodiment. [Figure 11]Figure 11 illustrates the bright-field image, first fluorescence image, second fluorescence image, and third fluorescence image of formed elements plotted within the extraction range in the scattergram of the third extraction step according to the embodiment. [Figure 12] Figure 12 is a table showing the number of formed components included in each range in the first to third extraction steps according to the embodiment. [Figure 13] Figure 13 shows the scattergram obtained in the first to third extraction steps for the urine sample shown in Figure 12, according to the embodiment. [Figure 14] Figure 14 shows the scattergram obtained in the first to third extraction steps for the urine sample shown in Figure 12, according to the embodiment. [Figure 15] Figure 15 shows the scattergram obtained in the first to third extraction steps for the urine sample shown in Figure 12, according to the embodiment. [Figure 16] Figure 16 shows the scattergram obtained in the first to third extraction steps for the urine sample shown in Figure 12, according to the embodiment. [Figure 17] Figure 17 shows the scattergram obtained in the first to third extraction steps for the urine sample shown in Figure 12, according to the embodiment. [Figure 18] Figure 18 shows the scattergram obtained in the first to third extraction steps for the urine sample shown in Figure 12, according to the embodiment. [Figure 19] Figure 19 is a schematic diagram showing the configuration of a screen for displaying a formed element image according to an embodiment. [Figure 20] Figure 20 is a schematic diagram showing the configuration of a screen for displaying a formed-element image according to an embodiment. [Figure 21] Figure 21 is a schematic diagram showing the configuration of a screen displaying an enlarged image area according to an embodiment. [Figure 22] Figure 22 illustrates a bright-field image of formed elements identified by the operator as mulberry bodies, according to an embodiment. [Figure 23] Figure 23 is a schematic diagram showing the configuration of an enlarged image region according to a modified embodiment. [Figure 24] Figure 24 is a schematic diagram showing the configuration of a screen that displays the history and changes in mulberry body counting results for the same subject, according to an embodiment. [Figure 25] Figure 25 is a schematic diagram showing the configuration of a screen for displaying a formed element image according to a modified embodiment. [Figure 26] Figure 26 is a flowchart showing the processing by the processing unit regarding the formed element detection method when only the second extraction step is performed as the extraction step, according to the first modified example of the formed element detection device, method, and program of the embodiment. [Figure 27] Figure 27 shows an example of a second modification of the formed element detection device, method, and program of the embodiment, in which the first extraction step is performed using only values ​​related to the size of formed elements based on a bright-field image. [Figure 28] Figure 28 shows an example in which the first extraction step is performed using only values ​​related to the focus state of formed elements based on a bright-field image, according to a second modification example of the formed element detection device, method, and program of the embodiment. [Figure 29] Figure 29 shows an example of a third modification of the formed element detection apparatus, method, and program of the embodiment, in which the third extraction step is performed using only the contrast values ​​of formed elements based on a bright-field image. [Figure 30] Figure 30 shows an example of a third modification of the formed element detection device, method, and program of the embodiment, in which the third extraction step is performed using only values ​​related to the variation in brightness of the formed elements. [Modes for carrying out the invention]

[0018] Figure 1 is a block diagram showing the functions of the formed element detection device 1.

[0019] The formed element detection device 1 is a device for detecting formed elements contained in a urine sample collected from a subject. The formed element detection device 1 comprises a processing unit 11, a storage unit 12, a communication unit 13, a display unit 14, an input unit 15, a reading unit 16, an aspiration unit 17, a sample preparation unit 18, and an imaging unit 19.

[0020] The processing unit 11 is composed of, for example, an FPGA or a CPU. The processing unit 11 receives signals output by each part of the formed element detection device 1 and controls each part of the formed element detection device 1. The storage unit 12 is composed of, for example, an HDD or an SSD. The storage unit 12 stores a program 12a that causes the processing unit 11 to execute the processing described later with reference to Figure 3. The processing unit 11 reads the program 12a stored in the storage unit 12 and executes the processing shown in Figure 3 according to the program 12a. The program 12a may be stored in the storage unit 12 via a storage medium, or it may be stored in the storage unit 12 by being received from another computer via the communication unit 13.

[0021] The communication unit 13 is, for example, a communication interface based on the Ethernet standard. The processing unit 11 receives subject information corresponding to the sample ID from the host computer via the communication unit 13 and transmits the counting results of formed components to the host computer.

[0022] The display unit 14 is configured, for example, as a liquid crystal display. The input unit 15 is configured, for example, as a keyboard, mouse, or pointing device including a touch panel. The operator operates buttons and other controls on the screen displayed on the display unit 14 by operating the input unit 15. The display unit 14 and the input unit 15 may be configured as an integrated unit. For example, the formed substance detection device 1 may have a touch panel display as both the display unit 14 and the input unit 15.

[0023] The reading unit 16 is comprised of, for example, a barcode reader. The reading unit 16 reads the barcode from the barcode label attached to the container containing the urine sample and obtains the sample ID. The container from which the sample ID has been read is then placed in the formed element detection device 1.

[0024] The suction unit 17 aspirates a urine sample from a container set in the formed element detection device 1 via a nozzle. The sample preparation unit 18 prepares a sample containing the urine sample, a first fluorescent dye for staining nucleic acids, an antibody that specifically binds to red blood cells, a second fluorescent dye that binds to the antibody that specifically binds to red blood cells, and a globotriaosylceramide (Gb3) antibody labeled with a third fluorescent dye. As a result, the nucleic acids of the formed elements in the urine sample are stained by the first fluorescent dye, the red blood cells in the urine sample are stained by the second fluorescent dye via the antibody, and the Gb3 in the urine sample is stained by the third fluorescent dye.

[0025] The first fluorescent dye is configured to produce fluorescence at wavelength λ21 when irradiated with light of wavelength λ11, the second fluorescent dye is configured to produce fluorescence at wavelength λ22 when irradiated with light of wavelength λ12, and the third fluorescent dye is configured to produce fluorescence at wavelength λ23 when irradiated with light of wavelength λ13. In this embodiment, wavelengths λ11, λ12, and λ13 are in different wavelength bands, but they may be in the same wavelength band.

[0026] In this example, the sample preparation unit 18 was carried out so that the sample contained Gb3 antibody labeled with the third fluorescent dye. However, since the fluorescence produced by the third fluorescent dye is not used in the mulberry body extraction step described later, it is not necessary to mix in Gb3 antibody labeled with the third fluorescent dye during sample preparation.

[0027] Figure 2 is a schematic diagram showing the configuration of the imaging unit 19.

[0028] The imaging unit 19 includes a flow cell 110, light sources 121-124, focusing lenses 131-134, dichroic mirrors 141 and 142, focusing lens 151, optical unit 152, focusing lens 153, and camera 154. The sample prepared in the sample preparation unit 18 flows through the flow channel 111 of the flow cell 110.

[0029] Light sources 121-124 irradiate the sample flowing through the flow cell 110 with light. Light sources 121-124 are semiconductor laser light sources. The light emitted from light sources 121-124 is laser light with wavelengths λ11-λ14, respectively. Focusing lenses 131-134 focus the light emitted from light sources 121-124, respectively. Dichroic mirror 141 transmits light with wavelength λ11 and reflects light with wavelength λ12. Dichroic mirror 142 transmits light with wavelengths λ11 and λ12 and reflects light with wavelength λ13. In this way, light with wavelengths λ11-λ14 is irradiated onto the sample flowing through the flow channel 111 of the flow cell 110.

[0030] When light with wavelengths λ11 to λ13 is irradiated onto a sample flowing through flow cell 110, fluorescence is emitted from the fluorescent dyes staining the formed elements. Specifically, when light with wavelength λ11 is irradiated onto the first fluorescent dye that stains nucleic acids, fluorescence at wavelength λ21 is emitted from the first fluorescent dye. When light with wavelength λ12 is irradiated onto the second fluorescent dye bound to red blood cells, fluorescence at wavelength λ22 is emitted from the second fluorescent dye. When light with wavelength λ13 is irradiated onto the third fluorescent dye that labels the Gb3 antibody, fluorescence at wavelength λ23 is emitted from the third fluorescent dye. Wavelengths λ21, λ22, and λ23 are, for example, the wavelength bands of green, red, and yellow light, respectively. When light with wavelength λ14 is irradiated onto a sample flowing through flow cell 110, this light penetrates the formed elements in the sample. Wavelength λ14 is a wavelength band suitable for bright-field imaging and has a wider wavelength band than wavelengths λ11 to λ13.

[0031] The focusing lens 151 focuses the fluorescence with wavelengths λ21 to λ23 generated from the sample flowing through the channel 111 of the flow cell 110, and the light with wavelength λ14 that has passed through the sample flowing through the channel 111 of the flow cell 110. The optical unit 152 has a configuration in which four dichroic mirrors are combined. The four dichroic mirrors of the optical unit 152 reflect the fluorescence with wavelengths λ21 to λ23 and the light with wavelength λ14 at slightly different angles from each other, separating them on the light-receiving surface of the camera 154. The focusing lens 153 focuses the fluorescence with wavelengths λ21 to λ23 and the light with wavelength λ14.

[0032] Camera 154 is, for example, a TDI (Time Delay Integration) camera. Camera 154 captures fluorescence at wavelengths λ21 to λ23 and light at wavelength λ14, and outputs a first fluorescence image, a second fluorescence image, and a third fluorescence image corresponding to the fluorescence at wavelengths λ21 to λ23, respectively, and a bright-field image corresponding to the light at wavelength λ14, as imaging signals. The processing unit 11 in Figure 1 uses the first fluorescence image, the second fluorescence image, and the bright-field image captured by camera 154 to extract mulberry bodies contained in the urine sample.

[0033] The third fluorescence image, as will be described later, serves as an indicator of the extent to which the formed elements contain Gb3, but in this embodiment it is not used for extracting mulberry bodies. Therefore, in the imaging unit 19 of Figure 2, the light source 123, the focusing lens 133, and the dichroic mirror 142 may be omitted.

[0034] Figure 3 is a flowchart showing the processing method for detecting formed elements by the processing unit 11.

[0035] The processing unit 11 performs the processing shown in Figure 3 by executing the program 12a stored in the storage unit 12.

[0036] In step S1, the processing unit 11 controls the sample preparation unit 18 to prepare a sample comprising a urine sample, a first fluorescent dye for staining nucleic acids, an antibody that specifically binds to red blood cells, a second fluorescent dye that binds to this antibody, and a Gb3 antibody labeled with a third fluorescent dye.

[0037] In step S2, the processing unit 11 controls the sample preparation unit 18 to flow the sample prepared in step S1 into the flow channel 111 of the flow cell 110. In step S3, the processing unit 11 controls the light sources 121 to 124 so that light is irradiated onto the sample flowing through the flow channel 111, and controls the camera 154 to image multiple formed elements in the sample, that is, to image the fluorescence of wavelengths λ21 to λ23 generated from the sample and the light of wavelength λ14 transmitted through the sample. As a result, the processing unit 11 acquires first to third fluorescence images and a bright-field image for each formed element in the sample. The first fluorescence image is a color image captured excluding wavelength bands other than the green band, the second fluorescence image is a color image captured excluding wavelength bands other than the red band, the third fluorescence image is a color image captured excluding wavelength bands other than the yellow band, and the bright-field image is a grayscale image.

[0038] In step S4, the processing unit 11 performs a first extraction step for mulberry bodies based on the bright-field image of all formed elements acquired in step S3. In step S5, the processing unit 11 performs a second extraction step for mulberry bodies based on the first and second fluorescence images of the formed elements extracted in the first extraction step of step S4. In step S6, the processing unit 11 performs a third extraction step for mulberry bodies based on the bright-field image of the formed elements extracted in the second extraction step of step S5. Details of the first to third extraction steps will be explained later with reference to Figures 4 to 11.

[0039] In step S7, the processing unit 11 displays the image of the formed elements extracted in the third extraction step of step S6 on the display unit 14. The display of the image of the formed elements will be explained later with reference to Figures 19 to 25. Note that the sample prepared in step S1 does not necessarily have to contain the Gb3 antibody labeled with the third fluorescent dye. In step S3, the third fluorescent image does not necessarily have to be acquired. The processing in step S6 may be omitted.

[0040] Next, the first to third extraction steps for mulberry bodies will be described with reference to Figures 4 to 11.

[0041] In the following, the extraction process is described assuming that the processing unit 11 generates scattergrams 210, 220, and 230 during extraction, as shown in Figures 4, 6, and 9. However, the generation of scattergrams is not necessarily required. That is, the processing unit 11 may perform the process of generating scattergrams and extracting formed elements within a predetermined range by data processing following a similar procedure.

[0042] Figure 4 shows the scattergram 210 generated in the first extraction step.

[0043] The processing unit 11 obtains the brightfield size and brightfield focus for each brightfield image from the brightfield images of all formed parts acquired by the camera 154.

[0044] Brightfield size is an example of a value relating to the size of formed elements based on a brightfield image, and represents morphological information of those elements. Brightfield size is, for example, the number of pixels identified as regions of formed elements in a brightfield image. Brightfield focus is an example of a value relating to the focus state of formed elements based on a brightfield image. Brightfield focus is, for example, the value obtained by normalizing the pixel values ​​of pixels identified as regions of formed elements in a brightfield image, summing the gradients of the pixel values ​​between adjacent pixels, and dividing the result by the number of pixels (average gradient). A larger brightfield focus value indicates that the formed elements are in focus (in sharp focus) in the brightfield image.

[0045] The processing unit 11 plots the total formed elements on a scattergram 210 using the brightfield size and brightfield focus obtained from the brightfield image. The horizontal axis of the scattergram 210 is the brightfield size, and the vertical axis is the brightfield focus.

[0046] The processing unit 11 extracts formed elements in the scattergram 210 that are contained within range R1, which corresponds to mulberry bodies. Range R1 is a fixed range pre-stored in the memory unit 12. The horizontal axis range of range R1 is set to a bright-field size range that is likely to contain mulberry bodies and can exclude single bacteria, large urothelial cells, and squamous epithelial cells. The vertical axis range of range R1 is set to a bright-field focus range that can exclude formed elements that are out of focus in the bright-field image. Note that the upper limit of the vertical axis of range R1 does not need to be set. The first extraction step is completed when the formed elements within range R1 are extracted.

[0047] The upper part of Figure 5 illustrates bright-field images of formed elements plotted in ranges other than range R1 in the scattergram 210 of Figure 4.

[0048] As shown in the three bright-field images on the upper left of Figure 5, if the formed elements are not in focus, the formed elements in the bright-field image will be unclear, and therefore the bright-field image cannot be used for subsequent extraction. Formed elements that are not in focus are distributed outside the range R1 and are therefore excluded in the first extraction step. Also, the bright-field image on the upper right of Figure 5 shows squamous epithelial cells that are several sizes larger than mulberry bodies. As shown in the bright-field image on the upper right of Figure 5, if the size of the formed elements is several sizes larger than the size of a normal mulberry body, it is unlikely that these formed elements are mulberry bodies. Formed elements that are several sizes larger are distributed outside the range R1 and are therefore excluded in the first extraction step.

[0049] The lower part of Figure 5 shows a bright-field image obtained by imaging beads with a size of 5 μm, which is equivalent in size to a mulberry body, using the formed element detection device 1. When setting the range R1 in advance, for example, the bright-field focus and bright-field size range of range R1 can be set using a bright-field image obtained by imaging beads of a predetermined size in this way.

[0050] Figure 6 shows the scattergram 220 generated in the second extraction step.

[0051] The processing unit 11 obtains the average brightness for nucleic acids and the average brightness for red blood cells, respectively, from the first fluorescence image and the second fluorescence image of each formed element within the range R1 extracted in the first extraction step.

[0052] The average brightness for nucleic acids is an example of a value relating to the first fluorescence amount produced from formed elements by the first fluorescent dye. The average brightness for nucleic acids is, for example, the average brightness value of the green component of pixels identified as formed elements in the first fluorescence image. The average brightness for red blood cells is an example of a value relating to the second fluorescence amount produced from formed elements by the second fluorescent dye. The average brightness for red blood cells is, for example, the average brightness value of the red component of pixels identified as formed elements in the second fluorescence image.

[0053] In this embodiment, the value relating to the first fluorescence amount generated from formed elements by the first fluorescent dye is the average brightness for nucleic acids, but is not limited to this. For example, it may be the sum, maximum, or median value of the brightness values ​​of the green component of pixels identified as regions of formed elements in the first fluorescence image. Similarly, in this embodiment, the value relating to the second fluorescence amount generated from formed elements by the second fluorescent dye is the average brightness for red blood cells, but is not limited to this. For example, it may be the sum, maximum, or median value of the brightness values ​​of the red component of pixels identified as regions of formed elements in the second fluorescence image.

[0054] The processing unit 11 plots all formed elements within range R1 extracted in the first extraction step on a scattergram 220 using the average brightness for nucleic acids and the average brightness for red blood cells obtained from the first and second fluorescence images, respectively. The horizontal axis of the scattergram 220 represents the average brightness for nucleic acids, and the vertical axis represents the average brightness for red blood cells.

[0055] The processing unit 11 extracts formed elements from the scattergram 220 that fall within range R2, which corresponds to mulberry bodies. Range R2 is a fixed range pre-stored in the memory unit 12. In Figure 6, range R41 corresponds to red blood cells. Range R42 corresponds to fungi. Range R42 includes not only fungi but also urothelial cells, tubular epithelial cells, and leukocytes. Range R2 is a range that does not overlap with range R41, which corresponds to red blood cells, and range R42, which corresponds to fungi. That is, range R2 is set to exclude red blood cells distributed in range R41 and urothelial cells, tubular epithelial cells, leukocytes, and fungi distributed in range R42. The second extraction step is completed when the formed elements within range R2 are extracted.

[0056] The upper part of Figure 7 shows examples of bright-field images, first fluorescence images, and second fluorescence images of red blood cells plotted in range R41 in the scattergram 220 of Figure 6.

[0057] Red blood cells appear as circular shapes in bright-field images and are morphologically similar to mulberry bodies. Therefore, it is difficult to distinguish between mulberry bodies (see Figure 11) and red blood cells by visual inspection of bright-field images. However, since red blood cells do not have a nucleus, the fluorescence from the first fluorescent dye is almost invisible in the first fluorescence image. Also, since red blood cells are labeled with the second fluorescent dye, the fluorescence from the second fluorescent dye is clearly visible in the second fluorescence image. Therefore, red blood cells are distributed in area R41, which does not overlap with area R2 in Figure 6, and are thus excluded by the second extraction step.

[0058] The lower section of Figure 7 and Figure 8 illustrate bright-field images, first fluorescence images, and second fluorescence images of urothelial cells, tubular epithelial cells, leukocytes, and fungi plotted in range R42 in the scattergram 220 of Figure 6.

[0059] Since most of these cells appear circular in bright-field images, it is difficult to distinguish them from mulberry bodies (see Figure 11) by visual inspection of bright-field images. In particular, fungi are morphologically similar to mulberry bodies, making it difficult to distinguish them from fungi by visual inspection. However, because these cells have nuclei, fluorescence from the first fluorescent dye is clearly visible in the first fluorescence image. Furthermore, since these cells are hardly labeled by the second fluorescent dye, fluorescence from the second fluorescent dye is hardly visible in the second fluorescence image. Therefore, these cells are distributed around area R42, which does not overlap with area R2 in Figure 6, and are thus excluded by the second extraction step.

[0060] Since mulberry bodies lack a nucleus, the fluorescence from the first fluorescent dye is barely visible in the first fluorescence image. Furthermore, because mulberry bodies are hardly labeled by the second fluorescent dye, the fluorescence from the second fluorescent dye is barely visible in the second fluorescence image. Therefore, range R2 is set to a region where the value related to the second fluorescence intensity is smaller than that of range R41, and the value related to the first fluorescence intensity is smaller than that of range R42.

[0061] Figure 9 shows the scattergram 230 generated in the third extraction step.

[0062] The processing unit 11 obtains the variation in bright-field contrast and bright-field brightness from the bright-field images of each formed element within the range R2 extracted in the second extraction step.

[0063] Brightfield contrast is an example of a value relating to the contrast of formed elements based on a brightfield image, and represents morphological information of the formed elements. Brightfield contrast is calculated, for example, by adding the gradient of pixel values ​​within a 3x3 pixel range for pixels identified as regions of formed elements in a brightfield image, shifting the pixel range to calculate the gradient of pixel values ​​for each pixel range, and then dividing the gradient of pixel values ​​for the entire pixel range by the number of pixels in the region of the formed elements. Brightfield luminance variability is an example of a value relating to the luminance variability of formed elements based on a brightfield image, and represents morphological information of the formed elements. Brightfield luminance variability is, for example, the standard deviation of the values ​​of pixels identified as regions of formed elements in a brightfield image.

[0064] Brightfield contrast indicates the amount of change in pixel values ​​in a brightfield image. For example, the greater the difference in density between the edges and interior of formed elements in a brightfield image, the greater the brightfield contrast. Brightfield contrast changes due to differences in density between the aggregate surface and the interior, and differences in density due to the spiral shape of the mulberry bodies. Brightfield brightness variability indicates the distribution of pixel values ​​in a brightfield image. For example, the greater the variation in pixel values ​​of formed elements in a brightfield image, and the more non-uniform the pixel values, the greater the brightness variability. Since many mulberry bodies have a spiral shape, the brightness variability and contrast of formed elements corresponding to mulberry bodies are relatively large.

[0065] The processing unit 11 uses the brightfield contrast and brightness variation obtained from the brightfield image to plot all formed elements within the range R2 extracted in the second extraction step on a scattergram 230. The horizontal axis of the scattergram 230 represents the brightfield contrast, and the vertical axis represents the brightness variation.

[0066] The processing unit 11 extracts formed elements in the scattergram 230 that are contained within the range R3 corresponding to mulberry bodies. Range R3 is a fixed range pre-stored in the memory unit 12. Range R3 is set to exclude formed elements shown only by brightness and darkness distributed around position R43, disbris distributed around position R44, out-of-focus formed elements distributed around position R45, and clusters of small formed elements distributed around position R46. Specifically, range R3 is set to a region where the contrast value is greater than that of position R44 where disbris appears, and the brightness variation value is greater than that of position R46 where clusters of small formed elements appear. The third extraction step is completed when the formed elements within range R3 are extracted.

[0067] Figure 10 illustrates bright-field images of formed elements plotted near positions R43, R44, R45, and R46 in the scattergram 230 of Figure 9.

[0068] As shown in the first row of Figure 10, the bright-field image of formed elements near position R43 shows formed elements that only appear as light and dark. Since formed elements that only appear as light and dark are distributed near position R43, outside the range R3, they are excluded by the third extraction step.

[0069] As shown in the second row of Figure 10, the bright-field image of the formed element near position R44 shows dislodgement. Dislodgement refers to, for example, necrotic tissue fragments. Since the dislodgement is distributed near position R44, outside the range R3, it is removed by the third extraction step.

[0070] As shown in the third row of Figure 10, the bright-field image of formed elements near position R45 contains out-of-focus formed elements. Since these out-of-focus formed elements are distributed outside the range R3, near position R45, they are removed by the third extraction step.

[0071] As shown in the fourth row of Figure 10, the bright-field image of the formed elements near position R46 shows small clusters of formed elements. Since these small clusters of formed elements are distributed near position R46, outside the range R3, they are removed by the third extraction step.

[0072] Figure 11 illustrates the bright-field images, first fluorescence image, second fluorescence image, and third fluorescence image of formed elements plotted in range R3 in the scattergram 230 of Figure 9.

[0073] The R3 region contains mulberry bodies, tubular epithelial cells, and salts, among other things.

[0074] Bright-field images in range R3 contain mulberry bodies with internal features such as spiral shapes, double circles, and white areas. Mulberry bodies may also have aggregate (mulberry-like) shapes, as shown in the bottom bright-field image of the mulberry bodies. Since mulberry bodies are hardly stained by the second fluorescent dye, fluorescence from the second fluorescent dye is hardly visible in the second fluorescence image. Also, many mulberry bodies, except for aggregated mulberry bodies, are hardly stained by the first fluorescent dye, so fluorescence from the first fluorescent dye is hardly visible in the first fluorescence image.

[0075] In this embodiment, the third fluorescence image is not used for the extraction of mulberry bodies, but in order to confirm the presence or absence of Gb3 in the formed elements, Gb3 in the urine sample is labeled with the third fluorescence dye and the third fluorescence image is acquired.

[0076] Referring to the third fluorescence image of mulberry bodies, Gb3 may be expressed in mulberry bodies, for example, when mulberry bodies are in a clustered shape. However, referring to the third fluorescence image of tubular epithelial cells, Gb3 may also be expressed in tubular epithelial cells. Therefore, it is difficult to classify mulberry bodies and tubular epithelial cells based on the presence or absence of Gb3 expression. In contrast, in the mulberry body extraction step according to the embodiment, although some tubular epithelial cells remain in range R3, in the scattergram 220 of Figure 6, most of the tubular epithelial cells are distributed in range R42, and since range R42 does not overlap with range R2, which corresponds to mulberry bodies, many tubular epithelial cells can be excluded, especially in the second extraction step.

[0077] Furthermore, among the mulberry bodies shown in the upper panel of Figure 11, the mulberry bodies in the second and third images from the top are almost not stained by the third fluorescent dye. Thus, conventional methods that improve the visibility of mulberry bodies using antibody staining dyes that specifically stain Gb3 may not always provide sufficient visibility.

[0078] Furthermore, while tubular epithelial cells shown in the middle of Figure 11 and salts shown in the lower part of Figure 11 remain in range R3, the mulberry bodies contained in range R3 are identified visually by the operator using the screen 300 displayed on the display unit 14. The screen 300 will be explained later with reference to Figure 19 and subsequent figures. According to the first to third extraction steps, many formed elements other than mulberry bodies contained in the urine sample are excluded; that is, formed elements contained in ranges R1, R2, and R3 corresponding to mulberry bodies are extracted, so the operator can efficiently identify mulberry bodies by referring to the image displayed on the screen 300, which will be described later.

[0079] Next, referring to Figures 12 to 18, we will describe an experiment in which mulberry bodies were extracted from 12 subjects by performing the first to third extraction steps.

[0080] In this experiment, UF-Fluorocell CR (Sysmex Corporation) was used as the first fluorescent dye for staining nucleic acids, Glycophorin A antibody: SC-53295 (SANTA CRUZ BIOTECHNOLOGY, INC.) was used as the antibody that specifically binds to red blood cells, and MIX-n-STAIN CF594 Antibody Labeling Kits: 92256 (Biotium) was used as the second fluorescent dye that binds to the antibody. The wavelength λ11 that excited the first fluorescent dye was 488 nm, and the wavelength λ12 that excited the second fluorescent dye was 592 nm.

[0081] Figure 12 is a table showing the number of formed components included in each range during the first to third extraction steps.

[0082] In this experiment, urine samples were collected from 12 subjects. A portion of the collected urine samples were examined under a microscope by a skilled laboratory technician, while the remaining portion underwent the processing shown in steps S1 to S6 of Figure 3.

[0083] The medical technologist performed microscopic examination on a portion of the urine samples. A sample was classified as positive if it contained mulberry bodies, and negative if it did not. Urine samples with IDs F005, F006, F032, F035, F047, F049, F060, F083, F084, and F095 were determined to be positive based on microscopic examination. Urine samples with IDs N007 and N008 were determined to be negative based on microscopic examination.

[0084] Meanwhile, for other portions of the same urine sample, the processing steps S1 to S6 in Figure 3 was performed, and the number of formed elements, the number of mulberry bodies, and the ratio of mulberry bodies in the scattergram 210 obtained in step S4, the scattergram 220 obtained in step S5, and the scattergram 230 obtained in step S6 were counted and calculated. Whether or not something was a mulberry body was determined by a skilled medical technologist visually inspecting the image captured by the imaging unit 19 in step S3. In the table, "All" represents the number of formed elements contained in the scattergram 210 obtained in step S4. In the table, "R1" represents the number of formed elements contained in range R1 of the scattergram 210 obtained in step S4. In the table, "R2" represents the number of formed elements contained in range R2 of the scattergram 220 obtained in step S5. In the table, "R3" represents the number of formed elements contained in range R3 of the scattergram 230 obtained in step S6. The proportion of mulberry bodies is calculated by dividing the number of mulberry bodies by the number of formed components within the same range.

[0085] As shown in Figure 12, by performing the first to third extraction steps on all formed elements imaged by the imaging unit 19, the number of formed elements was significantly reduced in all urine samples. On the other hand, the number of mulberry bodies was hardly reduced in any urine sample by the first to third extraction steps. In other words, the number of mulberry bodies extracted in range R3 was only slightly less than the total number of mulberry bodies imaged by the imaging unit 19. As a result, the proportion of mulberry bodies increased with each extraction, eventually rising significantly to about 1% to 5%. Therefore, it can be seen that mulberry bodies were extracted more accurately from all formed elements imaged by the imaging unit 19.

[0086] Figures 13 to 18 show the scattergrams 210, 220, and 230 obtained in the first to third extraction steps for the urine sample shown in Figure 12.

[0087] In Figures 13 to 18, the gray plots correspond to formed elements that were determined to be mulberry bodies by a skilled technician visually inspecting the images captured by the imaging unit 19 in step S3, while the white plots correspond to formed elements other than those determined to be mulberry bodies.

[0088] Similarly, the gray plots in the scattergrams 210, 220, and 230 in Figures 4, 6, and 9 correspond to formed elements that a skilled technician determined to be mulberry bodies by visually inspecting the images captured by the imaging unit 19 in step S3. However, the visual inspection by a skilled technician to determine whether or not an element is a mulberry body, and the color-coding of the plots based on the determination result, are intended to confirm the effectiveness of this experiment and can be omitted.

[0089] Furthermore, the urine sample with sample ID F005 is identical to the urine sample illustrated in the explanations of Figures 4 to 11. Therefore, the scattergrams 210, 220, and 230 shown in the upper part of Figure 13 are identical to the scattergrams shown in Figures 4, 6, and 9, but are reproduced for convenience.

[0090] As can be seen by referring to the scattergrams 210, 220, and 230 for each urine sample in Figures 13 to 18, the plots of mulberry bodies shown in gray are almost entirely within the extraction ranges R1, R2, and R3 from the first to third extraction steps. Thus, it can be seen that by using the ranges R1, R2, and R3 stored in the memory unit 12 beforehand, mulberry bodies can be properly extracted from any urine sample.

[0091] Next, the process of displaying images of formed elements, which is performed in step S7 of Figure 3, will be explained with reference to Figures 19 to 25. In step S7 of Figure 3, images of formed elements extracted by the first to third extraction steps (formed elements included in the range R3 of the scattergram 230 in Figure 9) are displayed on screen 300 shown below, and information based on the history of mulberry body counting results and changes in the counting results is displayed on screen 400 shown below.

[0092] Figure 19 is a schematic diagram showing the configuration of the screen 300 that displays the formed element image.

[0093] The screen 300 includes a specimen ID display area 301, a subject ID display area 302, and an image display area 310 for displaying images of formed elements. The screen 300 also includes a bright-field image display button 321, a first fluorescence image display button 322, a second fluorescence image display button 323, and a third fluorescence image display button 324 for selecting the type of image to display in the image display area 310. The screen 300 also includes a number order button 331 and a size order button 332 for setting the order in which images are displayed in the image display area 310. The screen 300 also includes a count result display area 341 for displaying the number of mulberry bodies selected in the image display area 310, and a confirmation button 342 for confirming the count result.

[0094] When the operator enters the sample ID and subject ID and then gives the instruction to display screen 300, the processing unit 11 displays screen 300, as shown in Figure 19, on the display unit 14.

[0095] When the operator operates one of the bright-field image display button 321, the first fluorescence image display button 322, the second fluorescence image display button 323, or the third fluorescence image display button 324, the processing unit 11 displays the type of image corresponding to the operated button in the image display area 310. The image display area 310 displays images of formed elements contained within range R3, obtained from the urine sample indicated in the sample ID display area 301. The example shown in Figure 19 shows the state when the bright-field image display button 321, one of the four buttons for selecting the image type, is operated. As a result, a bright-field image is displayed in the image display area 310.

[0096] When the operator operates either the number order button 331 or the size order button 332, the processing unit 11 rearranges the order of the images displayed in the image display area 310. Each image is assigned a sequential number for its formed elements, and when the number order button 331 is operated, the images are rearranged in the order of these sequential numbers. When the size order button 332 is operated, the images are rearranged in the order of the size of the formed elements on the image, based on the brightfield size acquired in step S4. The example shown in Figure 19 shows the state when the number order button 331, one of the two buttons for setting the sorting order, is operated. By operating either the number order button 331 or the size order button 332 consecutively, the display order is swapped between ascending and descending order.

[0097] The example shown in Figure 20 illustrates the state when the size sort button 332, one of the two buttons for setting the sorting order, is operated. As will be described later with reference to Figure 21, a thick border 312 is added to the images of the formed elements selected as mulberry bodies in Figure 20. Since mulberry bodies are of roughly the same size, sorting the images by size ensures that the images corresponding to mulberry bodies are placed close together. This allows the operator to select mulberry bodies more accurately.

[0098] When screen 300 is displayed, the processing unit 11 may display the images in the image display area 310 in numerical order or in order of size. Furthermore, screen 300 may have at least one of the following buttons: a button for sorting images by brightfield size, a button for sorting images by brightfield focus, a button for sorting images by average brightness related to nucleic acids, a button for sorting images by average brightness related to red blood cells, a button for sorting images by brightfield contrast, and a button for sorting images by brightness variation.

[0099] The operator refers to the image in the image display area 310 and, if it determines that the formed elements in the image may be mulberry bodies, performs an operation (for example, clicking) on ​​the image. When the image in the image display area 310 is operated on, the processing unit 11 overlays and displays the enlarged image area 311 on the screen 300.

[0100] Figure 21 is a schematic diagram showing the configuration of the screen 300 in which the enlarged image area 311 is displayed.

[0101] The enlarged image area 311 displays the image manipulated in the image display area 310 in an enlarged state. The enlarged image area 311 includes a checkbox 311a. The operator further observes the enlarged image and, if they determine that the formed elements in the image are mulberry bodies, they check the checkbox 311a.

[0102] Figure 22 shows an example of a bright-field image of formed elements identified by the operator as mulberry bodies.

[0103] Mulberry bodies may have a black spiral shape as shown in the first row, a white spiral shape as shown in the second row, a double circle as shown in the third row, or a white gap as shown in the fourth row. After understanding the shape of these mulberry bodies, the operator refers to the image display area 310 and the enlarged image area 311 shown in Figures 19-21 to determine whether the formed elements in the image are mulberry bodies or not. At this time, as described above, since formed elements other than mulberry bodies are largely excluded by the first to third extraction steps, the operator can perform the task of continuously determining whether or not an element is a mulberry body by referring to images such as those shown in Figures 19-21 with greater accuracy.

[0104] Returning to Figure 21, the processing unit 11 associates a flag with the image checked in checkbox 311a and stores it in the storage unit 12 (see Figure 1), and adds a thick border 312 to the corresponding image in the image display area 310. Adding a thick border 312 to the image makes it easier to identify the image containing the formed mulberry bodies from among multiple images in the image display area 310. Furthermore, once a flag is associated with the checked image, the image identified as containing mulberry bodies can be reviewed again at a later date.

[0105] Note that input of instructions for the image is not limited to being done via checkbox 311a; it may also be done through other operations on the image (for example, right-clicking or double-clicking).

[0106] Furthermore, the processing unit 11 displays the number of formed elements that have been checked in checkbox 311a in the counting result display area 341. This allows the operator to easily grasp the number of formed elements that have been identified as mulberry bodies (the number of formed elements that are marked with a thick border 312).

[0107] Furthermore, when the operator presses the confirmation button 342, the processing unit 11 transmits the counting results, including the number of formed elements determined to be mulberry bodies via the checkbox 341a, the ratio of the number of formed elements determined to be mulberry bodies to the total number of formed elements within range R3, the sample ID, the subject ID, and the current date and time, to an external host computer (see Figure 1). The counting results may be stored in the storage unit 12 instead of, or in addition to, being transmitted to the host computer.

[0108] In Figure 21, only the enlarged image of the image manipulated within the image display area 310 is displayed in the enlarged image area 311. However, this is not limited to this; as shown in Figure 23, all corresponding images (bright-field image, first fluorescence image, second fluorescence image, and third fluorescence image) of the formed elements in the image manipulated within the image display area 310 may also be displayed. In this case, the operator can smoothly determine whether the target formed element is a mulberry body by referring to all images. In this case as well, the display of the third fluorescence image can be omitted.

[0109] Furthermore, checkboxes for the shape of mulberry bodies (spiral, double circle, and white areas, etc.) may be placed in the magnified image area 311. In this case, the operator refers to the image to determine the shape of the mulberry bodies and checks the corresponding checkbox. The screen 300 then displays the counting results for each mulberry body shape, and buttons are provided to sort the images by mulberry body shape. This makes it easier to distinguish between typical mulberry bodies (spiral and double circle) and atypical mulberry bodies (other shapes) in the image display area 310, allowing, for example, confirmation of the effectiveness of enzyme replacement therapy administered to patients with Fabry disease.

[0110] Furthermore, the scattergrams 210, 220, and 230 shown in Figures 4, 6, and 9 may be displayed on screen 300. In this case, when the operator manipulates the image in the image display area 310, a mark may be placed on the corresponding plot of formed elements on the scattergram, and when the operator manipulates the plot of formed elements on the scattergram, a mark may be placed on the corresponding image in the image display area 310.

[0111] Figure 24 is a schematic diagram showing the configuration of screen 400, which displays the history and changes in mulberry body counting results for the same subject.

[0112] The screen 400 includes a subject ID display area 401, a list display area 411, and a graph display area 412.

[0113] When the operator enters the subject ID and then instructs the system to display screen 400, the processing unit 11 displays screen 400, as shown in Figure 24, on the display unit 14. At this time, the processing unit 11 queries the host computer (see Figure 1) based on the subject ID and receives the counting results corresponding to the subject ID from the host computer. If there are multiple counting results corresponding to a single subject ID, the processing unit 11 receives the multiple counting results from the host computer as a history of counting results. Based on the received counting results, the processing unit 11 displays the list display area 411 and the graph display area 412.

[0114] Each row in the list display area 411 corresponds to one counting result. The list display area 411 displays the number of formed elements identified as mulberry bodies and the ratio of the number of formed elements identified as mulberry bodies to the total number of formed elements within range R3. The graph display area 412 is a graph based on the data in the list display area 411. The graph in the graph display area 412 contains information based on changes in the counting results.

[0115] Note that the list display area 411 and graph display area 412 shown in Figure 24 are displayed on a different screen 400 from the screen 300 shown in Figures 19-21, but they may also be displayed together with screen 300, as shown in Figure 25.

[0116] <Effects of the Formed Element Detection Method, Formed Element Detection Apparatus, and Program According to the Embodiment> In step S1 of Figure 3, the processing unit 11 drives the sample preparation unit 18 to prepare a sample containing a urine sample collected from the subject, a first fluorescent dye for staining nucleic acids, an antibody that specifically binds to red blood cells, and a second fluorescent dye that binds to the antibody that specifically binds to red blood cells. In step S3, the processing unit 11 images multiple formed elements in the sample and acquires a fluorescence image. In step S5, the processing unit 11 extracts formed elements that are included in the range R2 (see Figure 6) corresponding to mulberry bodies based on the information obtained from the fluorescence image.

[0117] This process extracts formed elements within the R2 range corresponding to mulberry bodies from the fluorescence image, allowing for more accurate extraction of mulberry bodies. In particular, it can more accurately extract mulberry bodies by excluding erythrocytes and fungi, which are morphologically similar to mulberry bodies.

[0118] The information obtained from the fluorescence image includes values ​​relating to the first fluorescence amount generated from the formed elements by the first fluorescent dye and values ​​relating to the second fluorescence amount generated from the formed elements by the second fluorescent dye.

[0119] This composition allows for the exclusion of red blood cells, tubular epithelial cells, urothelial cells, white blood cells, and fungi, among others.

[0120] The fluorescence image includes a first fluorescence image capturing fluorescence produced by a first fluorescence dye and a second fluorescence image capturing fluorescence produced by a second fluorescence dye.

[0121] With this configuration, by individually imaging the fluorescence generated by the first fluorescent dye and the fluorescence generated by the second fluorescent dye, values ​​related to the first fluorescence amount and values ​​related to the second fluorescence amount can be smoothly acquired.

[0122] The process of extracting formed elements includes a second extraction step, step S5 in Figure 3, in which formed elements whose values ​​for the first fluorescence amount and the second fluorescence amount fall within the range R2 (i.e., meet the predetermined conditions).

[0123] This configuration allows for the extraction of mulberry bodies under predetermined conditions, thus facilitating the extraction process.

[0124] In step S3 of Figure 3, the processing unit 11 further acquires a bright-field image by imaging multiple formed elements in the sample. The process of extracting formed elements includes a first extraction step in step S4, a second extraction step in step S5, and a third extraction step in step S6, which extract formed elements that are included in the ranges R1, R2, and R3 (see Figures 4, 6, and 9) corresponding to mulberry bodies based on the information obtained from the fluorescence image and the bright-field image.

[0125] With this configuration, mulberry bodies can be extracted more accurately by using bright-field imaging.

[0126] The information obtained from the brightfield image includes values ​​related to the size of formed elements based on the brightfield image and values ​​related to the focus state of formed elements based on the brightfield image.

[0127] This configuration allows for the exclusion of formed elements other than mulberry bodies from urine samples (such as single bacteria, large urothelial cells, and squamous epithelial cells) based on the size of the formed elements. Additionally, the focus state allows for the exclusion of images of formed elements unsuitable for visual inspection.

[0128] The information obtained from the brightfield image includes values ​​related to the contrast of formed elements based on the brightfield image and values ​​related to the luminance variation of formed elements based on the brightfield image.

[0129] This configuration allows for the exclusion of formed elements shown only by light and dark areas, disbris, out-of-focus formed elements, and small clusters of formed elements.

[0130] The process of extracting formed elements includes a first extraction step in step S4, a second extraction step in step S5, and a third extraction step in step S6, as shown in Figure 3. In the first extraction step, the processing unit 11 extracts formed elements whose size values ​​and focus state values ​​based on the bright-field image are included in the range R1 of Figure 4 (i.e., satisfying the first condition). In the second extraction step, the processing unit 11 obtains a value related to the first fluorescence amount generated by the first fluorescent dye from the formed elements from the first fluorescence image, and obtains a value related to the second fluorescence amount generated by the second fluorescent dye from the formed elements from the second fluorescence image, and extracts formed elements whose first fluorescence amount values ​​and second fluorescence amount values ​​are included in the range R2 of Figure 6 (i.e., satisfying the second condition). In the third extraction step, the processing unit 11 extracts formed elements whose contrast values ​​and luminance variation values ​​based on the bright-field image fall within the range R3 of Figure 9 (i.e., satisfy the third condition).

[0131] The first to third extraction steps allow for more accurate extraction of mulberry bodies.

[0132] The range R2, which corresponds to mulberry bodies based on the information obtained from the fluorescence image, does not overlap with the range R41, which corresponds to red blood cells, and the range R42, which corresponds to fungi.

[0133] This configuration allows for more accurate extraction of mulberry bodies by excluding erythrocytes and fungi that are morphologically similar to mulberry bodies.

[0134] In step S7 of Figure 3, the processing unit 11 displays images of the formed elements included in the range corresponding to the extracted mulberry bodies (for example, range R3 in Figure 9) on the screen 300 shown in Figures 19 to 21.

[0135] This process allows the operator to easily identify formed elements that are likely to be mulberry bodies by referring to images of formed elements within the range corresponding to mulberry bodies.

[0136] In step S3 of Figure 3, the processing unit 11 captures images of multiple formed elements in the sample to obtain bright-field images, and in step S7, it displays the extracted bright-field images of the formed elements in the image display area 310 shown in Figures 19 to 21.

[0137] Mulberry bodies are easily characterized in bright-field images. Therefore, this process allows the operator to easily identify formed elements that are likely to be mulberry bodies by referring to the bright-field image.

[0138] In step S7 of Figure 3, the processing unit 11 displays the extracted images of formed elements in a list in the image display area 310 shown in Figures 19 to 21.

[0139] This process displays images of formed elements in a list, making it easy for the operator to sequentially review multiple images of formed elements.

[0140] In step S7 of Figure 3, the processing unit 11 displays the extracted formed element images in order of size in the image display area 310, as shown in Figure 20.

[0141] When the formed elements are sorted by size, the mulberry bodies and other formed elements become easier to distinguish in the image display area 310. This allows the operator to more accurately identify the mulberry bodies.

[0142] In step S7 of Figure 3, the processing unit 11 receives an instruction input via the size sort button 332 in Figures 19-21 to sort the extracted formed element images in order of their size, and sorts the displayed list of formed element images according to the received instruction input.

[0143] This process allows the operator to smoothly rearrange the displayed formed components.

[0144] In step S7 of Figure 3, the processing unit 11 receives an instruction input to identify any one of the images of formed elements displayed in the list, and displays an enlarged image of the formed element identified by the instruction input, as shown in the enlarged image area 311 of Figure 21.

[0145] This process allows the operator to more easily see the formed elements down to the finest detail, enabling them to more accurately determine whether or not the formed elements in question are mulberry bodies.

[0146] In step S7 of Figure 3, the processing unit 11 receives an instruction input via the checkbox 311a in Figure 21 to identify the image of a mulberry body from the list of images of formed elements, and counts the images of the formed elements identified by the instruction input, and displays the counting result in the counting result display area 341 as shown in Figure 21.

[0147] This process allows the operator to use the number of formed elements identified as Mulberry bodies as useful information for pathological diagnosis, such as determining whether or not the subject has Fabry disease.

[0148] The processing unit 11 stores a history of counting results for the same subject, as shown in the list display area 411 of Figure 24.

[0149] By displaying this history of counting results, operators can periodically check the changes in the state of mulberry bodies in the subjects being examined.

[0150] The processing unit 11 displays information based on changes in the counting results, based on the history of the counting results, as shown in the graph display area 412 of Figure 24.

[0151] This process allows operators to smoothly observe changes in the number of mulberry bodies in the subjects being examined. Therefore, pathological diagnoses, such as determining whether or not a subject has Fabry disease and assessing the effectiveness of medication administered to the subject, can be carried out more accurately.

[0152] As shown in screen 300 of Figure 25, the processing unit 11 displays a list of images of formed elements shown in the image display area 310, along with information based on the changes in the counting results shown in the graph display area 412.

[0153] This process allows for simultaneous viewing of a list of images and changes in the counting results. For example, it can be used to confirm whether medication or other treatments administered to the subjects were effective.

[0154] In step S7 of Figure 3, the processing unit 11 receives an instruction input via the checkbox 311a of Figure 21 to identify an image of a mulberry body from the list of images of formed elements, and stores a flag associated with the image of the formed element identified by the instruction input.

[0155] This process allows the operator to display images of formed elements identified as mulberry bodies, along with their markings. This enables, for example, a physician to easily review the operator's assessment results at a later date, thereby leading to a more accurate diagnosis.

[0156] As shown in Figure 2, the imaging unit 19 is equipped with a flow cell 110 through which the sample flows, and images multiple formed elements in the sample flowing through the flow cell 110.

[0157] This configuration allows for the efficient acquisition of numerous images of formed elements.

[0158] <First example of modifications to the formed element detection device, method, and program> In the above embodiment, both the first extraction step and the third extraction step were performed, but either the first extraction step or the third extraction step may be omitted, or both the first extraction step and the third extraction step may be omitted.

[0159] Figure 26 is a flowchart showing the processing method for detecting formed elements by the processing unit 11 when the sample prepared in step S1 does not contain the Gb3 antibody labeled with the third fluorescent dye, and only the second extraction step is performed as an extraction step.

[0160] The process in Figure 26 omits the first extraction step in step S4 and the third extraction step in step S6 compared to the process in the embodiment of Figure 3.

[0161] In this case, in the second extraction step of step S5, the processing unit 11 acquires a bright-field image, a first fluorescence image, and a second fluorescence image. The processing unit 11 also plots all formed elements captured by the imaging unit 19 on the scattergram 220 in Figure 6 and extracts the formed elements within range R2 of the scattergram 220. Then, in step S7, the processing unit 11 displays the images of the formed elements extracted in the second extraction step of step S5 on the screen 300.

[0162] Thus, even when only the second extraction step is performed as an extraction process, formed elements within the range R2 defined by the value of the first fluorescence amount generated from the formed elements by the first fluorescent dye and the value of the second fluorescence amount generated from the formed elements by the second fluorescent dye are extracted. As mentioned above, the range R2 includes mulberry bodies, but does not contain red blood cells, tubular epithelial cells, urothelial cells, leukocytes, or fungi. Therefore, even in the process shown in Figure 26, formed elements other than mulberry bodies can be excluded, and mulberry bodies can be extracted more accurately.

[0163] <Second example of modification of the formed-matter detection device, method, and program> In the above embodiment, as shown in Figure 4, both a value relating to the size of the formed element based on a bright-field image and a value relating to the focus state of the formed element based on a bright-field image were used in the first extraction step. However, the invention is not limited to this; only the size value may be used, or only the focus state value may be used. Below, the first extraction step in the formed element detection method of this second modified example will be explained using an image obtained from a urine sample with sample ID "F005" shown in Figure 12 as an example.

[0164] Figure 27 shows an example in which the first extraction step is performed using only values ​​related to the size of formed elements based on a bright-field image.

[0165] The graph in Figure 27 is based on all formed elements captured by the imaging unit 19. In the graph in Figure 27, the horizontal axis represents the brightfield size, and the vertical axis represents the frequency. Range R11 is the same as the range (range in the horizontal axis direction) of range R1 in Figure 4 in terms of brightfield size.

[0166] In this case, the first extraction step extracts formed elements contained within range R11 (first condition). As a result, in the case of a urine sample with sample ID "F005", 84,328 formed elements within range R11 are extracted from the total 276,928 formed elements captured by the imaging unit 19. In other words, 30.5% of the total formed elements are extracted, and 69.5% are excluded.

[0167] Figure 28 shows an example in which the first extraction step is performed using only values ​​related to the focus state of formed elements based on a bright-field image.

[0168] In the graph in Figure 28, the horizontal axis represents bright-field focus, and the vertical axis represents frequency. The graph in Figure 28 is based on all formed elements captured by the imaging unit 19. Range R12 is the same as the range (range in the vertical axis direction) of range R1 in Figure 4 in bright-field focus.

[0169] In this case, the first extraction step extracts formed elements contained in range R12 (first condition). As a result, in the case of a urine sample with sample ID "F005", 240,649 formed elements from range R12 are extracted from the total 276,928 formed elements imaged by the imaging unit 19. In other words, 86.9% of the total formed elements are extracted, and 13.1% are excluded. Since ranges R11 and R12 include at least the formed elements within range R1, ranges R11 and R12 contain the same number or more mulberry bodies as those in range R1.

[0170] As described above, even if either the value relating to the size of formed elements based on the bright-field image or the value relating to the focus state of formed elements based on the bright-field image is used in the first extraction step, formed elements that are unlikely to be mulberry bodies can be largely excluded, and mulberry bodies can be extracted more accurately.

[0171] Furthermore, as can be seen from the example of sample ID "F005" above, the proportion of formed elements excluded by the first extraction step using only size-related values ​​(69.5%) is greater than the proportion of formed elements excluded by the first extraction step using only focus-related values ​​(13.1%). Therefore, it is preferable to use at least bright-field size (values ​​related to the size of formed elements) in the first extraction step.

[0172] Furthermore, the first, second, and third extraction steps are not limited to being performed in the order shown in the above embodiment, but if at least a size value is used in the first extraction step, it is preferable that the first extraction step be performed prior to the second and third extraction steps. In this case, since the number of formed components can be significantly reduced in the first extraction step, subsequent processing can be carried out, thereby reducing the amount of computation required by the processing unit 11 in the later steps.

[0173] <Third example of modification of the formed-matter detection device, method, and program> In the above embodiment, as shown in Figure 9, both the contrast value of the formed elements and the luminance variation value of the formed elements based on the bright-field image were used in the third extraction step. However, the method is not limited to this; only the contrast value may be used, or only the luminance variation value may be used. Below, the third extraction step in the formed element detection method of this third modified example will be explained using an image obtained from a urine sample with sample ID "F005" shown in Figure 12 as an example.

[0174] Figure 29 shows an example in which the third extraction step is performed using only the contrast values ​​of formed elements based on a bright-field image.

[0175] The graph in Figure 29 is based on the formed elements extracted by the second extraction process (formed elements extracted by range R2 in Figure 6). In the graph in Figure 29, the horizontal axis represents bright-field contrast, and the vertical axis represents frequency. Range R31 is the same as the range (range along the horizontal axis) in the bright-field contrast of range R3 in Figure 9.

[0176] In this third extraction step, formed elements contained within range R31 are extracted. As a result, in the case of a urine sample with sample ID "F005", 14,798 formed elements from range R31 are extracted from the 42,430 formed elements extracted from range R2. In other words, 34.9% of the formed elements from range R2 are extracted, and 65.1% are excluded.

[0177] Figure 30 shows an example in which the third extraction step is performed using only values ​​related to the luminance variation of formed elements.

[0178] The graph in Figure 30 is based on the formed elements extracted by the second extraction process (formed elements extracted by range R2 in Figure 6). In the graph in Figure 30, the horizontal axis shows the variation in brightness in brightfield, and the vertical axis shows the frequency. Range R32 is the same as the range (range in the vertical axis direction) in the variation in brightness in brightfield for range R3 in Figure 9.

[0179] In this third extraction step, formed elements contained in range R32 are extracted. As a result, in the case of the urine sample with sample ID "F005", 14,945 formed elements from range R32 are extracted from the 42,430 formed elements extracted from range R2. In other words, 35.2% of the formed elements in range R2 are extracted, and 64.8% are excluded. Since ranges R31 and R32 contain at least the same number of formed elements as range R3, ranges R31 and R32 contain the same number or more mulberry bodies as range R3.

[0180] As described above, even if either the contrast value of the formed elements or the luminance variation value of the formed elements based on the bright-field image is used in the third extraction step, formed elements that are unlikely to be Mulberry bodies can be largely excluded, and Mulberry bodies can be extracted more accurately.

[0181] <Other examples of changes> In the above embodiment, as shown in Figure 3, the first extraction step, the second extraction step, and the third extraction step were performed in this order. However, the execution order of the first extraction step, the second extraction step, and the third extraction step is not limited to this. For example, the second extraction step, the first extraction step, and the third extraction step may be performed in this order. If the execution order of the three extraction steps is changed, the first extraction step, the second extraction step, and the third extraction step will perform extraction on the formed subgroups extracted by the previously executed extraction step. Even if the execution order of the three extraction steps is changed, the formed components ultimately extracted will be the same as in the above embodiment.

[0182] However, as can be seen by referring to the formed fractions in Figure 12, the number of formed components that can be excluded in the first extraction step is several times greater than the number of formed components that can be excluded in the other extraction steps. Therefore, if the first extraction step is performed first among the three extraction steps, the subsequent processing can proceed with a reduced number of formed components, thereby reducing the computational load on the processing unit 11 in the later steps.

[0183] In the above embodiment, the image of the formed elements was displayed in step S7 of Figure 3, but the processing in step S7 may be omitted. In this case, for example, the formed elements extracted by the first to third extraction steps may be image-analyzed by the processing unit 11 or other device, and it may be determined that each formed element is likely to be a mulberry body.

[0184] In the above embodiment, the average brightness for nucleic acids (a value relating to the first fluorescence amount produced by the first fluorescent dye) is obtained from the first fluorescence image, and the average brightness for red blood cells (a value relating to the second fluorescence amount produced by the second fluorescent dye) is obtained from the second fluorescence image. However, the embodiment is not limited to this, and both the fluorescence produced by the first fluorescent dye and the fluorescence produced by the second fluorescent dye may be guided onto the same light-receiving surface of the camera 154, and the camera 154 may capture these two fluorescences to generate a single fluorescence image represented in color. In this case, for example, the processing unit 11 extracts both the average brightness for nucleic acids and the average brightness for red blood cells from the generated single fluorescence image by image processing.

[0185] In the above embodiment, as shown in Figure 2, the imaging unit 19 includes a flow cell 110 and images multiple formed elements in the sample flowing through the flow cell 110. However, the imaging unit 19 is not limited to this and may be composed of a microscope. In this case, the microscope images multiple formed elements in the sample on a glass slide and generates at least a fluorescence image for each formed element, preferably a first fluorescence image, a second fluorescence image, and a bright-field image. The processing unit 11 uses the images acquired by the microscope to extract mulberry bodies more accurately, similar to the above embodiment. In the above embodiment, the range R1 to R3 was a fixed region, but at least one of the ranges R1 to R3 may be a variable region. For example, at least one of the ranges R1 to R3 may be a range that changes according to the distribution state of formed elements in the scattergram.

[0186] Embodiments of the present invention can be modified in various ways as appropriate within the scope of the technical idea set forth in the claims. [Explanation of Symbols]

[0187] 1. Formed element detection device 11 Processing Section 12a Program 18 Sample Preparation Section 19 Imaging Unit 110 Flow Cell R1, R2, R3 range

Claims

1. A sample is prepared containing a urine sample collected from the subject, a first fluorescent dye that stains nucleic acids, an antibody that specifically binds to red blood cells, and a second fluorescent dye that binds to the antibody. Multiple formed elements in the sample are imaged to obtain a first fluorescence image of the fluorescence produced by the first fluorescent dye, a second fluorescence image of the fluorescence produced by the second fluorescent dye, and a bright-field image. Based on the first fluorescence image, the second fluorescence image, and the bright-field image, formed elements corresponding to mulberry bodies are extracted. The step of extracting the formed elements is as follows: A first extraction step in which formed elements are extracted whose size value and focus state value based on the bright-field image meet the first condition, A second extraction step involves obtaining a value relating to the first fluorescence amount generated by the first fluorescent dye from the formed elements from the first fluorescence image, obtaining a value relating to the second fluorescence amount generated by the second fluorescent dye from the formed elements from the second fluorescence image, and extracting formed elements whose values ​​relating to the first fluorescence amount and the second fluorescence amount meet the second condition. A third extraction step includes extracting formed elements in which at least one of the contrast value of the formed elements and the luminance variation value of the formed elements, based on the bright-field image, meets a third condition. Method for detecting formed elements.

2. The first extraction step, the second extraction step, and the third extraction step perform extraction on the formed subgroups extracted by the previously performed extraction step. The method for detecting formed elements according to claim 1.

3. The first extraction step includes a step of extracting formed elements whose size values ​​at least match the first condition, The first extraction step is performed prior to the second and third extraction steps. The method for detecting formed elements according to claim 2.

4. The second condition is that the value relating to the first fluorescence amount and the value relating to the second fluorescence amount fall within the range corresponding to the mulberry body, The range corresponding to the mulberry bodies is a range that does not overlap with the range corresponding to red blood cells and the range corresponding to fungi. The method for detecting formed elements according to claim 1.

5. The step further includes displaying an image of the formed elements corresponding to the extracted mulberry bodies. The method for detecting formed elements according to claim 1.

6. The step of displaying the image of the formed element is to display the bright-field image of the extracted formed element. The method for detecting formed elements according to claim 5.

7. The step of displaying the images of the formed elements involves displaying the extracted images of the formed elements in a list. The method for detecting formed elements according to claim 5.

8. The step of displaying the images of the formed elements involves displaying the extracted images of the formed elements in a list in order of size. The method for detecting formed elements according to claim 7.

9. The step of displaying the images of formed elements includes receiving an instruction input to arrange the extracted images of formed elements in order of size, and rearranging the displayed list of images of formed elements according to the received instruction input. The method for detecting formed elements according to claim 7.

10. The step of displaying the image of the formed element includes receiving an instruction input to identify any one of the images of the formed element displayed in the list, and enlarging and displaying the image of the formed element identified by the instruction input. The method for detecting formed elements according to claim 7.

11. The step of displaying images of formed elements includes receiving an instruction input to identify an image of a mulberry body from the list of images of formed elements, and further displaying the counting result obtained by counting the images of the formed elements identified by the instruction input. The method for detecting formed elements according to claim 7.

12. The system stores the history of the aforementioned counting results for the same subject. The method for detecting formed elements according to claim 11.

13. Based on the aforementioned history, information based on the changes in the aforementioned counting results is displayed. The method for detecting formed elements according to claim 12.

14. The system displays a list of images of the formed elements, along with information based on the changes in the counting results. The method for detecting formed elements according to claim 13.

15. The step of displaying the images of formed elements includes receiving an instruction input to identify the image of the mulberry body from the list of images of formed elements, and associating a flag with the image of the formed element identified by the instruction input and storing it. The method for detecting formed elements according to claim 7.

16. A sample preparation unit prepares a sample containing a urine sample collected from a subject, a first fluorescent dye for staining nucleic acids, an antibody that specifically binds to red blood cells, and a second fluorescent dye that binds to the antibody. An imaging unit that images multiple formed elements in the sample and acquires a first fluorescence image of fluorescence produced by the first fluorescent dye, a second fluorescence image of fluorescence produced by the second fluorescent dye, and a bright-field image. The system includes a processing unit that extracts formed elements corresponding to mulberry bodies based on the first fluorescence image, the second fluorescence image, and the bright-field image, The aforementioned processing unit, A step of extracting formed elements in which at least one of the values ​​relating to the size of the formed elements and the values ​​relating to the focus state of the formed elements, based on the bright-field image, meets the first condition. A step of obtaining a value relating to the first fluorescence amount generated by the first fluorescent dye from the formed elements from the first fluorescence image, obtaining a value relating to the second fluorescence amount generated by the second fluorescent dye from the formed elements from the second fluorescence image, and extracting formed elements whose values ​​relating to the first fluorescence amount and the second fluorescence amount meet the second condition, The process involves extracting formed elements whose contrast value and the luminance variation value based on the bright-field image meet the third condition. Formed element detection device.

17. The imaging unit includes a flow cell through which the sample flows, and images the plurality of formed elements in the sample flowing through the flow cell. The formed element detection device according to claim 16.

18. A program that causes a computer to perform a process to detect formed elements in a urine sample collected from a subject, Based on the first fluorescence image of the fluorescence produced by the first fluorescence dye, the second fluorescence image of the fluorescence produced by the second fluorescence dye, and the bright-field image obtained by imaging multiple formed elements in the urine sample, which includes the urine sample, a first fluorescence dye that stains nucleic acids, an antibody that specifically binds to red blood cells, and a second fluorescence dye that binds to the antibody, formed elements corresponding to mulberry bodies are extracted. The process for extracting the aforementioned formed elements is as follows: A process for extracting formed elements in which at least one of the values ​​relating to the size of the formed element and the values ​​relating to the focus state of the formed element, based on the bright-field image, meets the first condition. A process to obtain a value relating to the first fluorescence amount generated by the first fluorescent dye from the formed elements from the first fluorescence image, obtain a value relating to the second fluorescence amount generated by the second fluorescent dye from the formed elements from the second fluorescence image, and extract formed elements whose values ​​relating to the first fluorescence amount and the second fluorescence amount meet the second condition, The process includes extracting formed elements in which at least one of the contrast value of the formed element and the luminance variation value of the formed element, based on the bright-field image, satisfies the third condition. program.

Images