Abnormal spotting detection method, analysis method, abnormal spotting detection device, and analysis device

Abstract

This abnormal spotting detection method detects the abnormal spotting of a specimen liquid on a reagent pad on which the specimen liquid is spotted, by using at least one portion of an upper surface image of the reagent pad and at least one portion of a side surface image of the reagent pad connected to the upper surface.

Description

Dropping abnormality detection method, analysis method, dropping abnormality detection device, and analysis device

[0001] The present disclosure relates to a dropwise deposition abnormality detection method, an analysis method, a dropwise deposition abnormality detection device, and an analysis device.

[0002] Japanese Patent Laid-Open Publication No. 5-256783 describes a method for detecting abnormal immersion of a color test paper, which measures the amount of reflected light from a reflectance correction piece of color test paper immersed in a specimen or water, calculates a threshold based on the average value of the most recent S number of reflected light amounts, and determines that a color test paper with an amount of reflected light higher than the threshold is a color test paper that has not been immersed in a specimen.

[0003] Japanese Patent Laid-Open Publication No. 7-333152 describes a reflectance measurement method in which an antenna that emits high frequency waves supplied from an oscillator circuit is brought into contact with or close to the reagent layer of a test strip itself, and a change in the current consumption of the oscillator circuit caused by the application of a sample to the reagent layer is detected, and the measurement time is automatically started simultaneously with the supply of the sample.

[0004] In an analyzer for analyzing the components of a sample, a sample liquid is applied to a reagent pad attached to a test strip, and the analysis is performed using this reagent pad. In this case, it is desirable to be able to detect abnormalities in the application of the sample liquid to the reagent pad.

[0005] For example, the technology described in Japanese Patent Laid-Open No. 5-256783 assumes that the color test paper is immersed in a specimen or water, making it difficult to detect abnormalities in the placement of the test paper on the reagent pad. The technology described in Japanese Patent Laid-Open No. 7-333152 detects the supply of sample and automatically starts the measurement time, but it is not possible to determine whether the amount of sample required for analysis has been supplied. Furthermore, if the antenna comes into contact with the reagent pad, the antenna may be contaminated by the sample.

[0006] An object of the present disclosure is to enable detection of abnormalities in the placement of sample liquid on a reagent pad.

[0007] The technology disclosed herein detects abnormal application of sample liquid to a reagent pad using at least a portion of an image of the top surface of the reagent pad onto which the sample liquid is applied and at least a portion of an image of the side surface of the reagent pad connected to the top surface.

[0008] The technology of the present disclosure can detect abnormalities in the placement of a sample on a reagent pad.

[0009] FIG. 1 is a perspective view showing an analytical device of the first embodiment. FIG. 2 is a perspective view showing a test strip and a sample container used in the analytical device of the first embodiment. FIG. 3 is a schematic diagram showing an example of the internal configuration of the analytical device of the first embodiment. FIG. 4 is a side view showing a state in which an image of a reagent pad is being captured in the analytical device of the first embodiment. FIG. 5 is a front view showing a state in which an image of a reagent pad is being captured in the analytical device of the first embodiment. FIG. 6 is a configuration diagram showing an example of the hardware configuration of the analytical device of the first embodiment. FIG. 7 is a configuration diagram showing an example of the functional configuration of the analytical device of the first embodiment. FIG. 8 is a flowchart outlining the overall operation of the analytical device of the first embodiment. FIG. 9 is an explanatory diagram showing a test strip imaged by the imaging device of the analytical device of the first embodiment, and a first extracted image and a second extracted image extracted from the test strip. FIG. 10 is an explanatory diagram showing how the image processing unit of the analytical device of the first embodiment extracts the second extracted image for each reagent pad. FIG. 11 is an explanatory diagram showing a state in which a reagent pad has fallen off the test strip. FIG. 12A is a diagram showing an example of an image of an actual test strip imaged by the imaging device of the analytical device of the first embodiment. FIG. 12B is a diagram showing an example of an image of an actual reagent pad, obtained by cutting out an image including the outer edge of one reagent pad from the example image shown in FIG. 12A. FIG. 13A is an explanatory diagram showing a reagent pad before a color reaction in the analyzer of the first embodiment. FIG. 13B is an explanatory diagram showing a reagent pad after a color reaction in the analyzer of the first embodiment. FIG. 13C is an explanatory diagram showing a light-colored reagent pad after a color reaction in the analyzer of the first embodiment. FIG. 14 is a flowchart showing details of the process for generating a reagent pad cut-out image in the flowchart of FIG. 8. FIG. 15 is a diagram showing how two minimum rectangles are detected in which the proportion of white pixels is abnormal to a preset value between the threshold values ​​of 255 and 0. FIG. 16 is a diagram showing how a reagent pad cut-out image is generated by cutting out the area between the upper and lower bases of the two minimum rectangles detected in the reagent pad image. FIG. 17 is a flowchart showing details of the process for determining whether a reagent pad has fallen off in the flowchart of FIG. 8. FIG. 18 is a diagram showing how a minimum rectangle is detected in which the proportion of black pixels is equal to or greater than a preset value when the reagent pad is in a normal state between the threshold values ​​of 255 and 0.FIG. 19 is a diagram showing an example of a cutout image of a reagent pad when the reagent pad has fallen off the urine test strip. FIG. 20 is a diagram showing a state in which, when the threshold value is changed from 255 to 0, a minimum rectangle with a black pixel ratio equal to or greater than a predetermined value is not detected when the reagent pad has fallen off. FIG. 21 is an explanatory diagram showing a test strip used in the analyzer of the first embodiment and a cutout image cut out from this test strip. FIG. 22 is a partially enlarged perspective view of a test strip used in the analyzer of the first embodiment. FIG. 23 is an enlarged perspective view of a reagent pad of a test strip used in the analyzer of the first embodiment. FIG. 24 is a diagram showing a cutout image generated in the analyzer of the first embodiment. FIG. 25 is a flowchart showing details of the reagent pad deposition abnormality detection process in the flowchart of FIG. 8. FIG. 26 is a front view showing a state in which a reagent pad is being imaged in the analyzer of the second embodiment.

[0010] An example of an embodiment of the technology of the present disclosure will be described in detail below with reference to the drawings. Note that components and processes that perform the same operations, actions, and functions are given the same reference numerals throughout the drawings, and duplicated descriptions may be omitted as appropriate. Each drawing is merely a schematic illustration to allow a sufficient understanding of the technology of the present disclosure. Therefore, the technology of the present disclosure is not limited to the illustrated examples. Furthermore, in this embodiment, descriptions of configurations that are not directly related to the present disclosure or well-known configurations may be omitted.

[0011] Fig. 1 shows an analyzer 12 according to a first embodiment. Fig. 2 shows a test strip 18 used in the analyzer 12 together with a sample container 14. In the drawing, the width, depth, and height directions of the analyzer 12 are indicated by arrows W, D, and H, respectively. The analyzer 12 is used in an orientation in which the width and depth directions are horizontal.

[0012] The analyzer 12 is a device that analyzes a sample liquid (e.g., urine) contained in a sample container 14 shown in Figure 2. The sample container 14 is, for example, cylindrical overall, with a diameter that gradually decreases in a conical manner from the middle to the bottom in the longitudinal direction. Sample containers of this shape are sometimes called Spitz tubes. The sample container 14 is not limited to a Spitz tube, and may be any container that can contain a sample.

[0013] The analyzer 12 is equipped with a mounting base 22 and a spotting device 24. A plurality of test strips 18 are placed in a predetermined arrangement on the mounting base 22. The test strips 18 have a long backing 18D. A plurality of reagent pads 20 are attached to the backing 18D at regular intervals along the length of the backing 18D.

[0014] Each reagent pad 20 contains a reagent that reacts with a different component in the sample liquid and develops a color corresponding to the concentration of that component. Various reagents are used depending on the test items of the sample liquid. Therefore, before the sample liquid is applied, the reagent pad 20 has a different color depending on the color of the reagent. Even after the sample liquid is applied and develops a color, the reagent pad 20 has a different color depending on the type of reagent.

[0015] When a urine sample is to be analyzed, the test strip 18 used is a urine test strip that includes reaction areas for multiple test items, such as urinary protein, urinary ketone bodies, and urinary sugar. The sample to be analyzed is not limited to a urine sample, and may be a biological component other than urine (e.g., blood, plasma, saliva, etc.). The sample may also be a liquid for water quality testing.

[0016] FIG. 3 shows an example of the internal configuration of the analysis device 12 of this embodiment.

[0017] The analyzer 12 has a test strip supplying device 54. The test strip supplying device 54 supplies test strips 18 for performing qualitative tests on urine samples to a predetermined location P1. The spotting device 24 uses a nozzle 24N to collect urine samples from the sample container 14 and deposits the collected urine sample onto the test strip 18 supplied to the predetermined location P1. The nozzle 24N is movable vertically and horizontally by a drive mechanism (not shown).

[0018] In the disclosed technology, the reagent pad 20 has a flat rectangular parallelepiped shape. As shown in Figures 4 and 5, the reagent pad 20 has one top surface 56 and four side surfaces 58 connected to the top surface 56. When viewed in the normal direction of the test paper 18, the top surface 56 of the reagent pad 20 has a rectangular (including square) shape, and the side surfaces 58 also have a rectangular shape. The top surface 56 of the reagent pad 20 is located opposite the backing paper 18D and is the surface onto which the drop liquid is applied.

[0019] A plurality of test papers 18 are placed on the table 22 with their longitudinal direction aligned with the depth direction of the analyzer 12 and at regular intervals across the width of the analyzer 12 .

[0020] The spotting device 24 includes a nozzle 24N. The nozzle 24N can be raised and lowered and moved in the depth direction of the analyzer 12 by a moving mechanism (not shown). The nozzle 24N aspirates a predetermined amount of sample liquid from the sample container 14 and dispenses it onto each of the reagent pads 20. This causes a predetermined amount of sample liquid to be dispensed onto the upper surface 56 of each of the reagent pads 20. Then, each of the reagent pads 20 is analyzed by the analysis unit 26.

[0021] As shown in FIGS. 4 and 5 , the analyzer 12 of this embodiment includes an imaging device 44. The imaging device 44 is an example of an imaging unit of the disclosed technology. The imaging device 44 is located above the test paper 18 placed in a predetermined position on the mounting base 22. The imaging device 44 includes a camera. The angle of view α of this camera is set so that all of the reagent pads 20 on one test paper 18 can be imaged from above the test paper 18. In the example shown in FIG. 4 , all of the reagent pads 20 on one test paper 18 are captured within the angle of view α by a single imaging device 44 located at the center of the longitudinal direction of the test paper 18, i.e., the arrangement direction of the multiple reagent pads 20. In particular, a single imaging by the imaging device 44 can image the top surface 56 and one of the side surfaces 58 (side surface 58A closer to the imaging device 44) of all of the reagent pads 20. Furthermore, by imaging the top surface 56, the four sides 60 of the top surface 56 (the outer edges of the top surface 56) can also be imaged.

[0022] Specifically, of the four side surfaces of the reagent pad 20, the side surface 58A on the imaging device 44 side is the side surface that is imaged by the imaging device 44. During imaging, an edge 60A that forms the boundary between the top surface 56 of the reagent pad 20 and the side surface 58A is also imaged. In contrast, the side surface 58B opposite the side surface 58A is not imaged by the imaging device 44. However, the edge 60B that forms the boundary between the top surface 56 of the reagent pad 20 and the side surface 58B is imaged. Furthermore, the two side surfaces 58C that are orthogonal to the side surfaces 58A and 58B are not imaged by the imaging device 44, but the edge 60C that forms the boundary between the top surface 56 of the reagent pad 20 and the side surfaces 58C is imaged.

[0023] Depending on the relative positions of the test paper 18 and the imaging device 44, the optical axis CA of the imaging device 44 may be perpendicular to the top surface 56 of the reagent pad 20 and within the range of the top surface 56, as in the case of the reagent pad 20C shown in FIG. 4 . Even in this case, the top surface 56 of the reagent pad 20C, including its four sides 60, can be imaged by the imaging device 44. Each of the four sides 60 is also a side of the corresponding one of the four side surfaces 58. Therefore, it can be said that the imaging device 44 also images a portion of the side surfaces 58 of the reagent pad 20C. Note that, because the arrangement of the reagent pad 20 on the test paper 18 is known in advance, the relative position of the test paper 18 with respect to the imaging device 44 and the angle of the optical axis CA may be set in advance so that the optical axis CA of the imaging device 44 is outside the range of the top surface 56.

[0024] 6 , the analyzer 12 includes an analyzer unit 26, a processor 30, a memory 32, a storage 34, a spotting device 36, a communication device 38, an operation panel 40, an imaging device 44, and a test paper supply device 54. These elements in the device main body 16 are communicatively connected to one another via a bus 42. The processor 30, the memory 32, and the storage 34 form a computer 28. The computer 28 controls the operation of the analyzer 12.

[0025] The analysis section 26 is a section where the actual analysis is performed on the sample liquid applied to the test paper.

[0026] The storage 34 stores an analysis program for executing the analysis process of the sample by the analyzer 12. The storage 34 also records the operation history of the analyzer 12 and various data.

[0027] The processor 30 is capable of executing various programs and controlling each element in the device main body 16. Specifically, the processor 30 reads a program from the storage 34 and executes the program using the memory 32 as a work area. That is, the processor 30 controls each element in the device main body 16 and performs various arithmetic processing in accordance with the program stored in the storage 34.

[0028] The memory 32 can temporarily store programs and various data as a working area.

[0029] The storage 34 is, for example, a read-only memory (ROM), a hard disk drive (HDD), or a solid state drive (SDD), and stores various programs and data. These programs include not only application programs such as the measurement support program described above, but also the operating system.

[0030] The communication device 38 is a device that communicates with devices external to the analytical device 12. For communication, for example, wired connection standards such as Ethernet (registered trademark) and FDDI (Fiber Distributed Data Interface) and wireless connection standards such as Wi-Fi and Bluetooth (registered trademark) are used.

[0031] In the computer 28, the processor 30 executes a control program, thereby functioning as an image processing unit 46, a measurement unit 48, and a detection unit 50 shown in FIG.

[0032] The image processing unit 46 cuts out a partial area of ​​the image of the reagent pad 20 captured by the imaging device 44, thereby generating a first cut image 61 and a second cut image (reagent pad image) 62 shown in Fig. 9, and a cut image 52 shown in Fig. 21, for each reagent pad 20. The first cut image 61 is an image of an area smaller than the shape of the reagent pad 20 and does not include the outer edge of the reagent pad 20. The second cut image 62 is an image of an area larger than the shape of the reagent pad 20 and includes the area outside the outer edge of the reagent pad 20. The cut image 52 is an image of an area including the shape of a part of the edge 60 that is the boundary between the top surface 56 and the side surface 58, from the images of the top surface 56 and the side surface 58 of the reagent pad 20.

[0033] For example, each of the multiple reagent pads 20 has a rectangular shape in a plan view. In this case, the first cut-out image 61 is a rectangular cut-out image with dimensions of 80% to 90% of the length of each side of each reagent pad 20. The second cut-out image 62 is a rectangular cut-out image with dimensions of 110% to 150% of the length of each side of each reagent pad 20.

[0034] The measurement unit 48 measures the color state of the urine sample using the extracted image 52. Alternatively, the measurement unit 48 may measure the color state of the urine sample using the first extracted image 61. Specifically, the measurement unit 48 converts the color values ​​in the RGB color space of the extracted image 52 or the first extracted image 61 into color values ​​in the Lab color space, and compares them with preset Lab values ​​to determine the color state of each reagent pad, thereby performing a qualitative analysis of the urine sample applied to the reagent pad 20.

[0035] The detection unit 50 detects abnormal deposition of the sample liquid on the reagent pad 20 using the cropped image 52. The detection unit 50 may also detect abnormal deposition of the sample liquid on the reagent pad 20 using the second cropped image 62. Here, "abnormal deposition" includes a state in which a sufficient amount of sample liquid is not deposited on the reagent pad 20, and there is a portion of the upper surface 56 where the sample liquid is not deposited. It also includes a state in which the sample liquid is deposited on the entire upper surface 56, but the amount deposited is less than a predetermined amount, and the sample liquid has not sufficiently penetrated in the thickness direction of the reagent pad 20 in, for example, a portion of the side surface 58.

[0036] In the detection unit 50, for example, a cut-out image 52 of the reagent pad 20 in a normal state of sample liquid deposition and a cut-out image 52 of the reagent pad 20 in an abnormal state of sample liquid deposition are used as learning data to generate an abnormal dispensing model. Specifically, the abnormal dispensing model is generated by machine learning based on both a normal state and a state in which an abnormal dispensing occurs, using a top image and a side image of the cut-out image 52. The newly generated cut-out image 52 is input into this abnormal dispensing model to detect abnormal dispensing of sample liquid on the reagent pad 20.

[0037] Furthermore, the detection unit 50 detects the state in which the reagent pad 20 has fallen off the test paper 18 as an abnormality in the reagent pad 20 by identifying the shape of an area in the second cut-out image 62 that is a color different from the color of the test paper 18.

[0038] If the detector 50 detects that the reagent pad 20 has peeled off from the test paper 18, the abnormality is detected by the following process.

[0039] Specifically, the detection unit 50 generates a reagent pad cutout image by cutting out only the test paper region from the reagent pad image (second cutout image 62) cut out by the image processing unit 46. The detection unit 50 then binarizes the generated reagent pad cutout image using the maximum or minimum value in a predetermined brightness range as a threshold value and determines whether a rectangular shape can be detected. If a rectangular shape cannot be detected, the detection unit 50 raises or lowers the threshold value and repeats the binarization process until a rectangular shape is detected or until all brightness values ​​in the predetermined brightness range are used as the threshold value. If a rectangular shape can be detected at any threshold value, the detection unit 50 determines that the reagent pad has not fallen off. If a rectangular shape cannot be detected even when all brightness values ​​in the predetermined brightness range are used as the threshold value, the detection unit 50 determines that the reagent pad has fallen off.

[0040] The detection unit 50 determines that a rectangular shape has been detected when the ratio of white pixels or black pixels in the smallest rectangle surrounding the area of ​​white pixels or black pixels detected while changing the threshold value becomes equal to or greater than a preset value.

[0041] The image processing unit 46 cuts out reagent pad images for each reagent pad 20 from the image of the test paper 18 captured by the imaging device 44 by dividing the image of the test paper 18 in the longitudinal direction based on the assumed position of each reagent pad 20 on the test paper 18. The detection unit 50 then converts the reagent pad image into a monochrome image and generates a reagent pad cut-out image by cutting out only the area sandwiched between the upper and lower bases of the reagent pad 20 from the converted monochrome reagent pad image.

[0042] Specifically, the detection unit 50 binarizes the reagent pad image converted into a monochrome image using the maximum value of a predetermined brightness range as a threshold, detects the smallest rectangle enclosing a region of white pixels in the binarized image, and determines whether two smallest rectangles with a white pixel ratio equal to or greater than a predetermined value can be detected. If the detection unit 50 cannot detect two smallest rectangles with a white pixel ratio equal to or greater than the predetermined value, it lowers the threshold and repeats the binarization process until two smallest rectangles with a white pixel ratio equal to or greater than the predetermined value are detected. If the detection unit 50 detects two smallest rectangles with a white pixel ratio equal to or greater than the predetermined value, it determines that the two smallest rectangles are regions of a urine test strip where no reagent pad is placed, and generates a reagent pad cropped image by cropping only the region between the upper and lower bases of the reagent pad 20 from the reagent pad image converted into a monochrome image.

[0043] Next, the operation of the analysis device 12 of this embodiment and the analysis method will be described.

[0044] FIG. 8 shows a schematic flow of the overall operation of the analyzer 12 of this embodiment.

[0045] 8 , first, in step S101, the image processing unit 46 performs tilt correction and distortion correction on the image of the test paper 18 captured by the imaging device 44. Because the test paper 18 is long in the landscape orientation, when the image is captured by a single imaging device 44, the edges of the test paper 18 may be tilted more than the center, resulting in a distorted image. Therefore, the image processing unit 46 identifies the region of the test paper 18 from the image captured by the imaging device 44, and performs image processing to suppress tilt and distortion in the identified region of the test paper 18.

[0046] Next, in step S102, the image processing unit 46 executes an image cutting process. In the image cutting process, a first cut image 61, a second cut image 62, and a cut image 52 are cut out for each reagent pad 20 from the image of the test paper 18. As shown in FIG. 9 , the first cut image 61 is an image of an area smaller than the shape of the reagent pad 20 and does not include the outer edge of the reagent pad 20. As also shown in FIG. 9 , the second cut image 62 is an image of an area larger than the shape of the reagent pad 20 and includes the area outside the outer edge of the reagent pad 20. The second cut image 62 is also a reagent pad image. As shown in FIG. 21 , the cut image 52 is an image of an area including the shape of a portion of the edge 60 that is the boundary between the top surface 56 and the side surface 58 of the reagent pad 20 from the image of the top surface 56 and the side surface 58.

[0047] Next, in step S103, the detection unit 50 generates a reagent pad cutout image. The reagent pad cutout image generation process is a process of cutting out only the area of ​​the test paper 18 from the reagent pad image, which is the second cutout image 62, by cutting out the background area other than the test paper 18. Next, in step S104, the detection unit 50 executes a process of determining whether the reagent pad 20 has fallen off. The process of determining whether the reagent pad 20 has fallen off is a process of determining whether the reagent pad 20 has fallen off from the test paper 180, using the generated reagent pad cutout image. Furthermore, in step S105, the detection unit 50 executes a deposition abnormality detection process of detecting whether or not a deposition abnormality has occurred with the sample liquid on the reagent pad 20. The processes of steps S103, S104, and S105 will be described in detail below. Then, in step S106, the detection unit 50 executes an analysis of the sample liquid.

[0048] 9 shows how a first extracted image 61 and a second extracted image (reagent pad image) 62 are extracted from an image of one reagent pad 20. The first extracted image 61 is sent to the measurement unit 48 and used for a qualitative test based on the color state of the reagent pad 20. The second extracted image 62 is sent to the detection unit 50 and used for detecting a dropped reagent pad and detecting an abnormal application of the reagent pad.

[0049] As can be seen from FIG. 9 , the area of ​​the test paper 18 other than the reagent pad 20 is white, while the background when the test paper 18 is photographed is dark. FIG. 21 shows how a cropped image 52 is cropped from an image of the reagent pad 20. Specifically, an image of a portion of the top surface 56 of the reagent pad 20 (top image) and an image of the side surface 58 connected to the top surface 56 (side image) are cropped for each reagent pad 20 from the captured image of the test paper 18 to form the cropped image 52. As shown in FIG. 4 , of the multiple reagent pads 20, the top surface 56 and the side surface 58 connected to the top surface 56 are included in the cropped image 52, except for the reagent pads 20 for which the optical axis CA of the imaging device 44 is within the range of the top surface 56. In each of these cropped images 52, a portion of the top surface 56 of the reagent pad 20 and the side surface 58 connected to the top surface 56 appear as an image, including the edge 60 that is the boundary between these surfaces. The image processing unit 46 performs this image cropping process for each of the multiple reagent pads 20. For each reagent pad 20, the cropped image 52 is generated not as an entire top surface 56 of the reagent pad 20, but as two images: a portion including side 60A and close to side 60A, and a portion including side 60B and close to side 60B. The cropped images 52 are sent to the measurement unit 48 and used for detecting abnormal deposition of the reagent pad 20 and for qualitative testing based on the color state of the reagent pad 20. The image processing unit 46 associates the multiple reagent pads 20 on one test paper 18 with the images captured of each reagent pad 20.

[0050] The image processing unit 46 executes such image extraction processing for each of the plurality of reagent pads 20 .

[0051] The detection unit 50 detects an abnormality in each reagent pad 20 or an abnormality in the application of a urine sample to the reagent pad 20 based on the second cropped image 62 cropped by the image processing unit 46. First, a detailed description will be given of a case in which the detection unit 50 determines whether each reagent pad 20 is properly attached or has come off and fallen off based on the second cropped image 62, and a method for detecting an abnormality in the application of a urine sample to the reagent pad 20, i.e., an abnormal application detection process, will be described later.

[0052] An example image of a test paper 18 in which a certain reagent pad 20 has fallen off is shown in Figure 11. Referring to Figure 11, it can be seen that the image of the test paper 18 includes an area 64 where a certain reagent pad 20 has fallen off. The urine sample is also applied to this area 64 by the application device 24, so that the urine sample is applied directly to the test paper 18. The detection unit 50 detects that a reagent pad 20 has fallen off from the image of this area 64.

[0053] Fig. 12A shows an example of an image of an actual test paper 18 captured by the imaging device 44. Fig. 12B shows an example of an actual reagent pad image obtained by cutting out an image including the outer edge of one reagent pad 20 from the example image of the test paper 18 shown in Fig. 12A as a reagent pad image. Fig. 12A shows that the color density of each reagent pad 20 is different.

[0054] Although the example images shown in Figures 12A and 12B are black and white, they are actually color images. The test paper 18 has a white background, and the reagent pads 20 attached to the test paper 18 are various colors, such as purple, green, pink, blue, yellow, brown, and white, depending on the color of the reagent contained therein. Each reagent pad 20 changes color from its original color to various colors when a urine sample is applied. For example, a GLU (glucose) pad used to measure the glucose concentration in a urine sample will darken if the glucose concentration in the urine sample is high.

[0055] Figures 13A and 13B show how the color of the reagent pad 20 changes due to this color reaction. Figure 13A shows an example of a reagent pad image corresponding to a certain reagent pad 20. In the reagent pad image of Figure 13A, the area of ​​the test paper 18 is white, while the background area other than the test paper 18 is black. As described above, the reagent pad 20 can take on various colors depending on the color of the reagent. However, when the reagent pad 20 changes to a dark color due to the application of a urine sample, the color of the reagent pad 20 may change to a darker color, as shown in Figure 13B. As a result, the color of the background area and the color of the reagent pad area in the reagent pad image may have similar intensities. Here, when attempting to determine whether the reagent pad 20 has fallen off using image processing as described above, it is necessary to detect whether the reagent pad 20 has a rectangular shape. To do this, it is necessary to determine where the test paper 18 area ends and where the background area other than the test paper 18 begins, otherwise it is impossible to determine whether the reagent pad 20 has a rectangular shape.

[0056] Conversely, some reagent pads 20 may be nearly white even when a urine sample is applied, similar in color to the test paper 18. An example of a reagent pad image of such a light-colored reagent pad 20 is shown in FIG. 13C . Because the color density of the reagent pads 20 varies, it can be difficult to make the same determination for light-colored reagent pads 20 and dark-colored reagent pads 20 when attempting to detect the presence or absence of a rectangular shape using a fixed threshold value. Specifically, unless a threshold value is set between the color density value of the test paper 18 and the color density value of the reagent pad 20, it is difficult to distinguish between the test paper 18 area and the reagent pad 20 area. However, if the threshold value were changed for each color of the reagent pad 20, it would be impossible to determine whether or not reagent pads 20 of various colors have fallen off using the same process.

[0057] Therefore, in the analyzer 12 of this embodiment, the detached state of the reagent pad 20, which changes color in various ways, can be determined by the same process by performing the process described below.

[0058] First, the details of the process for generating the cut-out image of the reagent pad (step S103) in the flowchart of FIG. 8 will be described with reference to the flowchart of FIG.

[0059] In step S201, the detection unit 50 performs a monochrome conversion process on the reagent pad image cut out by the image processing unit 46. Here, the image of the test paper 18 captured by the imaging device 44 is a color image in the RGB color space, with each pixel value ranging from 0 to 255. Therefore, the reagent pad image cut out by the image processing unit 46 is also a color image in the RGB color space. The detection unit 50 first converts the color image of the reagent pad image into an HSV image. An HSV image is an image in which the color value of each pixel is expressed using H (Hue), S (Saturation), and V (Value). The detection unit 50 acquires the V image of the HSV image as a monochrome image.

[0060] Next, in step S202, the detection unit 50 sets the threshold value for the binarization process to 255. Note that instead of setting the threshold value to 255, the largest luminance value in the monochrome image may be set as the threshold value for the binarization process.

[0061] Then, in step S203, the detection unit 50 performs binarization processing on the monochrome reagent pad image based on the set threshold value.

[0062] In step S204, the detection unit 50 detects the smallest rectangles enclosing the white pixels from the binary image obtained after the binarization process has been performed. Next, in step S205, the detection unit 50 calculates the proportion of white pixels in each of the detected smallest rectangles.

[0063] In step S206, the detection unit 50 determines whether the ratio of white pixels in the two minimum rectangles in the binary image is equal to or greater than a predetermined value, for example, 90%. Note that this value of 90% is merely an example, and any value appropriate for detecting the minimum rectangle is set.

[0064] If it is determined in step S206 that the ratio of white pixels in both of the two smallest rectangles in the binary image is 90% or more, the process proceeds to step S207. In step S207, the detection unit 50 generates a reagent pad cutout image by cutting out the reagent pad image using the upper and lower bases of the detected smallest rectangle. If it is determined in step S206 that the ratio of white pixels in either of the two smallest rectangles in the binary image is not 90% or more, the process proceeds to step S208. In step S208, the detection unit 50 determines whether the currently set threshold is 0.

[0065] If it is determined in step S208 that the currently set threshold is not 0, the process proceeds to step S209. In step S209, the detection unit 50 decrements the threshold by 1, returns to step S203, and performs the binarization process again.

[0066] If it is determined in step S208 that the currently set threshold is 0, the process proceeds to step S210. In step S210, the detection unit 50 determines that an error has occurred. If an error has been determined in step S210, the detection unit 50 abnormally terminates, determines that there is some abnormality in the reagent pad in the reagent pad image, and terminates the process.

[0067] By performing the above-described process, steps S203 to S206 are repeated until the threshold value changes from 255 to 0. As a result, as shown in Fig. 15, when the reagent pad 20 is in a normal state, two minimum rectangles are detected in which the proportion of white pixels is equal to or greater than a preset value at either threshold value. When the two minimum rectangles are detected, the detection unit 50 determines that these two minimum rectangles are areas of the test paper 18 on which the reagent pad 20 is not placed, and generates a reagent pad cut-out image by cutting out only the area between the upper and lower bases of the reagent pad 20 from the reagent pad image converted into a monochrome image.

[0068] FIG. 16 is a diagram showing how a reagent pad cut-out image is generated by cutting out the area sandwiched between the upper and lower bases of the two minimum rectangles detected in this manner.

[0069] If the threshold value becomes 0 without detecting two minimum rectangles even after repeating the processes of steps S203 to S206, an error is determined in step S210.

[0070] Next, the details of the process of determining whether or not the reagent pad has fallen off (step S104) in the flowchart of FIG. 8 will be described with reference to the flowchart of FIG.

[0071] First, in step S301, the detection unit 50 sets the threshold value for the binarization process to 255. Note that instead of setting the threshold value to 255, the largest luminance value in the monochrome image may be set as the threshold value for the binarization process.

[0072] Next, in step S302, the detection unit 50 performs binarization processing on the cut-out image of the reagent pad cut out in the processing of step S103 based on the set threshold value.

[0073] In step S303, the detection unit 50 detects the smallest rectangle enclosing the black pixels from the binary image after the binarization process has been performed. Next, in step S304, the detection unit 50 calculates the proportion of black pixels in the detected smallest rectangle.

[0074] In step S305, the detection unit 50 determines whether the proportion of black pixels in the minimum rectangle in the binary image is equal to or greater than a predetermined value, for example, 90%. Note that this value of 90% is merely an example, and any value suitable for detecting the minimum rectangle is set.

[0075] If it is determined in step S305 that the proportion of black pixels in the minimum rectangle in the binary image is 90% or more, the process proceeds to step S308, in which the detection unit 50 determines that the reagent pad is not in a detached state.

[0076] If it is determined in step S305 that the proportion of black pixels in the minimum rectangle in the binary image is not 90% or more, the process proceeds to step S306. In step S306, the detection unit 50 determines whether the currently set threshold is 0.

[0077] If it is determined in step S306 that the currently set threshold is not 0, the process proceeds to step S307. In step S307, the detection unit 50 decrements the threshold by 1 and re-executes the binarization process of step S302.

[0078] If it is determined in step S306 that the currently set threshold is 0, the process proceeds to step S309. In step S309, the detection unit 50 determines that the reagent pad 20 is in a detached state.

[0079] By performing the above-described process, steps S302 to S305 are repeated until the threshold value changes from 255 to 0. As a result, as shown in Fig. 18, if the reagent pad 20 is in a normal state, the smallest rectangle in which the proportion of black pixels is equal to or greater than a preset value at any threshold value is detected. When this smallest rectangle is detected, the detection unit 50 determines that this smallest rectangle is the outer shape of the reagent pad 20, and determines that the reagent pad 20 is not in a dropped state.

[0080] Figure 19 shows an example of a cut-out image of a reagent pad in which the reagent pad 20 has fallen off the test paper 18. In the example of the cut-out image of a reagent pad shown in Figure 19, it can be seen that the reagent pad 20 is not present in the image, and a urine sample has been directly applied to the test paper 18.

[0081] Figure 20 shows the change in the binary image when the above-described processes of steps S302 to S305 are repeated for the reagent pad cutout image shown in Figure 19. Referring to Figure 20, it can be seen that no matter how the threshold value is changed, a minimum rectangle in which the proportion of black pixels is equal to or greater than a predetermined value is not detected. In other words, if the reagent pad 20 has fallen off the test paper 18, no rectangular shape is detected from the reagent pad cutout image. Therefore, if no rectangular shape is detected from the reagent pad cutout image no matter how the threshold value is changed during the binarization process, the detection unit 50 determines that the reagent pad 20 has fallen off the test paper 18.

[0082] As described above, in this embodiment, when determining whether or not a rectangular shape exists in the cut-out image of the reagent pad, the threshold value for the binarization process is not a fixed value, but is varied between 255 and 0. Therefore, according to this embodiment, even if the reagent pad 20 has various original colors and changes to various colors due to a color reaction, it is possible to reliably determine whether the reagent pad 20 has fallen off the test paper 18 or is properly attached.

[0083] Next, the computer 28 executes the dropping abnormality detection process shown in Fig. 25. In step S402, the computer 28 functions as the detection unit 50 and detects dropping abnormalities on each reagent pad 20 based on the cropped image 52 generated by the image processing unit 46. The computer 28 then determines whether or not a dropping abnormality exists. For example, if the upper surface 56 of the reagent pad 20 is sufficiently soaked with sample liquid, the portion of the upper surface 56 including the side 60 will have changed color to the color after the sample liquid has soaked in. Therefore, a dropping abnormality can be detected based on whether or not a color change has occurred in the portion of the upper surface 56 including the side 60 of the upper surface 56.

[0084] Furthermore, when the upper surface 56 of the reagent pad 20 is sufficiently saturated with sample liquid, the upper surface 56 may deform, appearing slightly wavy. As shown in Figure 23, this deformation of the upper surface 56 also occurs on the edge 60. In contrast, when the reagent pad 20 is not sufficiently saturated with sample liquid, the edge 60 is not deformed by the sample liquid, and the edge 60 maintains its shape before deposition, i.e., a straight line. In other words, the degree of deformation of the edge 60 from a straight line can be used as an indicator for determining whether or not a deposition abnormality has occurred on the reagent pad 20.

[0085] The degree of deformation can be quantified, for example, by integrating the length by which each position of the actual side 60 is displaced from a straight line connecting one end of the side 60 to the other end.

[0086] In this case, the disclosed technology captures an image of not only the top surface 56 of the reagent pad 20 but also the side surface 58A that is continuous with the top surface 56. Therefore, deformation of the edge 60A that is the boundary between the top surface 56 and the side surface 58A is more easily detected than when the side surface 58A is not imaged. In particular, deformation of the edge 60 is more likely to appear in the height direction (direction of arrow H) of the reagent pad 20. Therefore, by imaging the edge 60 with the imaging device 44, the degree of deformation of the edge 60 can be easily detected.

[0087] As described above, the detection unit 50 generates an abnormal dispensing model by machine learning using as learning data images of the reagent pad 20 in a normal state where the sample liquid is dispensed and images of the reagent pad 20 in an abnormal state where the sample liquid is dispensed. The detection unit 50 inputs the newly generated cropped image 52 into this abnormal dispensing model and detects abnormal dispensing of the sample liquid on the reagent pad 20. The detection unit 50 then performs machine learning on the deformation of the edge 60 as learning data. Therefore, it is possible to detect abnormal dispensing more accurately than in a configuration that does not use machine learning.

[0088] In the disclosed technology, the cut-out image 52 is generated as two images, one of a portion close to side 60A and the other of a portion close to side 60B of the reagent pad 20. Compared to when a cut-out image is generated that includes the entire upper surface 56 of the reagent pad 20, the amount of data to be detected by the detection unit 50 is reduced, and the time required for detection is also shorter.

[0089] If the computer 28 detects a dropping abnormality in step S402, the process proceeds to step S404. In step S404, the computer 28 displays a message indicating the dropping abnormality on the operation panel 40. In this case, the image processing unit 46 associates the multiple reagent pads 20 on one test strip 18 with the captured images of each reagent pad 20, so that the reagent pad 20 with the dropping abnormality can be identified and displayed on the operation panel 40. On the other hand, if no dropping abnormality is detected in step S406, the computer 28 terminates the dropping abnormality detection process. After completing the dropping abnormality detection process, the process proceeds to step S106 shown in FIG. 8. In step S106, the computer 28 controls the analysis unit 26 to perform the analysis process, i.e., analyze the components of the sample liquid. The analysis process is performed, for example, by observing the color development state in the extracted image 52. Then, the entire operation of the analyzer 12 is terminated.

[0090] As described above, in the analyzer 12 of this embodiment, the imaging device 44 captures an image of the top surface 56 of the reagent pad 20 (top surface image) and an image of the side surface 58 connected to the top surface 56 (side surface image). This image also includes the edge 60 that is the boundary between the top surface 56 and the side surface 58. The detection unit 50 can then detect any deposition abnormalities on the reagent pad 20 from the captured top surface image and side surface image. For example, if only one of the top surface 56 and the side surface 58 is imaged, it may be difficult to detect deposition abnormalities. However, the analyzer 12 of this embodiment can detect deposition abnormalities with high accuracy.

[0091] In the above analysis process, for example, the processes from step S402 onwards may be performed for each reagent pad 20. That is, the reagent pad 20 in which a dropping abnormality has occurred may be identified in step S404, and the sample liquid may be analyzed (step S106) for the reagent pad 20 in which a dropping abnormality has occurred, while the fact that a dropping abnormality has occurred may be notified for the reagent pad 20 in which a dropping abnormality has occurred (step S404).

[0092] In the above-described embodiment, the imaging device 44 captures an image of each of the reagent pads 20 on the test paper 18 once, thereby obtaining an image of the top surface 56 and the side surface 58 of each reagent pad 20. This reduces the number of times imaging is required compared to capturing separate images of the top surface 56 and the side surface 58 of each reagent pad 20. Furthermore, if the top surface 56 and the side surface 58 of each reagent pad 20 are captured separately, a process is required to associate the image of the top surface 56 with the image of the side surface 58 for each reagent pad 20. In contrast, in this embodiment, images of the top surface 56 and the side surface 58 of each reagent pad 20 are obtained by capturing an image once, eliminating the need for such association.

[0093] In the above, an image of the top surface 56 and an image of the side surface 58 are acquired for multiple reagent pads 20 of the test paper 18 in one image capture, but it is also possible to image one reagent pad 20 in one image capture. By capturing images of multiple reagent pads 20 in one image capture as described above, the number of times images are captured can be reduced.

[0094] In the above example, the image of the reagent pad 20 is captured by the imaging device 44 from directly above the test paper 18. However, as shown in Fig. 12, an image may be captured from diagonally above the test paper 18, i.e., from a position where the top surface 56 and one of the side surfaces 58C of the reagent pad 20 can be captured.

[0095] In the disclosed technology, the above-described method for detecting a drop abnormality, i.e., a method for detecting a drop abnormality from an image of the top surface 56 and an image of the side surface 58 of the reagent pad 20, may be performed on a specific reagent pad 20 among the multiple reagent pads 20 on one test paper 18. For example, if it is known in advance that it will be difficult to detect a drop abnormality for a reagent pad 20 corresponding to a specific reagent among the reagent pads 20, the drop movement detection method of the disclosed technology may be used for the reagent pad 20 of this specific reagent, and drop abnormalities may be detected for the other reagent pads 20 by another method.

[0096] The following supplementary notes are further disclosed. (Supplementary Note 1) A dispensing abnormality detection method for detecting abnormal dispensing of sample liquid onto a reagent pad using at least a portion of a top surface image of the reagent pad onto which the sample liquid is dispensed and at least a portion of a side surface image of the reagent pad contiguous to the top surface. (Supplementary Note 2) The dispensing abnormality detection method according to Supplementary Note 1, wherein the top surface image and the side surface image of the reagent pad are obtained by a single imaging. (Supplementary Note 3) The dispensing abnormality detection method according to Supplementary Note 2, wherein the top surface image and the side surface image of a plurality of reagent pads are obtained by a single imaging. (Supplementary Note 4) The dispensing abnormality detection method according to Supplementary Note 1, wherein the dispensing abnormality is detected using a dispensing abnormality trained model that has been machine-learned using as training data at least a portion of the top surface image and at least a portion of the side surface image of a plurality of the reagent pads onto which the sample liquid is dispensed. (Supplementary Note 5) An analysis method comprising: applying sample liquid to a reagent pad; detecting abnormal deposition of the sample liquid on the reagent pad using at least a portion of an upper surface image of the reagent pad and at least a portion of a side image of the reagent pad connected to the upper surface; and analyzing components of the sample liquid applied to the reagent pad where no abnormal deposition has occurred. (Supplementary Note 6) A deposition abnormality detection device comprising: an imaging unit that captures at least a portion of the upper surface image of the reagent pad on which sample liquid has been applied and at least a portion of the side image of the reagent pad connected to the upper surface; and a detection unit that detects abnormal deposition of the sample liquid on the reagent pad using the upper surface image and the side image captured by the imaging unit. (Supplementary Note 7) The deposition abnormality detection device according to Supplementary Note 6, wherein the imaging unit acquires the upper surface image and the side image of the reagent pad in a single imaging operation. (Supplementary Note 8) The dropping abnormality detection device according to Supplementary Note 7, wherein the imaging unit captures the top surface image and the side surface image of the plurality of reagent pads in a single imaging operation.(Supplementary Note 9) An analytical device comprising: an imaging unit that captures at least a portion of an image of a top surface of a reagent pad onto which a sample liquid is deposited and at least a portion of an image of a side surface of the reagent pad that is continuous with the top surface; a detection unit that detects abnormalities in the deposition of the sample on the reagent pad using the top surface image and the side surface image captured by the imaging unit; and an analysis unit that analyzes components of the sample deposited on the reagent pad.

[0097] The disclosure of Japanese Patent Application No. 2024-126170, filed on August 1, 2024, is incorporated herein by reference in its entirety. All documents, patent applications, and technical standards mentioned herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. A method for detecting abnormal dispensing of sample liquid onto a reagent pad, using at least a portion of an image of the top surface of the reagent pad onto which the sample liquid is dispensed and at least a portion of an image of the side surface of the reagent pad connected to the top surface.

2. The method for detecting an abnormal drop of reagent according to claim 1, wherein the top image and the side image of the reagent pad are acquired in a single image capture.

3. The method for detecting abnormal dropping according to claim 2, wherein the top surface image and the side surface image are obtained for a plurality of reagent pads in a single image capture.

4. A method for detecting a drop abnormality as described in claim 1, wherein the drop abnormality is detected using a drop abnormality trained model that has been machine-learned using at least a portion of the top surface image and at least a portion of the side surface image of the plurality of reagent pads onto which the sample liquid has been dropped as learning data.

5. An analytical method comprising: applying a sample liquid to a reagent pad; detecting an abnormality in the application of the sample liquid to the reagent pad using at least a portion of an image of the top surface of the reagent pad and at least a portion of an image of a side surface of the reagent pad connected to the top surface; and analyzing the components of the sample liquid applied to the reagent pad where the abnormal application is not occurring.

6. A deposition abnormality detection device comprising: an imaging unit that captures at least a portion of an image of a top surface of a reagent pad onto which a sample liquid is deposited and at least a portion of an image of a side surface of the reagent pad connected to the top surface; and a detection unit that detects abnormal deposition of the sample liquid on the reagent pad using the top surface image and the side surface image captured by the imaging unit.

7. The dropping abnormality detection device according to claim 6, wherein the imaging unit captures the top image and the side image of the reagent pad in a single imaging operation.

8. The dropping abnormality detection device according to claim 7, wherein the imaging unit captures the top surface image and the side surface image for a plurality of the reagent pads in a single imaging operation.

9. An analytical device comprising: an imaging unit that captures at least a portion of an image of a top surface of a reagent pad onto which a sample liquid is applied and at least a portion of an image of a side surface of the reagent pad connected to the top surface; a detection unit that detects abnormal application of the sample to the reagent pad using the top surface image and the side surface image taken by the imaging unit; and an analysis unit that analyzes the components of the sample applied to the reagent pad.

Images