Sample state determination device, sample state determination method and sample inspection device

Abstract

To provide inspection propriety determination of a sample container in an image.SOLUTION: A sample state determination device comprises: a processor which executes a program for image processing to an image of a target sample container; and a memory for storing a result of image processing. The processor executes processing of inputting the image, processing of detecting a boundary position from the image, processing of determining the boundary position of each inclusion by determining the type of upper and lower regions of each boundary position, and processing of determining inspection propriety of the target sample container on the basis of the boundary position of each inclusion.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a sample state determination device, a sample state determination method, and a sample inspection device, and more particularly to an image processing technique using machine learning to detect and identify specific sample contents (e.g., serum (plasma)) contained in a captured image. [Background technology]

[0002] In recent years, the importance of blood testing has increased, and the number of tests and test items in blood analysis has also increased, and there is a demand for improving test efficiency. In order to improve test efficiency, there is an increasing need to detect the exact content of serum, etc., or to determine sample containers that can be tested before testing. For example, Patent Documents 1, 2, and 3 propose techniques for detecting the content of serum, etc. In Patent Document 1, a plurality of color extraction means are used to detect the boundary position between the upper and lower parts of the serum. In Patent Document 2, saturation information is used to determine the boundary position between the serum part and other parts, and the amount of serum is calculated. In Patent Document 3, a mechanism for irradiating and receiving infrared light is moved up and down to detect the positions of the separating agent, serum, and blood clot. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 10-232228 [Patent Document 2] Japanese Patent Application Publication No. 9-133687 [Patent Document 3] JP 2005-265813 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, there are cases where air bubbles are generated on the specimen, and there is a problem that the influence of the air bubbles cannot be suppressed and an accurate boundary position of serum, etc. cannot be obtained even when using the multiple color extraction means of Patent Document 1, the saturation information of Patent Document 2, and the infrared irradiation and light receiving mechanism of Patent Document 3. Furthermore, there is a problem in that Patent Documents 1, 2, and 3 cannot determine the content of all ingredients such as serum, separating agent, and blood clot, or cannot determine whether a specimen container containing a specimen composed of a combination of these ingredients is testable.

[0005] The present invention has been made in view of the above circumstances, and provides a technique capable of detecting the boundary position of serum (plasma) or the like in an image and accurately determining which sample containers can be inspected. [Means for solving the problem]

[0006] A sample state determination device of one embodiment of the present invention includes a processor that executes a program for image processing of an image of a target sample container, and a memory for storing the results of the image processing, and the processor executes the following processes: inputting the image, detecting one or more boundary positions from the image, determining the nature (type) of the upper and lower areas of each of the one or more boundary positions to determine the boundary position of each inclusion, and determining whether the target sample container can be inspected based on the determined boundary positions of each inclusion.

[0007] Further features related to the present invention will become apparent from the description of the present specification and the accompanying drawings. Also, aspects of the present invention may be realized and realized by the elements and combinations of various elements and aspects of the following detailed description and the appended claims.

[0008] It should be understood that the descriptions in this specification are exemplary and illustrative only and are not intended to limit the scope or application of the present invention in any manner. Effect of the Invention

[0009] According to one aspect of the present invention, it is possible to detect the boundary position in an image and accurately determine which sample containers can be inspected. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing the functions of a sample state determination device according to a first embodiment. FIG. [Diagram 2] 1 is a diagram illustrating an example of the hardware configuration of a specimen state determination device according to a first embodiment. [Diagram 3] 13 is a diagram for explaining an example of the operation of the container identification unit. FIG. [Figure 4] 10A and 10B are diagrams for explaining an example of the operation of a boundary position detection unit; [Diagram 5] 11 is a diagram for explaining an example of the operation of a contained object determining unit. FIG. [Figure 6] 10A to 10C are diagrams for explaining an example of the operation of a boundary position correction unit. [Figure 7] 11 is a diagram for explaining an example of the operation of a content calculation unit. FIG. [Figure 8] 11 is a diagram for explaining an example of the operation of a contained object determining unit. FIG. [Figure 9] 13 is a diagram for explaining an example of upper and lower region discrimination of each boundary of the inclusion determining unit. FIG. [Figure 10] 11A and 11B are diagrams for explaining an example of boundary identification by the boundary position correction unit; [Figure 11] 11A and 11B are diagrams for explaining an example of bubble identification by the boundary position correction unit. [Figure 12] 11 is a diagram for explaining an example of the operation of a content calculation unit. FIG. [Figure 13] 11 is a diagram for explaining an example of the operation of the sample state determining unit. FIG. [Figure 14] 11A and 11B are diagrams for explaining an example of display of a determination result, etc., of an output unit. [Figure 15] 4 is a flowchart for explaining the overall operation of the sample state determination device 1 according to the first embodiment. [Figure 16]FIG. 11 is a diagram showing a schematic configuration of a specimen testing apparatus according to a second embodiment, which is equipped with a sample state determination device according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The embodiments of the present specification provide a sample state determination device and a sample state determination method that detect the boundary positions of multiple inclusions, determine the type (properties) of the inclusions around the boundary positions, and further calculate the content while correcting the boundary positions, and determine the presence or absence of each inclusion (e.g., serum (plasma), separating agent, blood clots, urine, etc.) in the sample in the container from the determination results of the multiple boundary positions and the inclusions, thereby enabling highly accurate determination of the content of each inclusion (e.g., serum (plasma), separating agent, blood clots, urine, etc.) and determination of whether the target sample container can be inspected.

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, functionally identical elements may be indicated by the same numerals. Note that the accompanying drawings show specific embodiments and implementation examples according to the principles of the present invention, but these are for understanding the present invention and are by no means used to interpret the present invention in a limiting manner.

[0013] In the present embodiment, the present invention has been described in sufficient detail for those skilled in the art to practice the present invention, but it should be understood that other implementations and forms are possible, and that changes in configuration and structure and substitutions of various elements are possible without departing from the scope and spirit of the technical ideas of the present invention. Therefore, the following description should not be interpreted as being limited thereto.

[0014] Furthermore, embodiments of the present invention may be implemented in software running on a general purpose computer, in dedicated hardware, or in a combination of software and hardware, as described below.

[0015] In the following, each process in an embodiment of the present invention will be described with "each processing unit as a program (e.g., a content determination unit, etc.)" as the subject (acting entity); however, since a program is executed by a processor (CPU, etc.) to perform the defined processes using a memory and a communication port (communication control device), the description may also be made with the processor as the subject.

[0016] The sample state determination device according to an embodiment of the present specification may include a function for detecting a boundary position in a container to determine the contents, and a function for determining the state of the sample container. The sample container is a combination of a container and a sample contained in the container. The sample state determination device may further include a function for correcting the boundary position. In an embodiment of the present specification, even if air bubbles or the like are present near the boundary position, the device detects the boundary position, determines the contents and air bubbles around the boundary position, and corrects the boundary position by analyzing in detail the accuracy of the boundary around the boundary position. This makes it possible to realize a highly accurate determination of the boundary position and content of serum (plasma) and the like in the image. In addition, by determining the presence or absence of each content (serum (plasma), separating agent, blood clot, urine, etc.) in the container from the determination results of multiple boundary positions and contents, it becomes possible to determine whether the target sample container can be inspected.

[0017] (1) First embodiment <Functional configuration of the sample state determination device> FIG. 1 is a block diagram showing the functional configuration of a sample state determination device according to an embodiment. The sample state determination device 1 has an input unit 10, a container identification unit 11, a boundary position detection unit 12, a content determination unit 13, a boundary position correction unit 14, a content calculation unit 15, a specimen state determination unit 16, a control unit 19, and a memory 90. However, in the case where container information is provided in advance in the sample state determination device, the container identification unit 11 is not a necessary component. The sample state determination device 1 may be implemented in a specimen testing device, as described later (second embodiment). In addition, the embodiment described below can be applied to samples other than those collected from humans.

[0018] In the sample state determination device 1, the container identification unit 11, boundary position detection unit 12, content determination unit 13, boundary position correction unit 14, content amount calculation unit 15, and specimen state determination unit 16 may be realized by a program or may be realized as a module.

[0019] Image data is input to the input unit 10. For example, the input unit 10 may acquire encoded still image data in JPG, JPEG2000, PNG, BMP format, etc., captured at a predetermined time interval by an imaging means such as a camera built into the image acquisition device, and use the image as an input image. The input unit 10 may also extract still image data of frames at a predetermined interval from moving image data in a format such as MotionJPEG, MPEG, H.264, HD / SDI, etc., and use the image as an input image. The input unit 10 may also use an image acquired by an imaging means via a bus, a network, etc., as an input image. The input unit 10 may also use an image already stored in a removable recording medium as an input image. The image input from the input unit 10 is output to the container identification unit 11, the boundary position detection unit 12, the content determination unit 13, and the boundary position correction unit 14 via the memory 90.

[0020] The container identification unit 11 detects and identifies a container from an input image using a network that detects the entire container or a part of the container (for example, a container detector such as YOLO). The container identification unit 11 also stores the calculated container information in the memory 90.

[0021] The boundary position detection unit 12 detects each boundary position and the boundary of the barcode from the input image using a network (for example, a boundary position detector such as YOLO) that detects each boundary position and the boundary of the barcode. The boundary positions are interfaces between inclusions or between inclusions and other substances. The barcode indicates, for example, information on the specimen or the patient from whom the specimen was collected. The boundary position detection unit 12 stores information such as the calculated boundary positions and the boundaries of the barcode in the memory 90.

[0022] The inclusion determination unit 13 calculates the identification results of each boundary position and the upper and lower regions of the barcode using a classifier consisting of a network (e.g., a Convolutional Neural Network) that identifies inclusions (e.g., inclusions such as serum (plasma), separating agent, blood clots, urine, etc.) and non-inclusions (e.g., background, etc.) in each boundary position and the upper and lower regions of the barcode detected by the boundary position detection unit 12. The inclusion determination unit 13 also stores information such as the calculated identification results of each boundary position and the upper and lower regions of the barcode in memory 90.

[0023] The boundary position correction unit 14 uses a classifier configured with a network (e.g., Convolutional Neural Network, etc.) that identifies each boundary position in detail to identify the surroundings of each boundary position detected by the boundary position detection unit 12 in detail, detects the accurate boundary position, and corrects the boundary position. Also, using a classifier configured with a network (e.g., Convolutional Neural Network, etc.) that identifies the presence or absence of bubbles around the boundary position, it identifies the presence or absence of bubbles, and if bubbles are present, corrects the boundary position to the bottom end of the bubbles. Also, the boundary position correction unit 14 stores information on each corrected boundary position, etc. in the memory 90.

[0024] The content calculation unit 15 calculates the content of the target contents in the container (for example, serum (plasma), separating agent, blood clot, urine, etc.) from each boundary position corrected by the boundary position correction unit 14, and the container information or the container information identified by the container identification unit 11. In addition, the content calculation unit 15 stores information such as the calculated content of each content in the container in the memory 90.

[0025] The specimen status determination unit 16 determines whether or not the target specimen container (container containing a specimen) can be inspected, using the content calculated by the content calculation unit 15 and / or the identification results of each region obtained by the content determination unit 13. The specimen status determination unit 16 also stores information such as the determination result of whether or not the target container can be inspected in the memory 90.

[0026] The control unit 19 is realized by a processor and is connected to each element in the sample state determination device 1. The operation of each element of the sample state determination device 1 is performed by the autonomous operation of each of the above-mentioned components or by instructions from the control unit 19.

[0027] As described above, in the sample state determination device 1 of this embodiment, the container identification unit 11 identifies the type of container. The boundary position detection unit 12 detects each boundary position of the container and the boundary position of the barcode. The content determination unit 13 calculates the identification results of each boundary position of the container and the upper and lower areas of the barcode. The boundary position correction unit 14 identifies the periphery of each boundary position in detail and corrects the boundary position, and the content calculation unit 15 calculates the content of the target content in the container. The specimen state determination unit 16 determines the state of the specimen in the container using the calculated content and the identification results of the surrounding areas of each boundary position, thereby detecting the content of the content in the container with high accuracy from the image and determining whether the target container can be inspected.

[0028] <Hardware configuration of the classifier generation device> FIG. 2 is a diagram showing an example of the hardware configuration of a specimen state determination device 1 according to an embodiment of the present invention.

[0029] The sample state determination device 1 has a CPU (processor) 201 that executes various programs, a memory 202 that stores various programs, a storage device (corresponding to memory 90) 203 that stores various data, an output device 204 for outputting each boundary position, each content amount, and determination results of whether the target container can be inspected, and an input device 205 for inputting user instructions, images, etc., all of which are interconnected by a bus 207.

[0030] The CPU 201 reads and executes various programs from the memory 202 as necessary. The memory 202 stores, as programs, an input unit 10, a container identification unit 11, a boundary position detection unit 12, a content determination unit 13, a boundary position correction unit 14, a content calculation unit 15, and a specimen state determination unit 16.

[0031] The memory device 203 stores the input image, information on the type of container identified by the container identification unit 11, information on each boundary position of the container and the boundary position of the barcode detected by the boundary position detection unit 12, the identification results of each boundary position of the container and the upper and lower areas of the barcode determined by the content determination unit 13, information on the boundary positions corrected by the boundary position correction unit 14, information on the content of the target contents in the container calculated by the content calculation unit 15, and the determination result of whether the target container can be inspected determined by the specimen status determination unit 16.

[0032] The output device 204 is composed of devices such as a display, a printer, a speaker, etc. For example, the output device 204 displays data generated by the container identification unit 11, the boundary position detection unit 12, the content determination unit 13, the boundary position correction unit 14, the content calculation unit 15, and the specimen state determination unit 16 on a display screen.

[0033] The input device 205 is composed of devices such as a keyboard, a mouse, a microphone, etc. User instructions (including the decision to input an input image) are input to the specimen state determination device 1 via the input device 205.

[0034] The sample status determination device of the present invention identifies the container type, detects each boundary position of the container and the boundary position of the barcode, calculates the identification results of each boundary position of the container and the upper and lower areas of the barcode, identifies the area around each boundary position in detail to correct the boundary positions, calculates the content of the target contents in the container, and further determines the state of the specimen in the container using the calculated content and the identification results of the area around each boundary position, thereby accurately detecting the content of the contents in the container from the image and accurately determining whether the target container can be inspected.

[0035] <Configuration and operation of each part> The configuration and operation of each element will be described in detail below.

[0036] (i) Container identification section 11 An example of a container is shown in Figure 3. The container identification unit 11 detects and identifies a container from an input image using a network that detects the entire container or part of a container (for example, a container detector such as YOLO) to determine the container type ct and the position of the container top 31, and also calculates the position in the image of the container bottom 36 shown in Figure 3 from the calculated container type ct and the container information in the memory 90. The container identification unit 11 saves information on the identified container type and the positions in the image of the container top and container bottom in the memory 90.

[0037] (ii) Boundary position detection unit 12 The boundary position detection unit 12 detects the boundary positions (interfaces or boundaries) of each of the contents in the container and the boundary of the barcode. FIG. 3 shows a barcode 32, serum 33, a separating agent 34, and a blood clot 35. FIG. 4 shows an example of boundary position detection. The boundary position detection unit 12 detects each boundary position and the boundary of the barcode 32 from the input image as shown in FIG. 4 using a network (for example, a boundary position detector such as YOLO) that detects each boundary position (such as the boundary of the contents) in the container and the boundary of the barcode. In the example shown in FIG. 4, a boundary 41 between serum (plasma) 33 and the background, a boundary 42 between the serum 33 and the separating agent 34, and a boundary 43 between the blood clot and the separating agent are detected. In addition, an upper boundary 44 and a lower boundary 45 of the barcode 32 are detected.

[0038] When the boundary position detection unit 12 cannot detect any boundaries such as boundary positions, it sets the boundary position detection result lr to 0 and stores the boundary position detection result lr in memory. When the boundary position detection unit 12 detects one or more boundaries such as boundary positions, it sets the number of detected boundaries to the boundary position detection result lr. The boundary position detection unit 12 saves the calculated boundary position detection result lr and information on each detected boundary position in the memory 90.

[0039] (iii) Content determination section 13 5, the content determination unit 13 calculates the identification results of each boundary position and the upper and lower regions of the barcode for each boundary position and the upper and lower regions of the barcode detected by the boundary position detection unit 12 using a classifier configured with a network (e.g., Convolutional Neural Network, etc.) that identifies contents (e.g., contents such as serum (plasma), separating agent, blood clot, urine, etc.) and non-contents (e.g., background, barcode, etc.). In addition, when the containers include containers made of various materials such as transparent containers and opaque containers, the content determination unit 13 selects a classifier suitable for the container using the container type information identified by the container identification unit 11, and calculates the identification results of each boundary position and the upper and lower regions of the barcode using the selected classifier.

[0040] As an example, an example of obtaining a classification result is shown in Fig. 9. An input image A1 is input to a feature extractor A91, which outputs a feature amount FAi. A logistic regression layer 92 identifies an area in the input image A1 from the feature amount FAi. CNN in Fig. 9 stands for Convolutional Neural Network.

[0041] The inclusion determining unit 13 reads the weight w, the filter coefficient wj, and the offset values ​​b and bi of the expressions (1) and (2) from the memory 90.

[0042] For example, using the feature extractor A91, the feature amount FAi of inclusions (e.g., inclusions such as serum (plasma), separating agent, blood clots, urine, etc.) and non-inclusions (e.g., background, barcode, etc.) is calculated for each boundary position and the upper and lower regions of the barcode in the input image A1 using equation (1).

[0043] The filter coefficient wj shown in equation (1) is a coefficient obtained by machine learning or the like so as to distinguish each inclusion from the other inclusions (e.g., serum (plasma) as serum (plasma), separating agent as separating agent, blood clot as blood clot, urine as urine, etc.) and to distinguish non-inclusions from non-inclusions (e.g., background as background, barcode as barcode, etc.).

[0044] In formula (1), pj is a pixel value, bi is an offset value, m is the number of filter coefficients, and h is a nonlinear function. Using formula (1), the feature value fi of an arbitrary filter i is found by calculating the calculation results of each filter from the upper left to the lower right of each region of the target image. For example, the matrix of feature values ​​fi found by feature extractor A91 is taken as feature value FAi of input image A1.

[0045]

number

[0046] As shown in FIG. 9, using the calculated feature amount FAi (matrix f) of the feature extractor A91, the logistic regression layer 92 of the classifier calculates values ​​of the likelihood of an inclusion to be detected or an inclusion other than an inclusion to be detected (likelihood of serum (plasma), separating agent, blood clot, background, barcode, etc.) according to equation (2), and determines whether each region in the input image A1 (e.g., the upper and lower regions of the boundary position) is an inclusion to be detected (e.g., serum (plasma), separating agent, blood clot, urine, etc.) or an inclusion other than an inclusion to be detected (e.g., background, barcode, etc.).

[0047] In formula (2), w is a weight matrix, b is an offset value, g is a nonlinear function, and y is a classification result. The weight of w and the offset value of b are calculated in advance using a learning image by a known machine learning technique. For example, a convolutional neural network may be used as the machine learning technique. Furthermore, each region of the target image is classified using the weight of w and the offset value of b calculated during learning, and the classification result y is calculated.

[0048]

number

[0049] The content determination unit 13 determines what each boundary is based on each boundary position detected by the boundary position detection unit 12 and the identification result y of the upper and lower regions of the barcode boundary. For example, as shown in Fig. 8, if the identification result of the upper region is the background and the identification result of the lower region is the barcode 32, the boundary is determined to be the upper part of the barcode 44 (p1 is set to the determination result pr). Also, if the identification result of the upper region is the barcode 32 and the identification result of the lower region is any of the serum 33, blood clot 35, separating agent 34, and background, the boundary is determined to be the lower part of the barcode 45 (p2 is set to the determination result pr).

[0050] Moreover, if the identification result of the upper region is background and the identification result of the lower region is serum 33, the boundary 41 is determined to be the upper part of serum (p3 is set to the determination result pr). Moreover, if the identification result of the upper region is serum 33 and the identification result of the lower region is either separating agent 34 or blood clot 35, the boundary 42 is determined to be the lower part of serum (p4 is set to the determination result pr). Moreover, if the identification result of the upper region is separating agent 34 and the identification result of the lower region is either blood clot 35 or the bottom of the container, the boundary 43 is determined to be the lower part of the separating agent (p5 is set to the determination result pr).

[0051] Moreover, if the identification result of the upper region is either separating agent 34, serum 33, or background, and the identification result of the lower region is blood clot 35, then the boundary 43 is determined to be the upper part of the blood clot (p6 is set to the determination result pr). Moreover, if the identification result of the upper region is blood clot 35 and the identification result of the lower region is separating agent 34, then the boundary (not shown) is determined to be the lower part of the blood clot (p7 is set to the determination result pr). Moreover, if the identification result of the upper region of the container bottom is blood clot 35, then the boundary 44 is determined to be the lower part of the blood clot (p8 is set to the determination result pr).

[0052] Moreover, if the identification result of the upper region of the container bottom is serum 33, the boundary (not shown) is determined to be the lower part of the serum (p9 is set to the determination result pr). If the identification result of the upper region of the container bottom is separating agent 34, the boundary (not shown) is determined to be the lower part of the separating agent (p10 is set to the determination result pr). If the identification result of the upper region is either serum 33, background, or blood clot 35, and the identification result of the lower region is separating agent 34, the boundary 42 is determined to be the upper part of the separating agent (p11 is set to the determination result pr). For example, in the case of FIG. 5, p1, p2, p3, p4, p5, p6, p8, and p11 are set to the boundary determination result pr.

[0053] The content determining unit 13 eliminates boundary candidates that are not the boundary positions detected by the boundary position detecting unit 12 or the boundaries of barcodes.

[0054] The inclusion determining unit 13 stores the calculated identification result y and the determination result pr of each boundary in the memory 90.

[0055] (iv) Boundary position correction section 14 The boundary position correction unit 14 uses a classifier (including a feature extractor B) composed of a network (e.g., a Convolutional Neural Network, etc.) that distinguishes the boundaries of the contents (e.g., contents such as serum (plasma), a separating agent, a blood clot, and urine) in detail for the boundaries of each of the contents calculated by the content determination unit 13, and performs classification while moving a classification area 62 in a pixel unit in a search range 61 as shown in FIG. 6, and corrects the boundary position of each of the contents at the position where the value of the classification result y is the largest, according to equations (1) and (2). As an example, an example of determining the classification result is shown in FIG. 10. An input image A1 is input to a feature extractor B101, and a feature amount FBi is output. The logistic regression layer 102 identifies the boundaries in the input image A1 from the feature amount FBi. The CNN in FIG. 10 stands for Convolutional Neural Network.

[0056] The boundary position correction unit 14 reads the weights w, filter coefficients wj, and offset values ​​b and bi of equations (1) and (2) for identifying the boundaries of each inclusion from the memory 90, and calculates an identification result y of the likelihood of a boundary. For example, the serum boundary is identified by moving an identification area 62 up and down in pixel units within a search range 61 around the serum boundary, and the serum boundary position is corrected to the position where the identification result y is largest.

[0057] Furthermore, the boundary position correction unit 14 may use a classifier (including a feature extractor C) configured with a network (e.g., a Convolutional Neural Network, etc.) for identifying bubbles 63 for the boundary of the inclusion determined by the inclusion determination unit 13 to be the upper serum portion to identify the presence or absence of bubbles in the identification area 62 according to formulas (1) and (2), and may correct the boundary position of the upper serum portion to the position of the lowest end of the bubbles if the presence of bubbles is identified. As an example, an example of obtaining the identification result is shown in FIG. 11. An input image A1 is input to a feature extractor C111, and a feature amount FCi is output. A logistic regression layer 112 determines the presence or absence of bubbles in the input image A1 from the feature amount FCi. The CNN in FIG. 11 stands for Convolutional Neural Network.

[0058] The boundary position correction unit 14 reads the weight w, the filter coefficient wj, and the offset values ​​b and bi of equations (1) and (2) for identifying the presence or absence of an air bubble from the memory 90, and calculates the identification result y of the air bubble 63.

[0059] The boundary position correction unit 14 stores in the memory 90 the corrected boundary positions, the barcode boundary positions, and the identification results thereof and the identification results of air bubbles.

[0060] (v) Content calculation section 15 The content calculation unit 15 calculates the content lv of each inclusion as shown in Fig. 7 using the boundary positions of the upper and lower parts of each inclusion corrected by the boundary position correction unit 14. For example, from the information on the container type identified by the container identification unit 11, the content information of the height from the container bottom and the area of ​​the inside of the container at that height is read from the memory 90, and the content amounts from the height of the upper part of the serum to the height of the lower part of the serum are added together to calculate the content lv of the serum 33. The content calculation unit 15 also calculates the distance ds from the position of the upper part of the container calculated by the container identification unit 11 to the boundary of the upper part of the serum.

[0061] As shown in FIG. 12, depending on the viewpoint of the camera lens during shooting, the pixel information may become more distorted in the vertical direction. Therefore, the content calculation unit 15 may calculate the serum content lv after correcting the pixel information (x1, y1) at the boundary position between the upper serum and the lower serum to (x3, y3) using, for example, formulas (3) to (8). However, x and y in formula (3) represent points including distortion on the camera coordinates, x1 and y1 represent points including distortion on the image coordinates, cx and cy represent the image center, fx and fy represent the focal length, k1 and k2 in formula (5) represent radial distortion coefficients, p1 and p2 in formula (6) represent circumferential distortion coefficients, x2 and y2 in formula (7) represent points corrected for distortion on the camera coordinates, and x3 and y3 in formula (8) represent points corrected for distortion on the image coordinates.

[0062]

number

[0063]

number

[0064]

number

[0065]

number

[0066]

number

[0067]

number

[0068] The content calculation unit 15 stores the calculated content lv and distance ds in the memory 90.

[0069] (vi) specimen status determination unit 16 The specimen status determination unit 16 determines whether or not the target container can be inspected, using the boundary position detection result lr calculated by the boundary position detection unit 12, the determination result pr of each boundary calculated by the content determination unit 13, and the content lv calculated by the content calculation unit 15. A flowchart of the specimen status determination unit 16 is shown in FIG.

[0070] For example, if the boundary position detection result lr is 0, the specimen state determination unit 16 determines that the target container is an empty container, sets the test feasibility determination result to 0 (test not possible) and the test feasibility determination reason to "empty container." Also, if the content lv is less than the threshold TH, the test feasibility determination result is set to 0 (test not possible) and the test feasibility determination reason is set to "insufficient content."

[0071] If the boundary judgment result pr does not include p3 and p4, or p3 and p9, the test feasibility judgment result is set to 0 (test not possible) and the test feasibility judgment reason is set to "no serum." If the boundary position detection result lr is 1 or more, the content lv is equal to or greater than the threshold TH, and the boundary judgment result pr includes p3 and p4, or p3 and p9, the test feasibility judgment result iajr is set to 1 (test possible) and the test feasibility judgment reason is set to "serum present."

[0072] <Processing Procedure of the Sample State Determination Unit 16> (i) Step 1301 The sample state determining section 16 determines whether or not the boundary position detection result lr is 0. If lr is other than 0, the process proceeds to step 1302. If lr is 0, the process proceeds to step 1307.

[0073] (ii) Step 1302 The specimen state determination unit 16 determines whether the content lv calculated by the content calculation unit 15 is equal to or greater than a threshold TH (e.g., 4 μl). If the content lv is equal to or greater than the threshold TH, the process proceeds to step 1303. If the content lv is less than the threshold TH, the process proceeds to step 1307.

[0074] (iii) Step 1303 The sample state determination unit 16 determines whether or not the determination result pr of each boundary calculated by the content determination unit 13 includes p3. If the determination result pr of each boundary includes p3, the process proceeds to step 1304. If the determination result pr of each boundary does not include p3, the process proceeds to step 1307.

[0075] (iv) Step 1304 The sample state determination unit 16 determines whether or not the determination result pr of each boundary calculated by the content determination unit 13 includes p4. If the determination result pr of each boundary includes p4, the process proceeds to step 1305. If the determination result pr of each boundary does not include p4, the process proceeds to step 1306.

[0076] (v) Step 1305 The sample status determination section 16 sets the test feasibility determination result iajr to 1 (test possible).

[0077] (vi) Step 1306 The sample state determination unit 16 determines whether or not the determination result pr of each boundary calculated by the content determination unit 13 includes p9. If the determination result pr of each boundary includes p9, the process proceeds to step 1305. If the determination result pr of each boundary does not include p9, the process proceeds to step 1307.

[0078] (vii) Step 1307 The sample status determination section 16 sets the test feasibility determination result iajr to 0 (test not possible).

[0079] The sample status determination unit 16 stores the calculated test feasibility determination result iajr and the reason for the test feasibility determination in the memory 90.

[0080] (vii) Output section 17 As an example, the output unit 17 displays, in a GUI (graphical user interface) as shown in Figure 14, the container type ct calculated by the container identification unit 11, the lines of each boundary detected by the boundary position detection unit 12, the identification results of the upper and lower areas of each boundary and the boundary determination results determined by the content determination unit 13, the lines of each boundary corrected by the boundary position correction unit 14 (e.g., boundary (corrected)), the content lv and distance ds of each content calculated by the content calculation unit 15, and the determination result and reason for the determination as to whether the target container can be inspected obtained by the specimen status determination unit 16.

[0081] Fig. 14 is a diagram showing an example in which the target container contains serum, a separating agent, and a blood clot, and the boundary position of the serum has been corrected. Fig. 14 shows an example in which the testability of the target container is "Yes", the reason for the testability judgment is "serum present", the container type is "A", the content (serum) is 20 μl, the boundary judgment results are p1, p2, p3, p4, p5, p6, p8, p11, the air bubble identification result is "present", the distance from the top of the container is 18 mm, the boundary line of the top of the serum and the corrected boundary line of the top of the serum, the boundary line of the bottom of the serum (boundary line of the top of the separating agent), the boundary line of the bottom of the separating agent (boundary line of the top of the blood clot), and the boundary line of the bottom of the container are displayed.

[0082] The output section 17 is not an essential component of the sample state determination device 1, and if the sample testing device includes an output section, the sample state determination device 1 does not need to have the output section 17.

[0083] <Processing procedure of the sample state determination device> 15 is a flowchart for explaining the operation of the sample state determination device 1 according to the embodiment of the present invention. In the following, each processing unit (input unit 10, content determination unit 13, etc.) will be described as the main operating unit, but it may be interpreted as the CPU 201 being the main operating unit, and the CPU 201 executing each processing unit as a program.

[0084] (i) Step 1501 The input unit 10 receives a target image and outputs the input image to the container identification unit 11.

[0085] (ii) Step 1502 The container identification unit 11 uses a container detector to detect a container from the input image, and further identifies the container type ct, detects the positions of the top and bottom of the container, and stores the obtained information in memory 90.

[0086] (iii) Step 1503 The boundary position detection unit 12 uses a boundary position detector to detect each boundary position within the container (such as the boundaries of the contents) and the boundaries of the barcode, and stores the obtained information in the memory 90.

[0087] (iv) Step 1504 The contained material determination unit 13 calculates the identification results pr of each boundary position and the upper and lower regions of the barcode boundary and the determination results pr of each boundary using equations (1) and (2), and stores the obtained information in memory 90.

[0088] (v) Step 1505 The boundary position correction unit 14 obtains the corrected boundary positions, the barcode boundary positions, and their identification results and air bubble identification results using equations (1) and (2), and stores the obtained information in the memory 90.

[0089] (vi) Step 1506 The content calculation unit 15 uses information on the position of the top of the container and the boundary position between the top and bottom of each inclusion to determine the content lv of each inclusion and the distance ds from the top of the container to the top of the serum, and stores the determined information in memory 90. In addition, the content calculation unit 15 corrects the information on each boundary position using equations (3) to (8), determines the content lv of each inclusion and the distance ds from the top of the container to the top of the serum from the corrected information on each boundary position, and stores each content lv and distance ds in memory 90.

[0090] (vii) Step 1507 The sample status determination unit 16 determines whether or not the target container can be inspected using the boundary position detection result lr calculated by the boundary position detection unit 12, the judgment result pr of each boundary calculated by the content determination unit 13, and the content lv calculated by the content calculation unit 15, and stores the calculated inspection feasibility determination result iajr in memory 90.

[0091] According to the first embodiment, the container type is identified, each boundary position of the container and the boundary position of the barcode are detected, identification results of each boundary position of the container and the upper and lower areas of the barcode are calculated, the area around each boundary position is identified in detail to correct the boundary positions, the amount of the target contents in the container is calculated, and the state of the sample in the container is determined using the calculated amount and the identification results of the area around each boundary position, making it possible to accurately detect the amount of the contents in the container from the image while accurately determining whether the target container can be inspected.

[0092] In addition, by detecting the amount of material contained in a container with high accuracy from an image, it is possible to reduce the amount of sample taken from a patient to a very small amount.

[0093] (2) Second embodiment 16 is a functional block diagram showing the configuration of a sample testing apparatus 1600 according to the second embodiment of the present invention. The sample testing apparatus 1600 has the sample state determining device 1 according to the first embodiment, a sample aspirating device 1601, and a display device 1602.

[0094] The sample suction device 1601 has a device that controls a sample sampling mechanism equipped with, for example, a suction nozzle, and calculates the analysis results of test items related to the sample (for example, HbA1c, TP, AST, etc. related to biochemistry and immunoanalysis) using the aspirated sample and reagent.

[0095] The specimen aspirating device 1601 receives test feasibility information (test feasibility determination result iajr, etc.) from the specimen state determination device 1, and if the test feasibility determination result iajr is 1 (test possible), it controls the specimen sampling mechanism to aspirate a portion of the specimen and calculates the analysis results for each test item using a reagent. The display device 1602 displays the analysis results on the display screen. On the other hand, if the test feasibility determination result iajr is 0 (test not possible), the specimen sampling mechanism does not aspirate the specimen, and the display device 1602 displays "not possible" for the test feasibility on the display screen. The specimen aspirating device 1601 also receives information on the distance ds from the top of the container from the specimen state determination device 1, and controls the aspirating position of the aspirating nozzle of the specimen sampling mechanism.

[0096] The specimen aspirating device 1601 may be a biochemical analyzer, an immunoanalyzer, or the like equipped with a specimen sampling mechanism.

[0097] According to the second embodiment, it is possible to provide a specimen testing device that can perform testing efficiently and also minimize the amount of specimen collected from patients by using the test feasibility information determined by the sample state determination device to accurately control whether or not the specimen can be aspirated by the specimen aspirating device, analyzes each test item related to the specimen using the aspirated specimen and reagent, and displays the analysis results of each test item on the display device.

[0098] The embodiments described above can be modified as follows: In the inclusion determination unit 13 and the boundary position correction unit 14, a plurality of feature amounts are obtained using filters by machine learning, but other feature amounts such as HOG may be used with the same effect.

[0099] In addition, in the container identification unit 11 and the boundary position detection unit 12, YOLO was used as a container detector and a boundary position detector to detect containers and boundary positions, but SSD (Single Shot MultiBox Detector), R-CNN, Fast R-CNN, Faster R-CNN, etc. may also be used with the same effect.

[0100] The present invention can also be realized by software program code that realizes the functions of the embodiments. In this case, a storage medium on which the program code is recorded is provided to a system or device, and a computer (or a CPU or MPU) of the system or device reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-mentioned embodiments, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media for supplying such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, and ROMs.

[0101] Also, an operating system (OS) running on a computer may perform all or a part of the actual processing based on the instructions of the program code, and the functions of the above-mentioned embodiments may be realized by the processing. Furthermore, after the program code read from a storage medium is written into a memory on a computer, a CPU of the computer may perform all or a part of the actual processing based on the instructions of the program code, and the functions of the above-mentioned embodiments may be realized by the processing.

[0102] Furthermore, the program code of the software that realizes the functions of the embodiments may be distributed over a network and stored in a storage means such as a hard disk or memory of the system or device, or in a storage medium such as a CD-RW or CD-R, so that when used, the computer (or CPU or MPU) of the system or device reads out and executes the program code stored in the storage means or storage medium.

[0103] Finally, the processes and techniques described herein are not inherently related to any particular apparatus, but may be implemented by any suitable combination of components. Furthermore, various types of general-purpose devices may be used in accordance with the methods described herein. It may be beneficial to build a dedicated apparatus to perform the steps of the methods described herein. Various inventions may also be formed by suitable combinations of multiple components disclosed in the embodiments. For example, some components may be omitted from all components shown in the embodiments. Furthermore, components across different embodiments may be combined as appropriate. The present invention has been described with reference to specific examples, which are in all respects illustrative rather than limiting. Those skilled in the art will recognize that there are numerous combinations of hardware, software, and firmware suitable for implementing the present invention. For example, the described software may be implemented in a wide range of program or script languages, such as assembler, C / C++, perl, Shell, PHP, Java, and the like.

[0104] Furthermore, in the above-described embodiment, the control lines and information lines are those that are considered necessary for the explanation, and not all of the control lines and information lines in the product are necessarily shown. All of the components may be connected to each other.

[0105] In addition, other implementations of the invention will be apparent to those of ordinary skill in the art from consideration of the specification and embodiments of the invention disclosed herein. Various aspects and / or components of the described embodiments may be used alone or in any combination. [Explanation of symbols]

[0106] 1. Sample condition determination device 10 Input section 11 Container identification section 12 Boundary position detection section 13...Inclusion determination section 14...Boundary position correction section 15...Content calculation section 16... Sample condition determination section 17 Output section 19. Control section 1600: Sample testing equipment

Claims

1. a processor that executes a program for performing image processing on an image of a target sample container; a memory for storing a result of the image processing; The processor, A process of inputting the image; A process of detecting a boundary position from the image; A process of determining the types of upper and lower regions of each of the detected boundary positions to determine the boundary positions of each inclusion; A process of determining whether or not the target sample container can be inspected based on the determined boundary positions of each of the inclusions; A sample state determination device that performs the above.

2. 2. The sample state determination device according to claim 1, The processor executes a process of correcting each of the detected boundary positions based on the determined type of the upper and lower regions of each of the detected boundary positions.

3. 2. The sample state determination device according to claim 1, The processor, A process of identifying a container from the image and calculating container information; selecting a detector in response to the container information; A process of determining the types of upper and lower regions of each boundary position detected by the selected detector and determining the boundary position of each inclusion is executed; a sample state determination device that determines whether the target sample container can be inspected based on the determined boundary positions of each of the inclusions.

4. 4. The sample state determination device according to claim 1, 2 or 3, The processor, Calculating the content of each inclusion based on the determined boundary position of each inclusion; The sample state determination device determines whether the target sample container can be inspected based on the calculated content.

5. 3. The sample state determination device according to claim 2, The processor, Executing a process of identifying a container from the image and calculating container information; Executing a process of calculating a content based on the container information and the determined boundary positions of each of the contained substances; The sample state determination device determines whether the target sample container can be inspected based on the calculated content.

6. 1. A sample state determination method for determining a state of a target sample container in an image, comprising: A processor that executes a program for image processing of an input image, Input the image of the target sample container, Detecting a boundary position from the image; determining the types of upper and lower regions of each of the detected boundary positions to determine the boundary positions of each inclusion; A sample state determination method, comprising: determining whether or not the target sample container can be inspected based on the determined boundary positions of each of the inclusions.

7. 7. A method for determining a sample state according to claim 6, comprising: The method for determining a state of a specimen, wherein the processor corrects each of the detected boundary positions based on the determined type of the region above and below each of the detected boundary positions.

8. 7. A method for determining a sample state according to claim 6, comprising: The processor, Identifying a container from the image and calculating container information; Selecting a detector according to the container information; determining the types of upper and lower regions of each boundary position detected by the selected detector to determine the boundary position of each inclusion; A sample state determination method, comprising: determining whether or not the target sample container can be inspected based on the determined boundary positions of each of the inclusions.

9. A method for determining a sample state according to claim 6, 7 or 8, comprising the steps of: The processor, Calculating the content based on the determined boundary positions of each of the inclusions; The method for determining a sample state includes determining whether or not the target sample container can be inspected based on the calculated content.

10. The sample state determination method according to claim 7, The processor, Identifying a container from the image and calculating container information; Calculating the content based on the container information and the determined boundary positions of each of the contained substances; The method for determining a sample state includes determining whether or not the target sample container can be inspected based on the calculated content.

11. The sample state determination device according to claim 1, 2 or 3, a sample suction device for controlling whether or not a sample can be suctioned and analyzing test items of the sample; A display device for displaying the analysis results, the sample aspirating device controls whether or not the sample in the target sample container can be aspirated using information on whether or not the test can be performed determined by the sample state determining device, and analyzes test items related to the sample using the aspirated sample and a reagent; The display device displays the analysis results of the test items.

Images