Sample state determination device, sample state determination method, and sample analysis system

Abstract

To accurately determine the state of a sample.SOLUTION: A sample state determination device includes: an image input unit which receives an image obtained by capturing a container in which a sample is stored from the side surface; a region detection unit which identifies a serum or plasma region from the image; a state determination aptitude information calculation unit which calculates state determination aptitude information which is information for evaluating aptitude for a state determination of the sample, for each partial region of the serum or plasma region identified by the region detection unit; a state determination region selection unit which selects the partial region suitable for the state determination of the sample, based on the state determination aptitude information; a state determination unit which determines the state of the sample by using the partial region selected by the state determination region selection unit; and an output unit which outputs the state determination result of the sample determined by the state determination unit and / or location information in the image of the partial region selected by the state determination region selection unit.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a technique for determining the state of a sample, which is applicable to an automated blood analyzer such as a biochemical analyzer or an automated immunoanalyzer. [Background technology]

[0002] Automated blood analyzers, such as biochemical analyzers and immunoanalyzers, perform analysis based on the color or luminescence that occurs in the reaction solution resulting from the reaction of the sample with a reagent. If the sample contains lipemia (L), hemolysis (H), jaundice, etc. (I), the accuracy of the analysis results may be affected.

[0003] The task of visually selecting specimens containing such samples places a heavy burden on users, and there is a high demand for automation.For example, Patent Document 1 discloses a method for determining the LHI of a specimen by acquiring multiple images of a specimen container containing the serum or plasma portion of the specimen, determining the lipemic, hemolytic, and icteric nature of the serum or plasma region for each pixel or each patch from the multiple images using a convolutional neural network, and integrating the results within the serum or plasma region by majority vote or the like. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Special Publication No. 2020-519853 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the containers in which samples are stored and the conditions inside the samples are diverse, and the color and brightness of the images may not be uniform even within the serum or plasma region of the same specimen. For example, structures containing a separating agent contained in a test tube, labels or stickers attached to the test tube, and printing on the test tube may affect the color and brightness of the serum or plasma images or may mask the serum or plasma region.

[0006] Furthermore, in the case of serum or plasma, for example, when lighting is applied to the sample from above, the luminance value of the serum or plasma region may be non-uniform, such that the upper part of the serum or plasma is bright and the lower part is dark. In this way, the serum or plasma region may contain regions that are not suitable for determining the state of the sample (LHI, etc.) due to reasons such as the color or luminance value of the serum or plasma region having changed due to factors other than the serum or plasma, being non-uniform, or the serum or plasma region being obstructed. When an image of such a serum or plasma region is used to determine the state of the sample, conventional methods may adversely affect the results of determining the state of the sample.

[0007] SUMMARY OF THE PRESENT EMBODIMENT An object of the present invention is to provide an apparatus and method for determining the state of a sample with high accuracy. [Means for solving the problem]

[0008] A sample state determination device of one embodiment of the present invention includes an image input unit that accepts an image captured from the side of a container in which a sample is stored, an area detection unit that identifies an area of ​​serum or plasma from within the image, a status determination suitability information calculation unit that calculates status determination suitability information, which is information for evaluating the suitability of the sample's state determination, for each partial area of ​​the serum or plasma area identified by the area detection unit, a status determination area selection unit that selects a partial area suitable for determining the state of the sample based on the status determination suitability information, a status determination unit that determines the state of the sample using the partial area selected by the status determination area selection unit, and an output unit that outputs the sample state determination result determined by the status determination unit and / or position information within the image of the partial area selected by the status determination area selection unit. Effect of the Invention

[0009] According to the sample state determination device of one aspect of the present invention, it is possible to determine the state of the sample with high accuracy. [Brief description of the drawings]

[0010] [Figure 1]FIG. 2 is a diagram illustrating an example of a hardware configuration of the sample state determination device according to the first embodiment. [Diagram 2] FIG. 1 is a diagram showing an example of a functional block diagram of a sample state determination device according to a first embodiment. [Figure 3A] FIG. 2 is a diagram showing an example of upper and lower end detection of serum according to Example 1. [Figure 3B] FIG. 2 is a diagram showing an example of upper and lower end detection of serum according to Example 1. [Figure 4A] FIG. 4 is a diagram illustrating an example of state determination adequacy information according to the first embodiment. [Figure 4B] FIG. 4 is a diagram illustrating an example of state determination suitability information according to the first embodiment. [Diagram 5] FIG. 4 is a flow chart illustrating an example of a method for selecting a state determination region according to the first embodiment. [Figure 6] FIG. 2 is a diagram illustrating an example of a screen display according to the first embodiment. [Figure 7] FIG. 2 is a diagram illustrating an example of a processing flow according to the first embodiment. [Figure 8] 4 is a diagram illustrating an example of a candidate map and an area for state determination according to the first embodiment. FIG. [Figure 9] FIG. 13 is a diagram illustrating an example of a system configuration diagram of a sample analyzing system according to a second embodiment. [Figure 10] FIG. 4 is a diagram illustrating an example of state determination adequacy information according to the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of a sample state determination device, method, and sample analysis system according to the present invention will be described with reference to the accompanying drawings. In the following description and the accompanying drawings, components having the same functional configuration are designated by the same reference numerals, and duplicated explanations will be omitted. EXAMPLES

[0012] An example of an apparatus for determining a sample state in an image according to the first embodiment will be described with reference to Fig. 1. Fig. 1 shows an example of the hardware configuration of a sample state determination apparatus 101 and peripheral devices.

[0013] The specimen state determination device 101 includes an interface 110, a calculator 111, a memory 112, and a bus 113, and the interface 110, the calculator 111, and the memory 112 transmit and receive information via the bus 113. The specimen state determination device 101 also transmits and receives signals between the image capture device 120 and the display device 121 via the interface 110.

[0014] Each part of the sample state determination device 101 will be described. The interface 110 is a communication device that transmits and receives signals to and from devices external to the specimen state determination device 101. Devices that communicate with the interface 110 include an imaging device 120 and a display device 121. The imaging device 120 and the display device 121 will be described in detail later.

[0015] The calculator 111 is a device that executes various processes in the specimen state determination device 101, and is, for example, a CPU (Central Processing Unit) or an FPGA (Field-Programmable Gate Array), etc. The functions executed by the calculator 111 will be described later with reference to FIG.

[0016] The memory 112 is a device that stores the programs executed by the arithmetic unit 111, parameters, coefficients, processing results, etc., and is an HDD, SSD, RAM, ROM, flash memory, or the like.

[0017] The imaging device 120 is a device that captures an image of the specimen 130 from the side, and is, for example, a visible light camera or a hyperspectral camera. The imaging device 120 transmits the captured image to the specimen state determination device 101. The imaging device 120 may be replaced by a communication device that receives an image via a network or the like, or a recording device that receives an image by reading an image recorded on a recording medium.

[0018] The display device 121 is a device for displaying the sample state determination result output by the sample state determination device 101 and / or position information of pixels or areas within the image used for the sample state determination, and is, for example, a display or printer.

[0019] The sample state determination device 101 will be described in detail below.

[0020] 2 is an example of a functional block diagram according to the first embodiment of the specimen state determination device 101. These functions can be realized by a computing device, and each function in the computing device may be configured with dedicated hardware, or may be configured with a computing unit 111 and software that runs on the computing unit 111.

[0021] The sample state determination device 101 comprises an image input section 201, a serum (plasma) region detection section 202, a state determination adequacy information calculation section 203, a state determination region selection section 204, a state determination section 205, and an output section 206. Each section will be described below.

[0022] The image input unit 201 receives an image of a specimen, the state of which is to be determined, captured from the side and input via the interface 110 .

[0023] The serum (plasma) region detection unit 202 will be described.

[0024] The serum (plasma) region detection unit 202 detects a serum or plasma region from within the sample side image output from the image input unit 201. For example, a method for detecting the serum or plasma region is a method for detecting the upper and lower ends of the serum or plasma region in the input sample side image. As a detector, a machine learning technique such as YOLO or SSD may be used. Alternatively, manually designed features such as a Hough transform may be utilized to perform detection using a manually designed or machine learning detector.

[0025] Another example of a method for detecting serum or plasma regions is region segmentation (semantic segmentation). Region segmentation is a method for dividing an image into semantic sets in units of pixels or small regions within the image, for example, dividing an image into serum or plasma regions, separating agent regions, blood clot regions, label regions, etc. As a region divider, a machine learning method such as U-Net or SegNet may be used, or the regions may be divided using the Watershed method, etc.

[0026] Generally, a long calculation time is required to perform highly accurate region division, which causes a decrease in analysis throughput, so here, a method for detecting the upper and lower ends of serum or plasma will be described as an example. In the case of machine learning methods, a detector is trained using many combinations of input images and information on the correct upper and lower end positions of serum or plasma, and the serum (plasma) region detection unit 202 uses a trained (learned) detector.

[0027] The serum (plasma) region detection unit 202 detects the upper and lower ends of the serum or plasma using a serum or plasma upper and lower end detector, and extracts a partial image, for example from the upper end of the serum or plasma to the lower end of the serum or plasma, and outputs it as a serum or plasma region image.

[0028] 3A and 3B show examples of serum upper and lower end detection. Fig. 3A is an example in which serum 302 is stored in container 301. Serum upper end detection result 303 and serum lower end detection result 304 represent the upper and lower ends of serum detected by the serum (plasma) upper and lower end detector. Label 305 is a label attached to container 301, on which a barcode and specimen information are printed. Structure 306 is an example of a structure in which a separating agent is stored, and scale 307 is a scale printed directly on the test tube.

[0029] FIG. 3B shows serum region 308. Serum region 308 is an example of a partial image of the serum region extracted based on serum upper end detection result 303 and serum lower end detection result 304. Here, serum region image 308 represents an example of a region having a height of y2-y1 and a width of x2-x1 extracted from start coordinates (x1, y1), assuming that the vertical center coordinate of serum upper end detection result 303 is y1, the vertical center coordinate of serum lower end detection result 304 is y2, the left end of serum upper end detection result 303 and serum lower end detection result 304 is x1, and the right end is x2. In this way, serum (plasma) region detection unit 202 detects the plasma or plasma region and outputs a serum or plasma region image.

[0030] The state determination adequacy information calculation unit 203 will be described.

[0031] The status determination adequacy information calculation unit 203 calculates information for each pixel or small region to select a region for status determination from the serum or plasma region image output by the serum (plasma) region detection unit 202. This information is referred to as status determination adequacy information in this specification. The status determination adequacy information calculation unit 203 may calculate and output multiple pieces of status determination adequacy information.

[0032] As examples of the state judgment suitability information, edge information, color suitability information, and distance information will be described.

[0033] Edge information is information that represents changes in pixel values ​​(brightness values, RGB values, etc.) in small regions within an image. For example, it can be calculated using a Sobel filter or local variance values. An example of extracted edge information is shown in FIG. 4A. Edge information 401 shows an example of extracted edges from serum region image 308 in FIG. 3B. The closer to white it is, the more edge components are included.

[0034] The edge information 401 contains strong edge components near the boundaries of the label 305 and the boundaries of the barcode, scale 307, and structure 306. These regions may occlude the serum or plasma region, or the color or brightness of the serum or plasma region may change, making them unsuitable for use in determining the state.

[0035] Using edge information makes it possible to exclude these regions from candidate regions for state determination. A specific method of exclusion will be described in the state determination region selection unit 204. Note that in order to detect a region containing a large amount of edge components and its surrounding regions, the edge components may be enlarged by dilation, filter processing, or the like.

[0036] The color suitability information is information that indicates how well each pixel or each small region in the serum or plasma region image conforms to the appropriate range of color and / or brightness. Methods for calculating the color suitability information include, for example, a method of collecting appropriate serum or plasma images in advance (calibration data) and calculating a color histogram for each pixel in the calibration data, and a method of determining an appropriate color range on each axis or space of a color system such as RGB, HSV, or HLS.

[0037] Here, a method using a color histogram will be described as an example. An example of a calculation formula for color suitability information is shown in formula (1). Here, color(x, y) represents the color at coordinates (x, y) in the serum or plasma region image, and HIST(c) is a function that outputs the frequency of color c in the histogram of the calibration data calculated in advance. According to formula (1), color suitability information CA indicates color suitability, and is high for colors that appear frequently in the calibration data and low for colors that appear infrequently. CA(x, y)=HIST(color(x, y)) (1)

[0038] The color suitability information calculated in this manner is effective in identifying a region that is likely to be serum or plasma. For example, since a label or printed region is often a different color from serum or plasma, the color suitability is low. In addition, the color suitability tends to be low in regions where the brightness is excessively increased due to being too close to the light, regions where the brightness is excessively decreased due to being too far from the light, and regions where the color or brightness value of the serum or plasma region has changed due to foreign matter in the test tube.

[0039] FIG. 4B shows an example of color suitability information. Color suitability information 402 shows an example of calculation of color suitability information of serum region image 308 in FIG. 3B. The closer to white, the higher the color suitability, and the closer to black, the lower the color suitability. For example, label 305, scale 307, background region, etc. have very low color suitability and are shown in black. In addition, in the region including structure 306, the color and brightness value of the serum image change, so it is gray. In this way, color suitability information is effective for selecting an appropriate candidate region for state determination in state determination region selection unit 204 described later when there is unevenness in color and brightness value in serum or plasma.

[0040] Next, distance information will be described. Distance information is information that indicates how far each pixel is from a reference point, for example, the top or bottom of the serum in the image or the lighting position. For example, when the sample is illuminated from above, the upper end of the serum (closer to the lighting) tends to be brighter, and the lower end of the serum (farther from the lighting) tends to be darker.

[0041] In this way, since there is a correlation between the position in the image and the color and brightness value of the image, distance information may be output as one of the state judgment suitability information. An example of distance information is shown in FIG. 10. FIG. 10 shows an example in which serum 302 is stored in a container 301 and a light 1001 is illuminating serum 302 downward. Note that the light 1001 shows an example of the lighting position, and the lighting may be set at another position. In addition, the reference point of the distance is not limited to the light, and may be, for example, the upper end or lower end of the image, or an area or point detected from the image, such as the upper end of the serum or container, may be used as the reference point. Also, it is assumed that the serum upper end 303 and serum lower end 304 are detected as the serum upper end and lower end as in FIG. 3A.

[0042] Origin 1002, origin 1003, and origin 1004 represent examples of origins (reference points) when calculating distance information, with origin 1002 representing the point closest to illumination 1001, origin 1003 representing the center point of serum upper end 303, and origin 1004 representing the center point of serum lower end 304. Distance information 1005, distance information 1006, and distance information 1007 represent examples of distance information when origin 1002, origin 1003, and origin 1004 are used as the origins when calculating distance information, with the closer to black the shorter the distance and the closer to white the longer the distance.

[0043] In this way, the condition determination adequacy information calculation unit 203 may calculate the distance from an arbitrary point in the image to each pixel or each small region as the condition determination adequacy information. Note that instead of a point, the calculation may also be made of the distance from some reference line, such as a horizontal line at the upper / lower end of the serum, to each pixel or each small region.

[0044] The state determination region selection unit 204 will now be described.

[0045] The state determination area selection unit 204 determines and outputs one or more pixels or areas for the state determination unit 205, described below, to determine the state, using the state determination adequacy information (group) calculated by the state determination adequacy information calculation unit 203. Here, a selection method of the state determination area using the edge information, color appropriateness information, and distance information described in the state determination adequacy information calculation unit 203 will be described as an example.

[0046] FIG. 5 shows a flow diagram of an example of a method for selecting a state determination area. In the example shown in FIG. 5, first, edge information and color appropriateness information are used to select candidates for the state determination area on a pixel-by-pixel basis, and distance information is used to determine the area for the final state determination. Note that only a part of the edge information, color appropriateness information, and distance information may be referenced, and only one or two types of information among these may be referenced. For example, only color appropriateness information, or color appropriateness information and edge information or distance information may be referenced. By increasing the types of information referenced, a more accurate determination is possible.

[0047] In step S501, variables x and y representing the coordinates of the pixel or small region to be selected are initialized to 0, and a candidate map is generated and initialized. A pixel or small region is a partial region that is a part of the image of the serum region 308. A candidate map is temporary information for determining a region for status determination, and is a two-dimensional array that records whether a region is suitable as a candidate for status determination. Here, the candidate map is a two-dimensional array of binary values ​​(0 or 1) with the same width and height as the serum or plasma region image, and stores 1 in pixels or regions that are determined to be suitable as regions for status determination, and stores 0 in pixels or regions that are determined to be inappropriate. All values ​​are set to 0 as initialization.

[0048] In step S502, it is determined whether the edge information at the coordinates (x, y) is less than a predetermined threshold Th_e. If true (S502: Y), the flow proceeds to step S504, and if false (S502: N), the flow proceeds to step S503. In this step, as explained in the state determination suitability information calculation unit 203, since there is a high possibility that areas with large edge components and their surrounding areas are influenced by labels, printing, foreign matter, etc., such areas are excluded from candidates for state determination areas.

[0049] In step S503, the value of the candidate map at the coordinates (x, y) is set to 0.

[0050] In step S504, it is determined whether the color suitability information at the coordinates (x, y) is equal to or greater than a predetermined threshold Th_c. If true (S504: Y), the flow proceeds to step S505. If false (S504: N), the flow proceeds to step S503. In this step, only pixels or small areas with color suitability information equal to or greater than a certain level are extracted as areas or pixels suitable for state determination.

[0051] In step S505, the value of the candidate map at the coordinates (x, y) is set to 1.

[0052] In step S506, it is determined whether all pixels or all small regions in the serum or plasma region image have been processed. If true (S506: Y), the flow proceeds to step S508. If false (S506: N), the flow proceeds to step S507.

[0053] In step S507, the coordinates of the next pixel or small region to be selected are obtained. The coordinates can be obtained, for example, by scanning the image from the upper left to the lower right. Once new coordinates are obtained, the process returns to step S502 and repeats the process until step S506 becomes true.

[0054] In step S508, a region for final state determination is determined. In step S509, the coordinates of the selected region are output. When step S508 is executed, selection of candidates for each pixel has already been completed and the candidate map has been completed. Here, as an example, a case where a region for final state determination is determined using distance information will be described.

[0055] First, assume that information on the distance from the top of the serum or plasma and the distance from the bottom of the serum or plasma is input as distance information. As described above, in the example of the illumination position shown in Fig. 10, the serum tends to be brighter closer to the top and darker closer to the bottom, so among the candidate regions, a region close to the middle between the top and bottom of the serum is selected as the region for final status determination. In this way, a region close to the middle between the position closest to the illumination position and the position farthest from the illumination position can be selected as the region for final status determination.

[0056] Therefore, in this step, the number of candidate maps is 1, the absolute difference between the distance from the top of the serum or plasma and the distance from the bottom of the serum or plasma is calculated, and the pixel or small area in the serum or plasma region image corresponding to the coordinates with the smallest absolute difference value is selected as the pixel or small area for final status judgment.

[0057] FIG. 8 shows examples of a candidate map and a region for status determination. Candidate map 801 is an example of a candidate map obtained by inputting edge information 401 and color appropriateness information 402 described in FIG. 4A and FIG. 4B into the flow in FIG. 5. In candidate map 801, black represents 0 and white represents 1. Areas around edges and areas with low color appropriateness are excluded (black), and only serum areas suitable for status determination are extracted as candidates (white). Region for status determination 802 is an example of a region for status determination selected according to the procedure of S508. Region for status determination 802 in the serum or plasma region image is output as the final region for status determination.

[0058] In this way, the state determination area selection unit 204 selects and outputs pixels or small areas (partial areas) for state determination using the state determination adequacy information (group) calculated by the state determination adequacy information calculation unit 203. As described above, the state determination adequacy information calculation unit 203 may calculate new information by processing or combining the state determination adequacy information in order to select pixels or small areas for state determination. Also, for example, in step S508, multiple pixels or areas may be extracted, such as extracting an area with one candidate map that is closest to the upper end of the serum, an area that is closest to the lower end of the serum, and an area in between.

[0059] The flow shown in Figure 5 is merely an example, and it is not necessary to follow this flow. What is important is to select one or more pixels or regions for state judgment using the state judgment suitability information (group), and the selection process is not limited to the example shown here.

[0060] The state determination unit 205 will be described.

[0061] The state is determined using an image of the state determination region selected by the state determination region selection unit 204. The state determiner is, for example, a machine learning model such as a Neural Network, SVM, or decision tree. An image 802 of a partial region for state determination is input to the machine learning model. Alternatively, manually designed features may be extracted from the image 802 of the state determination region, and determination may be made from the extracted features by manual design or by a machine learning model.

[0062] The output of the state determiner may be an estimate of each LHI index as a real value, or may be a level in which the degree of LHI is divided into more general stages, or may be a binary value indicating whether or not the LHI exceeds a predetermined threshold (Positive or Negative).In the case of machine learning methods, the state determiner is trained using many combinations of input images and ground truth information on the sample state, and the state determination region selection unit 204 uses a trained (learned) state determiner.

[0063] When a single state determination region is selected by the state determination region selection unit 204, the state determination region selection unit 204 outputs the determination result for that region. When a plurality of state determination regions are selected, the classification result is calculated based on, for example, a majority vote of the determination results for the plurality of regions or the total of the classification scores.

[0064] Moreover, the state may be determined using one or more images obtained by performing statistical processing on the multiple regions for state determination and integrating them. This allows for more appropriate determination. For example, the state may be determined using an average image. The state determiner may determine the state from the input average image. The state may be determined using a difference image of the multiple regions for state determination. The difference image may be, for example, the difference between the maximum and minimum values ​​of pixels at the same position in each of the multiple regions. The state determiner may determine the state from either the input difference image or the multiple images, or from the average image thereof. The difference image is particularly effective when identification is performed taking into account the turbidity of the sample.

[0065] When the status judgment area selection unit 204 outputs a small area, the status judgment unit 205 judges the LHI for each pixel, i.e., based on the color and / or luminance of each pixel, and may calculate a final status judgment result using statistical values ​​such as majority vote, sum, average, weighted sum, etc. of the judgment results or judgment scores of each pixel in the small area.

[0066] Alternatively, statistical processing such as summation, average, weighted summation, etc. of values ​​of multiple pixels in the region may be performed, and the resulting value may be input to the state determiner. Alternatively, an image of the small region, that is, information on pixel positions and values, may be input to the determiner, and the state determination result may be calculated using texture information (spatial information on pixel values ​​(color and / or brightness)) as a feature. This allows for more accurate determination. Furthermore, as the weight of the weighted summation, a part of the state determination suitability information calculated by state determination suitability information calculation unit 203, such as color suitability information, may be used.

[0067] The output unit 206 will now be described.

[0068] The output unit 206 outputs the specimen state determination result output by the state determination unit 205 and / or the position information within the image of the state determination region output by the state determination region selection unit 204.

[0069] An example of the output results displayed on a screen is shown in Fig. 6. Window 601 is the window presented to the user, state determination result display section 602 shows the sample state determination result, and state determination area display section 603 shows the position of the state determination area within the input image as an image. However, the output format is not limited to the example in Fig. 6. For example, the position information of the state determination area may be displayed as a character string as a set of start coordinates and end coordinates or start coordinates, width, and height, rather than as an image. Also, instead of displaying each sample, the format may be a list of IDs and names of multiple samples, state determination results, and coordinate information of the state determination area (start coordinates and end coordinates, etc.).

[0070] Moreover, these displays may be recorded once in the memory 112 and then displayed in any format according to an instruction from the user.

[0071] In the above, the details of the first embodiment have been described for each functional block. However, the embodiment of the present invention does not necessarily need to be configured with the functional blocks of FIG. 2, as long as it can realize the processing for realizing the operation of each functional block. FIG. 7 shows an example of a processing flow diagram according to the first embodiment. Each step corresponds to each element of the functional block diagram shown in FIG. 2.

[0072] In image input step S701, an image of a specimen, the state of which is to be determined, taken from the side and inputted through the interface 110 is received.

[0073] In serum (plasma) region detection step S702, a serum or plasma region is detected from the sample image received in image input step S701.

[0074] In the condition determination adequacy information calculation step S703, information for evaluating the adequacy of the condition determination (condition determination adequacy information) is calculated from the serum or plasma region calculated in the serum (plasma) region detection step S702.

[0075] In the state determination region selection step S704, one or more partial regions for state determination are selected from the serum or plasma region based on the state determination adequacy information calculated in the state determination adequacy information calculation step S703.

[0076] In state determination step S705, the state (LHI) of the sample is determined using the partial region for state determination selected in state determination region selection step S704.

[0077] An output step S706 outputs the specimen state determination result calculated in the state determination step S705 and / or position information of the partial region for state determination selected in the state determination region selection step S704.

[0078] According to the sample state determination device described in Example 1, it is possible to accurately determine the state of the sample even when the image contains factors that affect the appearance of the serum or plasma region or when the serum or plasma region is not uniform. EXAMPLES

[0079] In the second embodiment, a sample analysis system is described which determines the state of a sample using the sample state determination device described in the first embodiment and controls the transport path of the specimen in accordance with the result of the sample state determination.

[0080] FIG. 9 shows a system configuration diagram according to the second embodiment. The sample analysis system 901 comprises an imaging device 120 that images the sample 130 from the side, a sample state determination device 101 described in Example 1, a sample transport device 902 that transports the sample 130, a sample acquisition device 903 that acquires a sample from the sample transported by the sample transport device 902, an analysis device 904 that analyzes the sample acquired by the sample acquisition device 903, and a display device 121 that presents to a user the sample analysis results by the analysis device 904, the sample state determination results by the sample state determination device 101 and / or position information of the state determination area.

[0081] The imaging device 120 is not described here because it is the same as the hardware components in the first embodiment. The specimen transport device 902, the sample acquisition device 903, the analysis device 904, and the display device 121 will be described below.

[0082] The specimen transport device 902 is a device for transporting specimens, such as a belt conveyor that transports specimens (test tubes containing specimens) fixed to a holder. The specimen transport device switches the transport path for each specimen based on the specimen state determination result for each specimen by the specimen state determination device 101. Specifically, specimens whose LHI is determined to be negative or whose LHI level index value is equal to or less than a predetermined threshold value are transported to the specimen acquisition device 903, and other specimens are transported, for example, to a temporary storage space. Specimens transported to the temporary storage space are not analyzed by the analysis device 904, and the user decides what to do with them, or they are transported to another device.

[0083] The sample acquiring device 903 is a device for acquiring a sample from within the specimen transported by the specimen transport device 902, and is a suction nozzle or the like.

[0084] The analysis device 904 is a device that performs biochemical analysis or immunoanalysis using a reaction solution obtained by reacting the sample acquired by the sample acquiring device 903 with a reagent.

[0085] The display device 121 displays the analysis results from the analysis device 904, and the sample state determination results from the sample state determination device 101 and / or position information of the state determination area. If any of the LHIs of a sample are positive, this may affect the analysis results. With the above configuration, samples whose LHIs are determined to be positive or have a high index value by the sample state determination device 101 are not acquired by the sample acquisition device 903 and are not analyzed by the analysis device 904. This makes it possible to analyze only normal samples, preventing the waste of sample samples containing LHIs and saving analysis time and reagents. <Modification>

[0086] Although a method has been described in which the serum (plasma) region detection unit 202 extracts and outputs a plasma or plasma region image as serum or plasma region information, for example, an input image received from the image input unit 201 and rectangular information or mask information representing the serum or plasma region may be output. Also, instead of acquiring the entire serum or plasma region, for example, only the top end of the serum or plasma may be detected, and a region at a certain height from the top end of the serum or plasma may be extracted as the serum or plasma region. Also, when a region division method is used, regions such as labels and printing may be determined at this point and excluded from the candidate region.

[0087] In the state determination suitability information calculation unit 203, the color suitability information has been described. For example, when there are images (training images, etc.) used to construct a state determiner used in the state determination unit 205, these images may be used as calibration data. In this case, it is expected that the higher the value of the color suitability information, the higher the reliability of the result of the state determiner. In addition, when determining color using a color histogram or the like, a determination method for serum and plasma may be prepared. For example, in the case of a color histogram, calibration data that collects only serum and calibration data that collects only plasma may be prepared, and a histogram for serum and a histogram for plasma may be created. In addition, when determining color similarity, any color system such as RGB, HSV, HLS, etc. may be used.

[0088] The state determination area selection unit 204 may select an area for state determination based on the magnitude of the value of any of the state determination suitability information. For example, a pixel or area with the highest color suitability in the candidate area may be selected as the area for state determination. Although an example of scanning all pixels has been described in FIG. 5, the number of scans may be reduced under some condition, such as proceeding to the next step when a certain number or more of candidate areas are found.

[0089] Furthermore, at least one of the serum (plasma) area detection unit 202, the status determination suitability information calculation unit 203, and the status determination unit 205 may perform serum (plasma) area detection, status determination suitability information calculation, status determination, etc., using the results of applying image processing such as color and brightness normalization and correction, filtering, and Retinex to the image to be processed.

[0090] As an example of the image processing, there is a method of formulating the characteristics of the lighting and normalizing and correcting the color and brightness values ​​in the image based on the characteristics. The illuminance that each pixel receives from the lighting can be formulated based on the distance and angle between each pixel and the lighting. Based on the formulated illuminance, it is possible to correct the brightness value so that, for example, the illuminance of a place with low illumination becomes the same as that of the pixel with the highest illumination.

[0091] Furthermore, when multiple lighting fixtures are used, the color temperature may differ depending on the lighting fixture. When the state of a sample is judged based on color, it is preferable for the color to be uniform. Therefore, for example, the amount of hue shift at each pixel is formulated based on the specifications of each lighting fixture or the color temperature measured in advance. By correcting the hue at each pixel based on the formulated amount of hue shift, it is possible to reduce the change in hue due to differences in lighting fixtures.

[0092] In addition, a component separation process such as Retinex may be applied. Retinex is a process that separates the constant color and brightness components of an object from the color and brightness components caused by lighting. By extracting the constant color and brightness information of an object using Retinex, a corrected image can be obtained in which the difference in color and brightness caused by lighting at each coordinate is reduced.

[0093] By using such image processing to detect the serum (plasma) region, calculate the state determination suitability information, determine the state, and so on, more accurate detection and determination are possible.

[0094] The present invention is not limited to the above-mentioned embodiment, but includes various modified examples. For example, the above-mentioned embodiment has been described in detail to clearly explain the present invention, and is not necessarily limited to those including all of the configurations described. It is also possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.

[0095] In addition, the above-mentioned configurations, functions, processing units, etc. may be realized in part or in whole by hardware, for example, by designing them as integrated circuits. In addition, the above-mentioned configurations, functions, etc. may be realized in software by a processor interpreting and executing a program that realizes each function. Information such as the program, table, file, etc. that realizes each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card or SD card.

[0096] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]

[0097] 101: Sample condition determination device 110: Interface 111: Arithmetic unit 112: Memory 113: Bus 120: Imaging device 121:Display device 130: Specimen 201: Image input unit 202: Serum (plasma) region detection section 203: State judgment appropriateness information calculation unit 204: State judgment area selection unit 205: Status determination unit 206: Output section 701: Image input step 702: Serum (plasma) region detection step 703: State judgment appropriateness information calculation step 704: State determination area selection step 705: Status determination step 706: Output Step

Claims

1. A sample state determination device that determines a state of a sample stored inside a container, comprising: an image input unit that receives an image captured from the side of a container in which a sample is stored; an area detection unit for identifying an area of ​​serum or plasma from within the image; a state determination adequacy information calculation unit that calculates state determination adequacy information, which is information for evaluating the adequacy of the state determination of the sample, for each partial region of the serum or plasma region identified by the region detection unit; a state determination region selection unit that selects a partial region suitable for determining the state of the sample based on the state determination suitability information; a state determination unit that determines a state of the sample by using the partial region selected by the state determination region selection unit; an output unit that outputs a result of the specimen state determination determined by the state determination unit and / or position information within the image of a partial region selected by a state determination region selection unit.

2. 2. The sample state determination device according to claim 1, The state determination adequacy information calculation unit calculates edge information for each of the partial regions as information included in the state determination adequacy information.

3. 2. The sample state determination device according to claim 1, A specimen state determination device characterized in that the state determination suitability information calculation unit calculates the suitability of the color or brightness of each of the partial regions for specimen state determination as information included in the state determination suitability information.

4. 2. The sample state determination device according to claim 1, The state determination adequacy information calculation unit calculates a distance from a pre-specified origin to the partial region as information included in the state determination adequacy information.

5. 2. The sample state determination device according to claim 1, At least one of the area detection unit, the state determination adequacy information calculation unit, and the state determination unit applies at least one of normalization and correction, filtering, and component separation processing to the image.

6. 2. The sample state determination device according to claim 1, The state determination unit determines the state of the sample by using texture information of the partial region.

7. 2. The sample state determination device according to claim 1, the state determination region selection unit extracts a plurality of partial regions; The state determination unit determines the state of the sample using an image obtained by integrating the plurality of partial regions through statistical processing.

8. A method for determining a state of a sample stored inside a container, comprising: Accepting an image of a container containing a sample taken from the side; Identifying an area of ​​serum or plasma in the image; Calculating state determination suitability information, which is information for evaluating suitability for state determination of the sample, for each partial region of the identified region; selecting a partial region suitable for judging the state of the sample based on the state judgment suitability information; determining a state of the sample using the selected partial region; A sample state determination method, comprising: outputting a result of the determination and / or position information of the selected partial region within the image.

9. 9. A method for determining a sample state according to claim 8, comprising: The sample state determination method, wherein the device calculates edge information for each of the partial regions as information included in the state determination suitability information.

10. 9. A method for determining a sample state according to claim 8, comprising: A specimen state determination method, characterized in that the device calculates the suitability of the color or brightness of each of the partial regions for specimen state determination as information included in the state determination suitability information.

11. 9. A method for determining a sample state according to claim 8, comprising: A sample state determination method, wherein the device calculates a distance from a pre-specified origin of the serum or plasma region in the image to the partial region as information included in the state determination suitability information.

12. 9. A method for determining a sample state according to claim 8, comprising: A specimen state determination method, wherein the apparatus determines the state of the specimen using texture information of the partial region.

13. 9. A method for determining a sample state according to claim 8, comprising: The apparatus extracts a plurality of subregions; A sample state determination method, wherein the device determines the state of the sample using an image obtained by integrating the plurality of partial regions through statistical processing.

14. A sample analysis system that determines a state of a sample stored in a container and controls transportation of the container storing the sample based on a result of the determination, an imaging device for acquiring an image of a container containing a sample from a side; A specimen state determination device according to claim 1 , which determines a state of the specimen in the image; a transport device that controls a transport path of a container in which the sample is stored based on a result of the determination of the state of the sample by the sample state determination device; a sample acquiring device for acquiring the sample from the container transported by the transport device; an analysis device that analyzes the sample acquired by the sample acquisition device; A sample analysis system including: a display device that displays the analysis results by the analysis device, and the judgment results by the sample state judgment device and / or position information within the image of the partial area used to judge the state of the sample.

Images