Sample state determination device, sample state determination method, and sample analysis system

Abstract

This sample state determination device comprises: an image input unit that receives an image obtained by imaging, from a side surface, a container having a sample housed therein; a region detection unit that identifies a region of serum or plasma in the image; a state-determining appropriateness information calculation unit that calculates state-determining appropriateness information for evaluating the degree of appropriateness for determining the state of the sample, for each partial region in the region of serum or plasma identified by the region detection unit; a state determination region selection unit that selects, on the basis of the state-determining appropriateness information, a partial region appropriate for determining the state of the sample; a state determination unit that determines the state of the sample using the partial region selected by the state determination region selection unit; and an output unit that outputs a state determination result of the sample determined by the state determination unit and / or information about the position of the partial region selected by the state determination region selection unit in the image.

Description

Sample state determination device, sample state determination method, and sample analysis system Incorporation by Reference

[0001] This application claims priority from Japanese Patent Application No. 2023-158397, filed on September 22, 2023, the contents of which are incorporated herein by reference.

[0002] The present invention relates to a technique for determining the state of a sample, which is applicable to an automated blood analyzer such as a biochemical analyzer or an automated immunoanalyzer.

[0003] Automated blood analyzers, such as biochemical analyzers and immunoanalyzers, perform analyses based on the color or luminescence generated from the reaction solution resulting from the reaction of a sample with a reagent. If the sample contains lipemia (L), hemolysis (H), jaundice (I), or other conditions, the accuracy of the analysis results may be affected.

[0004] The task of visually selecting specimens containing such samples places a heavy burden on the user, and there is a strong need for automation. For example, Patent Document 1 discloses a method for determining the LHI of a specimen by acquiring multiple images of a specimen container containing the serum or plasma portion of the specimen, determining the lipemic, hemolytic, and icteric nature of the serum or plasma region for each pixel or patch from the multiple images using a convolutional neural network, and integrating the results within the serum or plasma region by majority voting or the like.

[0005] Special Publication No. 2020-519853

[0006] However, the containers in which samples are stored and the conditions inside the samples vary widely, and the color and brightness values ​​of the images may not be uniform even within the serum or plasma region of the same specimen. For example, structures containing a separating agent contained within a test tube, labels or stickers attached to the test tube, or printing on the test tube may affect the color and brightness values ​​of the serum or plasma image or may mask the serum or plasma region.

[0007] Furthermore, in the case of serum or plasma, for example, when lighting is illuminating the sample from above, the brightness values ​​of the serum or plasma region may be uneven, such that the upper part of the serum or plasma is bright and the lower part is dark. In this way, the serum or plasma region may contain regions that are not suitable for determining the state of the sample (such as LHI) due to reasons such as the color or brightness values ​​of the serum or plasma region having changed or being uneven due to factors other than the serum or plasma, or the serum or plasma region being obstructed. When an image of such a serum or plasma region is used to determine the state of the sample, conventional methods may adversely affect the results of the determination of the state of the sample.

[0008] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide an apparatus and method for determining the state of a sample with high accuracy.

[0009] A sample state determination device according to one embodiment of the present invention includes an image input unit that receives an image of a container containing a sample taken from the side; an area detection unit that identifies a serum or plasma area from within the image; a state determination suitability information calculation unit that calculates, for each partial area of ​​the serum or plasma area identified by the area detection unit, state determination suitability information, which is information for evaluating the suitability of the sample for determining its state; a state determination area selection unit that selects a partial area that is suitable for determining the state of the sample based on the state determination suitability information; a state determination unit that determines the state of the sample using the partial area selected by the state determination area selection unit; and an output unit that outputs the state determination result of the sample determined by the state determination unit and / or position information within the image of the partial area selected by the state determination area selection unit.

[0010] According to the sample state determination device of one aspect of the present invention, it is possible to determine the state of the sample with high accuracy.

[0011] FIG. 1 is a diagram showing an example of the hardware configuration of a sample state determination device according to Example 1. FIG. 2 is a diagram showing an example of a functional block diagram of a sample state determination device according to Example 1. FIG. 3 is a diagram showing an example of upper and lower end detection of serum according to Example 1. FIG. 4 is a diagram showing an example of upper and lower end detection of serum according to Example 1. FIG. 5 is a diagram showing an example of state determination suitability information according to Example 1. FIG. 6 is a diagram showing an example of state determination suitability information according to Example 1. FIG. 7 is a flow diagram showing an example of a method for selecting a state determination region according to Example 1. FIG. 8 is a diagram showing an example of a screen display according to Example 1. FIG. 9 is a diagram showing an example of a processing flow according to Example 1. FIG. 10 is a diagram showing an example of a candidate map and a region for state determination according to Example 1. FIG. 11 is a diagram showing an example of a system configuration diagram of a sample analysis system according to Example 2. FIG. 12 is a diagram showing an example of state determination suitability information according to Example 1.

[0012] Hereinafter, an embodiment of a sample state determination device, method, and sample analysis system according to the present invention will be described with reference to the accompanying drawings. In the following description and accompanying drawings, components having the same functional configuration are designated by the same reference numerals, and redundant description will be omitted.

[0013] An example of an apparatus for determining the state of a sample in an image according to Example 1 will be described with reference to Fig. 1. Fig. 1 shows an example of the hardware configuration of a sample state determination apparatus 101 and peripheral devices.

[0014] The specimen state determination device 101 includes an interface 110, a computing unit 111, a memory 112, and a bus 113, and the interface 110, the computing unit 111, and the memory 112 transmit and receive information via the bus 113. The specimen state determination device 101 also transmits and receives signals to and from an imaging device 120 and a display device 121 via the interface 110.

[0015] The following describes each part of the specimen state determination device 101. The interface 110 is a communication device that transmits and receives signals to and from devices external to the specimen state determination device 101. Devices that communicate with the interface 110 include an imaging device 120 and a display device 121. Details of the imaging device 120 and the display device 121 will be described later.

[0016] The computing unit 111 is a device that executes various processes within the specimen state determination device 101, and is, for example, a CPU (Central Processing Unit) or an FPGA (Field-Programmable Gate Array). The functions executed by the computing unit 111 will be described later with reference to FIG. 2.

[0017] The memory 112 is a device that stores the programs executed by the arithmetic unit 111, parameters, coefficients, processing results, etc., and is an HDD, SSD, RAM, ROM, flash memory, etc.

[0018] The imaging device 120 is a device that images the specimen 130 from the side, and is, for example, a visible light camera or a hyperspectral camera. The imaging device 120 transmits the captured image to the specimen state determination device 101. Note that the imaging device 120 may be replaced by a communication device that receives images via a network or the like, or a recording device that receives images by reading images recorded on a recording medium.

[0019] The display device 121 is a device for displaying the sample state determination results output by the sample state determination device 101 and / or the position information of pixels or areas within the image used for the sample state determination, and is, for example, a display or printer.

[0020] The sample state determination device 101 will be described in detail below.

[0021] 2 is an example of a functional block diagram according to the first embodiment of the specimen state determination device 101. These functions can be realized by a computing device, and each function within the computing device may be configured with dedicated hardware, or may be configured with a computing unit 111 and software running on the computing unit 111.

[0022] The sample state determination device 101 includes an image input unit 201, a serum (plasma) region detection unit 202, a state determination adequacy information calculation unit 203, a state determination region selection unit 204, a state determination unit 205, and an output unit 206. Each unit will be described below.

[0023] The image input unit 201 receives an image of a specimen, the state of which is to be determined, taken from the side and input via the interface 110 .

[0024] The serum (plasma) region detection unit 202 will be described.

[0025] The serum (plasma) region detection unit 202 detects a serum or plasma region from within the sample side image output from the image input unit 201. For example, a method for detecting the serum or plasma region is to detect the upper and lower ends of the serum or plasma region in the input sample side image. Machine learning techniques such as YOLO and SSD may be used as the detector. Alternatively, manually designed features such as a Hough transform may be utilized to perform detection using a manually designed or machine learning detector.

[0026] Another example of a method for detecting serum or plasma regions is region segmentation (semantic segmentation). Region segmentation is a method for dividing an image into semantic groups in units of pixels or small regions within the image, for example, dividing the image into serum or plasma regions, separating agent regions, blood clot regions, label regions, etc. As a region divider, a machine learning method such as U-Net or SegNet may be used, or the regions may be divided using the Watershed method, etc.

[0027] Generally, performing highly accurate region segmentation requires a long calculation time, which reduces the throughput of analysis, so here we will explain an example of a method for detecting the upper and lower ends of serum or plasma. In the case of machine learning techniques, a detector is trained using many combinations of input images and information on the correct upper and lower end positions of serum or plasma, and the serum (plasma) region detection unit 202 uses a trained (learned) detector.

[0028] The serum (plasma) region detection unit 202 detects the upper and lower ends of the serum or plasma using a serum or plasma upper and lower end detector, extracts a partial image, for example, from the upper end of the serum or plasma to the lower end of the serum or plasma, and outputs it as a serum or plasma region image.

[0029] 3A and 3B show examples of serum upper and lower end detection. FIG. 3A shows an example in which serum 302 is stored in container 301. Serum upper end detection result 303 and serum lower end detection result 304 indicate the upper and lower ends of the serum detected by the serum (plasma) upper and lower end detector. Label 305 is a label attached to container 301, and includes a barcode and specimen information. Structure 306 is an example of a structure in which a separating agent is stored, and scale 307 is a scale printed directly on the test tube.

[0030] FIG. 3B shows a serum region 308. The serum region 308 is an example of a partial image of a serum region extracted based on the serum upper end detection result 303 and the serum lower end detection result 304. Here, the serum region image 308 represents an example in which a region with a height of y2-y1 and a width of x2-x1 is extracted from the starting coordinates (x1, y1), where y1 is the vertical center coordinate of the serum upper end detection result 303, y2 is the vertical center coordinate of the serum lower end detection result 304, x1 is the left end of the serum upper end detection result 303 and the serum lower end detection result 304, and x2 is the right end. In this way, the serum (plasma) region detection unit 202 detects the plasma or plasma region and outputs a serum or plasma region image.

[0031] The state determination adequacy information calculation unit 203 will be described.

[0032] The status determination adequacy information calculation unit 203 calculates information for each pixel or small region to select a region for status determination from the serum or plasma region image output by the serum (plasma) region detection unit 202. This information is referred to as status determination adequacy information in this specification. The status determination adequacy information calculation unit 203 may calculate and output multiple pieces of status determination adequacy information.

[0033] As examples of the state determination suitability information, edge information, color suitability information, and distance information will be described.

[0034] Edge information is information that represents changes in pixel values ​​(brightness values, RGB values, etc.) in small regions within an image. For example, it can be calculated using a Sobel filter or local variance values. FIG. 4A shows an example of extracted edge information. Edge information 401 shows an example of extracted edges from the serum region image 308 in FIG. 3B. The closer to white the image, the more edge components are included.

[0035] The edge information 401 contains strong edge components near the boundaries of the label 305 and the boundaries of the barcode, scale 307, and structure 306. These regions may occlude the serum or plasma region, or the color or brightness of the serum or plasma region may have changed, making them unsuitable for use in determining the state.

[0036] Using edge information makes it possible to exclude these regions from candidate regions for state determination. Specific exclusion methods will be described in the state determination region selection unit 204. Note that, in order to detect regions containing many edge components and their surrounding regions, the edge components may be enlarged by dilation, filter processing, or the like.

[0037] The color suitability information is information that indicates how well each pixel or each small region in a serum or plasma region image conforms to an appropriate range of color and / or brightness. Methods for calculating the color suitability information include, for example, a method of collecting appropriate serum or plasma images in advance (calibration data) and calculating a color histogram for each pixel in the calibration data, and a method of determining an appropriate color range on each axis or space of a color system such as RGB, HSV, or HLS.

[0038] Here, a method using a color histogram will be described as an example. An example of a formula for calculating color appropriateness information is shown in formula (1). Here, color(x, y) represents the color at coordinates (x, y) in the serum or plasma region image, and HIST(c) is a function that outputs the frequency of color c in the histogram of the calibration data calculated in advance. According to formula (1), color appropriateness information CA indicates color appropriateness, and takes a high value for colors that appear frequently in the calibration data and a low value for colors that appear infrequently. CA(x, y)=HIST(color(x, y)) (1)

[0039] The color suitability information calculated in this manner is effective in identifying regions that appear to be serum or plasma. For example, regions with labels or prints often have a different color from serum or plasma, resulting in a low color suitability. Furthermore, color suitability tends to be low in regions where brightness is excessively high because the region is too close to the light source, regions where brightness is excessively low because the region is too far from the light source, and regions where the color or brightness of the serum or plasma region has changed due to foreign matter in the test tube.

[0040] FIG. 4B shows an example of color appropriateness information. Color appropriateness information 402 shows an example of calculated color appropriateness information for the serum region image 308 in FIG. 3B. The closer to white, the higher the color appropriateness, and the closer to black, the lower the color appropriateness. For example, the label 305, scale 307, background region, etc. have very low color appropriateness and are represented in black. Furthermore, the region including structure 306 is gray because the color and brightness value on the serum image change. In this way, color appropriateness information is effective for the state determination region selection unit 204 (described later) to select an appropriate candidate region for state determination when there are variations in color and brightness value in the serum or plasma.

[0041] Next, distance information will be described. Distance information is information that indicates how far each pixel is from a reference point, such as the top or bottom of the serum in the image or the lighting position. For example, if the sample is illuminated from above, the upper end of the serum (closer to the lighting) tends to be brighter, and the lower end of the serum (farther from the lighting) tends to be darker.

[0042] As described above, since there is a correlation between the position in the image and the color and brightness of the image, distance information may be output as one piece of status determination suitability information. An example of distance information is shown in FIG. 10 . FIG. 10 illustrates an example in which serum 302 is stored in a container 301 and a light 1001 is illuminating the serum 302 downward. Note that the light 1001 is an example of the lighting position, and the lighting may be set at another position. Furthermore, the reference point for distance is not limited to the light and may be, for example, the top or bottom edge of the image, or an area or point detected within the image, such as the top edge of the serum or container, may be used as the reference point. Furthermore, it is assumed that a serum upper end 303 and a serum lower end 304 are detected as the top and bottom edges of the serum, as in FIG. 3A .

[0043] Origin 1002, origin 1003, and origin 1004 represent examples of origins (reference points) when calculating distance information, with origin 1002 representing the point closest to illumination 1001, origin 1003 representing the center point of serum upper end 303, and origin 1004 representing the center point of serum lower end 304. Distance information 1005, distance information 1006, and distance information 1007 represent examples of distance information when origin 1002, origin 1003, and origin 1004 are used as the origins when calculating distance information, with closer to black representing shorter distances and closer to white representing longer distances.

[0044] In this way, the condition assessment adequacy information calculation unit 203 may calculate the distance from an arbitrary point in the image to each pixel or each small region as the condition assessment adequacy information. Note that instead of a point, the calculation may also be made to calculate the distance from some reference line, such as a horizontal line at the top or bottom of the serum, to each pixel or each small region.

[0045] The state determination region selection unit 204 will now be described.

[0046] The state determination area selection unit 204 determines and outputs one or more pixels or areas for the state determination unit 205 (described later) to determine the state, using the state determination adequacy information (group) calculated by the state determination adequacy information calculation unit 203. Here, a method for selecting a state determination area using edge information, color adequacy information, and distance information described in the state determination adequacy information calculation unit 203 will be described as an example.

[0047] FIG. 5 shows a flow diagram of an example of a method for selecting a state determination area. In the example shown in FIG. 5, edge information and color appropriateness information are used to first select candidates for the state determination area on a pixel-by-pixel basis, and distance information is then used to determine the final area for state determination. Note that only part of the edge information, color appropriateness information, and distance information may be referenced, or only one or two types of information may be referenced. For example, only color appropriateness information may be referenced, or color appropriateness information may be referenced together with edge information or distance information. Increasing the types of information referenced enables more accurate determination.

[0048] In step S501, the variables x and y representing the coordinates of the pixel or small region to be selected are initialized to 0, and a candidate map is generated and initialized. A pixel or small region is a partial region that is a part of the image of the serum region 308. The candidate map is temporary information for determining a region for status determination, and is a two-dimensional array that records whether a region is appropriate as a candidate for status determination. Here, the candidate map is a two-dimensional array of binary values ​​(0 or 1) with the same width and height as the serum or plasma region image, with 1 stored for pixels or regions determined to be appropriate as regions for status determination, and 0 stored for pixels or regions determined to be inappropriate. All values ​​are set to 0 as initialization.

[0049] In step S502, it is determined whether the edge information at the coordinates (x, y) is less than a predetermined threshold value Th_e. If true (S502: Y), the flow proceeds to step S504. If false (S502: N), the flow proceeds to step S503. In this step, as explained in the state determination adequacy information calculation unit 203, areas with large edge components and their surrounding areas are likely to be affected by labels, printing, foreign matter, etc., so such areas are excluded from candidates for state determination areas.

[0050] In step S503, the value of the candidate map at the coordinates (x, y) is set to 0.

[0051] In step S504, it is determined whether the color suitability information at the coordinates (x, y) is equal to or greater than a predetermined threshold value Th_c. If true (S504: Y), the flow proceeds to step S505. If false (S504: N), the flow proceeds to step S503. In this step, only pixels or small regions with color suitability information equal to or greater than a certain level are extracted as regions or pixels suitable for state determination.

[0052] In step S505, the value of the candidate map at the coordinates (x, y) is set to 1.

[0053] In step S506, it is determined whether all pixels or all small regions in the serum or plasma region image have been processed. If true (S506: Y), the flow proceeds to step S508. If false (S506: N), the flow proceeds to step S507.

[0054] In step S507, the coordinates of the next pixel or small region to be selected are acquired. The coordinates can be acquired, for example, by scanning the image from the upper left to the lower right. Once new coordinates are acquired, the process returns to step S502 and repeats the process until step S506 becomes true.

[0055] In step S508, a region for final state determination is determined. In step S509, the coordinates of the selected region are output. When step S508 is executed, selection of candidates for each pixel has already been completed, and the candidate map has been completed. Here, as an example, an example of determining a region for final state determination using distance information will be described.

[0056] First, assume that information on the distance from the top of the serum or plasma and the distance from the bottom of the serum or plasma is input as distance information. As mentioned above, in the example of the illumination position shown in Figure 10, the serum tends to be brighter closer to the top and darker closer to the bottom. Therefore, among the candidate regions, a region close to the middle between the top and bottom of the serum is selected as the region for final status determination. In this way, a region close to the middle between the position closest to the illumination position and the position farthest from the illumination position can be selected as the region for final status determination.

[0057] Therefore, in this step, the candidate map is 1, the absolute difference between the distance from the top of the serum or plasma and the distance from the bottom of the serum or plasma is calculated, and the pixel or small area in the serum or plasma region image corresponding to the coordinate with the smallest absolute difference value is selected as the pixel or small area for final status determination.

[0058] Figure 8 shows examples of a candidate map and a region for status determination. Candidate map 801 is an example of a candidate map obtained by inputting the edge information 401 and color appropriateness information 402 described in Figures 4A and 4B into the flow of Figure 5. In candidate map 801, black represents 0 and white represents 1. Areas around edges and areas with low color appropriateness are excluded (black), and only serum regions suitable for status determination are extracted as candidates (white). Region for status determination 802 is an example of a region for status determination selected according to the procedure of S508. Region for status determination 802 in the serum or plasma region image is output as the final region for status determination.

[0059] In this way, the state determination region selection unit 204 selects and outputs pixels or small regions (partial regions) for state determination using the state determination adequacy information (group) calculated by the state determination adequacy information calculation unit 203. As described above, the state determination adequacy information calculation unit 203 may calculate new information by processing or combining the state determination adequacy information in order to select pixels or small regions for state determination. Furthermore, for example, in step S508, multiple pixels or regions may be extracted, such as by extracting a region with one candidate map that is closest to the upper end of the serum, a region that is closest to the lower end of the serum, and a region intermediate therebetween.

[0060] The flow shown in Figure 5 is merely an example, and it is not necessary to follow this flow. What is important is that one or more pixels or regions for state determination are selected using the state determination suitability information (group), and the selection process is not limited to the example shown here.

[0061] The state determination unit 205 will be described.

[0062] The state is determined using an image of the state determination region selected by the state determination region selection unit 204. The state determiner is, for example, a machine learning model such as a neural network, an SVM, or a decision tree. An image 802 of the partial region for state determination is input to the machine learning model. Alternatively, manually designed feature amounts may be extracted from the image 802 of the state determination region, and determination may be made from the extracted feature amounts by manual design or by a machine learning model.

[0063] The output of the state determiner may be the result of estimating each index of LHI as a real value, or may be a level obtained by dividing the degree of LHI into broader stages, or may be a binary value indicating whether or not the LHI exceeds a predetermined threshold (positive or negative). In the case of a machine learning method, the state determiner is trained using many combinations of input images and correct information on the sample state, and the state determination region selection unit 204 uses the trained (learned) state determiner.

[0064] If a single state determination region is selected by the state determination region selection unit 204, the state determination region selection unit 204 outputs the determination result for that region. If multiple state determination regions are selected, the identification result is calculated based on, for example, a majority vote of the determination results for the multiple regions or the sum of the identification scores.

[0065] Furthermore, the state determination may be performed using one or more images obtained by performing statistical processing on multiple regions for state determination and integrating them. This allows for more appropriate determination. For example, the state determination may be performed using an average image. The state determiner may determine the state from the input average image. The state determination may be performed using a difference image of multiple regions for state determination. The difference image may be, for example, the difference between the maximum and minimum values ​​of pixels at the same position in each of the multiple regions. The state determiner may determine the state from either the input difference image or multiple images, or from an average image thereof. The difference image is particularly effective when identification is performed taking into account the turbidity of the sample.

[0066] When the state determination area selection unit 204 outputs a small area, the state determination unit 205 determines the LHI for each pixel, i.e., based on the color and / or luminance of each pixel, and may calculate the final state determination result using statistical values ​​such as majority vote, sum, average, weighted sum, etc. of the determination results or determination scores of each pixel within the small area.

[0067] Alternatively, statistical processing such as summation, average, or weighted sum of values ​​of multiple pixels in the region may be performed, and the resulting value may be input to the state determiner. Alternatively, by inputting the image of the small region, i.e., information on the pixel positions and values, to the determiner, the state determination result can be calculated using texture information (spatial information on pixel values ​​(color and / or brightness)) as a feature. This enables more accurate determination. Furthermore, for the weight of the weighted sum, part of the state determination adequacy information calculated by the state determination adequacy information calculation unit 203, such as color adequacy information, may be used.

[0068] The output unit 206 will now be described.

[0069] The output unit 206 outputs the sample state determination result output by the state determination unit 205 and / or the position information within the image of the state determination region output by the state determination region selection unit 204 .

[0070] An example of a screen display of output results is shown in Figure 6. Window 601 is the window presented to the user, state determination result display section 602 displays the sample state determination result, and state determination area display section 603 displays the position of the state determination area within the input image as an image. However, the output format is not limited to the example in Figure 6. For example, the position information of the state determination area may be displayed as a character string containing start and end coordinates, or a set of start coordinates, width, and height, rather than as an image. Furthermore, instead of displaying each individual sample, the IDs and names of multiple samples, the state determination results, and coordinate information of the state determination area (start and end coordinates, etc.) may be displayed in a format that lists them.

[0071] Furthermore, these displays may be recorded in the memory 112 once and then displayed in any format according to instructions from the user.

[0072] The details of Example 1 have been described above for each functional block. However, the embodiment of the present invention does not necessarily need to be configured using the functional blocks of FIG. 2, as long as it can realize the processing that realizes the operation of each functional block. FIG. 7 shows an example of a processing flow diagram according to Example 1. Each step corresponds to each element of the functional block diagram shown in FIG. 2.

[0073] In the image input step S701, an image of a specimen, the state of which is to be determined, taken from the side and input via the interface 110 is accepted.

[0074] In the serum (plasma) region detection step S702, a serum or plasma region is detected from the sample image received in the image input step S701.

[0075] In the state determination adequacy information calculation step S703, information for evaluating the adequacy of the state determination (state determination adequacy information) is calculated from the serum or plasma region calculated in the serum (plasma) region detection step S702.

[0076] In the state determination region selection step S704, one or more partial regions for state determination are selected from the serum or plasma region based on the state determination adequacy information calculated in the state determination adequacy information calculation step S703.

[0077] In the state determination step S705, the state (LHI) of the sample is determined using the partial region for state determination selected in the state determination region selection step S704.

[0078] In the output step S706, the state determination result of the sample calculated in the state determination step S705 and / or the position information of the partial region for state determination selected in the state determination region selection step S704 is output.

[0079] According to the sample state determination device described in Example 1, it is possible to accurately determine the state of the sample even when the image contains factors that affect the appearance of the serum or plasma region or when the serum or plasma region is not uniform.

[0080] In the second embodiment, a sample analysis system is described that determines the state of a sample using the sample state determination device described in the first embodiment, and controls the transport path of the specimen in accordance with the result of the sample state determination.

[0081] 9 shows a system configuration diagram according to Example 2. The sample analysis system 901 includes an imaging device 120 that images the sample 130 from the side, the sample state determination device 101 described in Example 1, a sample transport device 902 that transports the sample 130, a sample acquisition device 903 that acquires a sample from the sample transported by the sample transport device 902, an analysis device 904 that analyzes the sample acquired by the sample acquisition device 903, and a display device 121 that presents to a user the sample analysis results by the analysis device 904, the sample state determination results by the sample state determination device 101, and / or position information of the state determination region.

[0082] The imaging device 120 is not described here because it is the same as the hardware components in the first embodiment. The specimen transport device 902, the sample acquisition device 903, the analysis device 904, and the display device 121 will be described below.

[0083] The specimen transport device 902 is a device for transporting specimens, such as a belt conveyor that transports specimens (test tubes containing specimens) fixed in holders. The specimen transport device switches the transport path for each specimen based on the specimen state determination result for each specimen by the specimen state determination device 101. Specifically, specimens determined to have a negative LHI or an LHI level / index value below a predetermined threshold are transported to the specimen acquisition device 903, while other specimens are transported, for example, to a temporary storage space. Specimens transported to the temporary storage space are not analyzed by the analyzer 904, and the user can decide what to do with them, or they are transported to another device.

[0084] The sample acquiring device 903 is a device that acquires a sample from within the specimen transported by the specimen transport device 902, and is a suction nozzle or the like.

[0085] The analysis device 904 is a device that performs biochemical analysis or immunological analysis using a reaction solution obtained by reacting the sample acquired by the sample acquisition device 903 with a reagent.

[0086] The display device 121 displays the analysis results from the analyzer 904, the sample state determination results from the sample state determination device 101, and / or the position information of the state determination area. If any of the LHIs is positive for a sample, this may affect the analysis results. With the above configuration, samples determined by the sample state determination device 101 to have a positive LHI or a high index value are not acquired by the sample acquisition device 903 or analyzed by the analyzer 904. This makes it possible to analyze only normal samples, preventing the waste of sample samples containing LHIs and saving analysis time and reagents. <Modification>

[0087] Although the method of extracting and outputting an image of the plasma or plasma region as serum or plasma region information in the serum (plasma) region detection unit 202 has been described, it is also possible to output, for example, an input image received from the image input unit 201 and rectangular information or mask information representing the serum or plasma region. Furthermore, instead of acquiring the entire serum or plasma region, it is also possible to detect, for example, only the top edge of the serum or plasma and extract a region at a certain height from the top edge of the serum or plasma as the serum or plasma region. Furthermore, when a region segmentation technique is used, regions of labels, printing, etc. may be identified at this point and excluded from the candidate region.

[0088] The color appropriateness information has been described in the state determination appropriateness information calculation unit 203. However, if there are images (training images, etc.) used to construct a state determiner used in the state determination unit 205, these images may be used as calibration data. In this case, the higher the value of the color appropriateness information, the higher the reliability of the results of the state determiner is expected to be. Furthermore, when determining color using a color histogram or the like, determination methods for serum and plasma may be prepared separately. For example, in the case of a color histogram, calibration data collecting only serum and calibration data collecting only plasma may be prepared, and a histogram for serum and a histogram for plasma may be created. Furthermore, when determining color similarity, etc., any color system, such as RGB, HSV, or HLS, may be used.

[0089] The state determination area selection unit 204 may select an area for state determination based on the magnitude of the value of any of the state determination suitability information. For example, the pixel or area with the highest color suitability within the candidate area may be selected as the area for state determination. While an example of scanning all pixels is described in Figure 5, the number of scans may be reduced based on some condition, such as proceeding to the next step when a certain number or more candidate areas are found.

[0090] Furthermore, at least one of the serum (plasma) region detection unit 202, the status determination adequacy information calculation unit 203, and the status determination unit 205 may perform serum (plasma) region detection, status determination adequacy information calculation, status determination, etc., using the results of applying image processing such as color and brightness normalization and correction, filtering, and Retinex to the image to be processed.

[0091] One example of image processing is a method of formulating the characteristics of lighting and normalizing and correcting the color and brightness values ​​in an image based on those characteristics. The illuminance that each pixel receives from lighting can be formulated based on the distance and angle between each pixel and the lighting. Based on the formulated illuminance, it is possible to correct the brightness value of areas with low illumination so that it becomes the same as the pixel with the highest illumination.

[0092] Furthermore, when multiple lighting sources are used, the color temperature may differ depending on the lighting source. When the condition of a sample is judged based on color, it is preferable for the color to be uniform. Therefore, for example, the amount of hue shift at each pixel is formulated based on the specifications of each lighting source or the color temperature measured in advance. By correcting the hue at each pixel based on the formulated amount of hue shift, it is possible to reduce the change in hue due to differences in lighting.

[0093] Furthermore, component separation processing such as Retinex may be applied. Retinex is a process that separates the constant color and brightness components of an object from the color and brightness components that are affected by lighting. By extracting the constant color and brightness information of an object using Retinex, a corrected image can be obtained in which the differences in color and brightness at each coordinate due to lighting are reduced.

[0094] By using such image processing to detect the serum (plasma) region, calculate the state determination suitability information, determine the state, etc., more accurate detection and determination become possible.

[0095] The present invention is not limited to the above-described embodiments, but includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations.

[0096] Furthermore, the above-described configurations, functions, processing units, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. Furthermore, the above-described configurations, functions, etc. may be implemented in software by a processor interpreting and executing a program that implements each function. Information such as the program, table, and file that implements each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card or SD card.

[0097] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines in the product. In reality, it can be assumed that almost all components are interconnected.

Claims

1. A sample state determination device that determines the state of a sample stored inside a container, comprising: an image input unit that accepts an image captured from the side of a container in which a sample is stored; an area detection unit that identifies an area of ​​serum or plasma from within the image; a state determination suitability information calculation unit that calculates state determination suitability information, which is information for evaluating the suitability of determining the state of the sample, for each partial area of ​​the serum or plasma area identified by the area detection unit; a state determination area selection unit that selects a partial area suitable for determining the state of the sample based on the state determination suitability information; a state determination unit that determines the state of the sample using the partial area selected by the state determination area selection unit; and an output unit that outputs the sample state determination result determined by the state determination unit and / or position information within the image of the partial area selected by the state determination area selection unit.

2. A specimen state determination device as described in claim 1, wherein the state determination adequacy information calculation unit calculates edge information for each of the partial regions as information included in the state determination adequacy information.

3. A specimen state determination device as described in claim 1, characterized in that the state determination suitability information calculation unit calculates the suitability of the color or brightness of each of the partial regions for specimen state determination as information contained in the state determination suitability information.

4. A sample state determination device as described in claim 1, wherein the state determination adequacy information calculation unit calculates the distance from a pre-specified origin to the partial region as information included in the state determination adequacy information.

5. A sample state determination device as described in claim 1, wherein at least one of the area detection unit, the state determination suitability information calculation unit, and the state determination unit apply at least one of normalization and correction, filtering, and component separation processing to the image.

6. A specimen state determination device according to claim 1, wherein the state determination unit determines the state of the specimen by using texture information of the partial region.

7. A sample state determination device as described in claim 1, wherein the state determination area selection unit extracts a plurality of partial areas, and the state determination unit determines the state of the sample using an image obtained by integrating the plurality of partial areas through statistical processing.

8. A sample status determination method in which an apparatus determines the status of a sample stored inside a container, the apparatus receiving an image of a container in which a sample is stored taken from the side, identifying an area of ​​serum or plasma from within the image, calculating status determination suitability information for each partial area of ​​the identified area, which is information for evaluating the suitability for determining the status of the sample, selecting a partial area suitable for determining the status of the sample based on the status determination suitability information, determining the status of the sample using the selected partial area, and outputting the result of the determination and / or position information within the image of the selected partial area.

9. A method for determining a specimen state as set forth in claim 8, wherein the apparatus calculates edge information for each of the partial regions as information included in the state determination suitability information.

10. A method for determining the state of a specimen as described in claim 8, characterized in that the device calculates the suitability of the color or brightness of each of the partial regions for determining the state of the specimen as information included in the state determination suitability information.

11. A method for determining a sample condition as described in claim 8, wherein the device calculates the distance from a pre-specified origin of the serum or plasma region in the image to the partial region as information included in the state determination suitability information.

12. A method for determining a specimen state according to claim 8, wherein the apparatus determines the state of the specimen using texture information of the partial region.

13. A method for determining a sample state as described in claim 8, wherein the device extracts a plurality of partial regions, and the device determines the state of the sample using an image obtained by integrating the plurality of partial regions through statistical processing.

14. A sample analysis system that determines the state of a sample stored inside a container and controls the transportation of the container that stores the sample based on the result of the determination, comprising: an imaging device that acquires an image of the container in which the sample is stored from the side; a sample state determination device as described in any one of claims 1 to 7 that determines the state of the sample in the image; a transport device that controls the transport path of the container in which the sample is stored based on the result of the determination of the state of the sample by the sample state determination device; a sample acquisition device that acquires the sample from the container transported by the transport device; an analysis device that analyzes the sample acquired by the sample acquisition device; and a display device that displays the analysis results by the analysis device, as well as the determination results by the sample state determination device and / or position information within the image of a partial area used in determining the state of the sample.

Images