Sample state determination device, sample state determination method, and sample analysis system

Abstract

A sample state determination device includes: an image input unit that receives an image obtained by capturing an image of a container in which a sample is stored from a side surface; a region detection unit that specifies a serum or plasma region from the image; a state determination suitability information calculation unit that calculates state determination suitability information, which is information for evaluating the suitability of the state determination for the sample, for each of the serum or plasma regions specified by the region detection unit; a state determination region selection unit that selects a partial region suitable for state determination of the sample on the basis of the state determination suitability information; a state determination unit that determines the state of the sample using the partial region selected by the state determination region selection unit; and an output unit that outputs a state determination result of the sample determined by the state determination unit and / or position information of the partial region selected by the state determination region selection unit in the image.

Description

[0001] Incorporation by reference

[0002] This application claims the benefit of Japanese patent application No. 2023-158397, filed on September 22, 2023, the contents of which are hereby incorporated by reference into this application. Technical Field

[0003] The present invention relates to a technique for determining the state of a sample that can be applied to an automatic blood analyzer such as a biochemical analyzer and an automatic immune analyzer. Background Art

[0004] Automated blood analyzers, such as biochemical analyzers and immunoassays, perform analysis based on the coloration and luminescence produced by the reaction solution resulting from the reaction of a sample with a reagent. However, if the sample contains lipemia (L), hemolysis (H), or icterus (I), the accuracy of the analysis results may be affected.

[0005] Visual screening of specimens containing such samples places a heavy burden on the user, and there is a high demand for automation. For example, Patent Document 1 discloses a method in which multiple images of a specimen container containing the serum or plasma portion of the specimen are acquired. Based on these multiple images, a convolutional neural network is used to determine the lipemic, hemolytic, or icteric nature of the serum or plasma region on a pixel-by-pixel or color-block basis. The results are then integrated within the serum or plasma region by a majority decision, for example, to determine the LHI of the specimen.

[0006] Prior art literature

[0007] Patent Literature

[0008] Patent Document 1: Japanese Patent Application No. 2020-519853 Summary of the Invention

[0009] Problems to be solved by the invention

[0010] However, due to the wide variety of conditions within sample storage containers and the internal state of the sample, even within the serum or plasma region of the same specimen, the image's hue and brightness may vary. For example, structures containing separation agents within the test tube, labels and stickers attached to the test tube, and printed text on the test tube may affect the hue and brightness of the serum or plasma image, or even obscure the serum or plasma region.

[0011] Furthermore, in serum or plasma, for example, when illumination is applied from above the specimen, the brightness of the serum or plasma region may be uneven, with the upper portion of the serum or plasma appearing brighter and the lower portion appearing darker. Thus, due to variations in the hue or brightness of the serum or plasma region due to factors other than the serum or plasma, resulting in unevenness, or due to the serum or plasma region being obscured, the serum or plasma region may contain an area unsuitable for determining the specimen's condition (e.g., LHI). When such an image of the serum or plasma region is used to determine the specimen's condition, conventional methods may adversely affect the specimen's condition determination results.

[0012] Therefore, an object of the present invention is to provide an apparatus and method for determining the state of a sample with high accuracy.

[0013] Means for solving problems

[0014] A sample state determination device according to one embodiment of the present invention includes: an image input unit that receives an image obtained by photographing a container storing a sample from the side; an area detection unit that determines an area of ​​serum or plasma from the image; a state determination appropriateness information calculation unit that calculates information for evaluating the appropriateness of state determination for the sample, i.e., state determination appropriateness information, based on a partial area of ​​the area of ​​serum or plasma determined by the area detection unit; a state determination area selection unit that selects a partial area suitable for state determination of the sample based on the state determination appropriateness information; a state determination unit that uses the partial area selected by the state determination area selection unit to determine the state of the sample; and an output unit that outputs the state determination result of the sample determined by the state determination unit and / or the position information within the image of the partial area selected by the state determination area selection unit.

[0015] Effects of the Invention

[0016] According to the sample state determination device of one aspect of the present invention, the state of the sample can be determined with high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a diagram showing an example of the hardware configuration of the sample state determination device of Example 1.

[0018] Figure 2 This is a diagram showing an example of a functional block diagram of the sample state determination device of Example 1.

[0019] Figure 3A This is a diagram showing an example of upper and lower end detection of serum in Example 1.

[0020] Figure 3B This is a diagram showing an example of upper and lower end detection of serum in Example 1.

[0021] Figure 4A This is a diagram showing an example of status determination appropriateness information in the first embodiment.

[0022] Figure 4B This is a diagram showing an example of status determination appropriateness information in the first embodiment.

[0023] Figure 5 This is a flowchart showing an example of a method for selecting a status determination area according to the first embodiment.

[0024] Figure 6 This is a diagram showing an example of screen display in Example 1.

[0025] Figure 7 This is a diagram showing an example of the processing flow of Example 1.

[0026] Figure 8 This is a diagram showing an example of a candidate graph and a state determination region in the first embodiment.

[0027] Figure 9 This is a diagram showing an example of a system configuration diagram of a sample analysis system according to Example 2.

[0028] Figure 10 This is a diagram showing an example of status determination appropriateness information in the first embodiment. DETAILED DESCRIPTION

[0029] Hereinafter, embodiments of the sample state determination device, method, and sample analysis system of the present invention will be described with reference to the accompanying drawings. In the following description and drawings, components having the same functional configuration are denoted by the same reference numerals, and redundant descriptions are omitted.

[0030] Example 1

[0031] use Figure 1 An example of a device for determining the state of a sample in an image according to the first embodiment will be described. Figure 1 An example of the hardware configuration of the sample state determination device 101 and peripheral devices is shown.

[0032] The sample state determination device 101 includes an interface 110, a computing unit 111, a memory 112, and a bus 113. The interface 110, computing unit 111, and memory 112 exchange information via the bus 113. Furthermore, the sample state determination device 101 exchanges signals with an imaging device 120 and a display device 121 via the interface 110.

[0033] Each component of the sample state determination device 101 will be described.

[0034] The interface 110 is a communication device that transmits and receives signals to and from devices outside the sample state determination device 101. Devices communicating with the interface 110 include an imaging device 120 and a display device 121. Details of the imaging device 120 and the display device 121 will be described later.

[0035] The arithmetic unit 111 is a device that executes various processes in the sample state determination device 101, and is, for example, a CPU (Central Processing Unit), an FPGA (Field-Programmable Gate Array), etc. Figure 2 This will be described later.

[0036] The memory 112 is a device for storing programs, parameters, coefficients, processing results, etc. executed by the calculation unit 111, and is an HDD, SSD, RAM, ROM, flash memory, etc.

[0037] The imaging device 120 is a device that images the specimen 130 from the side, and is, for example, a visible light camera or a hyperspectral camera. The imaging device 120 transmits the captured image to the specimen state determination device 101. Alternatively, the imaging device 120 may be replaced by a communication device that receives images via a network or the like, or a recording device that receives images by reading images recorded on a recording medium.

[0038] The display device 121 is a device for displaying the sample state determination result output by the sample state determination device 101 and / or position information of pixels or regions in an image used for sample state determination, and is, for example, a display or a printer.

[0039] Hereinafter, the sample state determination device 101 will be described in detail.

[0040] Figure 2 This is an example of a functional block diagram of the first embodiment of the sample state determination device 101. These functions can be implemented by the computing device, and each function in the computing device can be implemented by dedicated hardware or by the computing unit 111 and software running on the computing unit 111.

[0041] The sample state determination device 101 includes an image input unit 201, a serum (plasma) region detection unit 202, a state determination appropriateness information calculation unit 203, a state determination region selection unit 204, a state determination unit 205, and an output unit 206. Each unit will be described below.

[0042] The image input unit 201 receives an image input from the interface 110 , which is an image obtained by photographing a specimen to be subjected to condition determination from a side surface.

[0043] The serum (plasma) region detection unit 202 will be described.

[0044] The serum (plasma) region detection unit 202 detects a serum or plasma region from the specimen side image output by the image input unit 201. As a method for detecting a serum or plasma region, for example, there is a method for detecting the upper and lower ends of the serum or plasma region in the input specimen side image. As a detector, a machine learning method such as YOLO or SSD can also be used. Alternatively, manually designed feature quantities such as Hough transform can be effectively utilized to perform detection using a manually designed or machine learning-based detector.

[0045] Another example of a serum or plasma region detection method is region segmentation (semantic segmentation). Region segmentation is a method that divides an image into semantically distinct sets of pixels or small regions within the image. For example, an image can be segmented into serum or plasma regions, separator regions, blood clot regions, and label regions. As a region segmenter, machine learning methods such as U-Net and SegNet can be used, as well as methods such as the Watershed method.

[0046] Typically, high-precision region segmentation requires a long computation time, which reduces analysis throughput. Therefore, this article describes a method for detecting the upper and lower ends of serum or plasma as an example. Furthermore, in the case of a machine learning method, a detector is trained using a large number of combinations of input images and information on the correct upper and lower end positions of serum or plasma. The serum (plasma) region detection unit 202 uses a trained (learned) detector.

[0047] The serum (plasma) area detection unit 202 detects the upper and lower ends of serum or plasma through serum or plasma upper and lower end detectors, for example, extracts a partial image from the upper end of serum or plasma to the lower end of serum or plasma, and outputs it as a serum or plasma area image.

[0048] Figure 3A and 3B Shows the upper and lower end detection examples of serum. Figure 3A This is an example of serum 302 stored in container 301. Serum upper end detection result 303 and serum lower end detection result 304 indicate the upper and lower ends of the serum detected by the serum (plasma) upper and lower end detectors. Label 305 is a label attached to container 301 and contains a barcode and specimen information. Structure 306 is an example of a structure storing a separating agent, and scale 307 is a scale printed directly on the test tube.

[0049] Figure 3BThe serum region 308 is shown. The serum region 308 is an example of a partial image of the serum region extracted based on the serum upper end detection result 303 and the serum lower end detection result 304. Here, the serum region image 308 shows an example of a region with a height of y2-y1 and a width of x2-x1 extracted from the starting coordinates (x1, y1), with the vertical center coordinate of the serum upper end detection result 303 being y1, the vertical center coordinate of the serum lower end detection result 304 being y2, the left ends of the serum upper end detection result 303 and the right ends being x2. In this way, the serum (plasma) region detection unit 202 detects plasma or a plasma region and outputs a serum or plasma region image.

[0050] The state determination appropriateness information calculation unit 203 will be described.

[0051] The state determination adequacy information calculation unit 203 calculates information for each pixel or each small region used to select a region for state determination from the serum or plasma region image output by the serum (plasma) region detection unit 202. This information is referred to herein as state determination adequacy information. The state determination adequacy information calculation unit 203 may also calculate and output multiple pieces of state determination adequacy information.

[0052] Edge information, color appropriateness information, and distance information will be described as examples of state determination appropriateness information.

[0053] Edge information is information indicating changes in pixel values ​​(luminance value, RGB value, etc.) in a small area within an image, and can be calculated using, for example, a Sobel filter or a local variance value. Figure 4A An example of edge information extraction is shown. Edge information 401 indicates that Figure 3B An example of the edge of the serum region image 308. The closer to white it is, the more edge components it contains.

[0054] Edge information 401 contains strong edge components near the boundaries of label 305, barcode, scale 307, and structure 306. These areas may obscure the serum or plasma area, or the hue or brightness of the serum or plasma area may change, making them unsuitable for use as areas for state determination.

[0055] By using edge information, these areas can be excluded from candidate areas for state determination. A specific exclusion method will be described with reference to the state determination area selection unit 204. Furthermore, in order to detect areas containing many edge components and their surrounding areas, edge components may be amplified by dilation or filtering.

[0056] Color appropriateness information indicates the degree to which each pixel or small area within a serum or plasma region image fits within an appropriate range of color and / or brightness. Methods for calculating color appropriateness information include, for example, pre-collecting appropriate serum or plasma images (calibration data) and calculating a histogram of the colors at each pixel within the calibration data, or determining the appropriate color range on each axis or in space of a color system such as RGB, HSV, or HLS.

[0057] Here, a method using a color histogram is described as an example. Formula (1) represents an example of a calculation formula for color appropriateness information. Here, color(x, y) represents the color at coordinates (x, y) within the serum or plasma region image, and HIST(c) is a function that outputs the frequency of color c in the histogram of the calibration data calculated in advance. According to formula (1), the color appropriateness information CA represents color appropriateness, with high values ​​for colors that appear frequently in the calibration data and low values ​​for colors that appear infrequently.

[0058] CA(x, y)=HIST(color(x, y)) (1)

[0059] The color appropriateness information calculated in this way is effective for identifying areas similar to serum or plasma. For example, in many cases, labels or printed areas are different colors from serum or plasma, resulting in low color appropriateness. Furthermore, color appropriateness tends to decrease in areas where the brightness increases excessively due to proximity to lighting, areas where the brightness decreases excessively due to distance from lighting, or areas where the hue or brightness of the serum or plasma area changes due to foreign matter in the test tube.

[0060] Figure 4B This shows an example of color appropriateness information. The color appropriateness information 402 shows Figure 3B This is an example of calculating color appropriateness information for a serum region image 308. The closer to white the image, the higher the color appropriateness, and the closer to black the image, the lower the color appropriateness. For example, the color appropriateness of the label 305, scale 307, and background area is very low and is represented in black. Furthermore, the area containing the structure 306 appears gray due to variations in the hue and brightness of the serum in the image. This color appropriateness information is useful for selecting appropriate candidate regions for status determination by the status determination region selection unit 204, described later, when there are variations in color and brightness within the serum or plasma.

[0061] Next, we will explain distance information. Distance information is information that identifies reference points within an image, such as the top and bottom of the serum, or the position of the illumination, and indicates how far each pixel is from that reference point. For example, when the specimen is illuminated from above, the image tends to appear brighter closer to the top of the serum (closer to the illumination) and darker closer to the bottom of the serum (farther from the illumination).

[0062] As described above, since the position in the image is correlated with the hue and brightness value of the image, distance information may be output as one of the state determination appropriateness information. Figure 10 An example showing distance information. Figure 10 This shows an example where serum 302 is stored in container 301 and lighting 1001 is directed downwardly to illuminate serum 302. Furthermore, lighting 1001 is an example of a lighting position, and lighting can be set at other positions. Furthermore, the reference point for the distance is not limited to the lighting, and can be, for example, the top or bottom of the image, or an area or point detected from the image such as the serum or the top of the container can be used as the reference point. Figure 3A Likewise, as the serum upper end and the serum lower end, the serum upper end 303 and the serum lower end 304 are detected.

[0063] Origin 1002, origin 1003, and origin 1004 are examples of origins (reference points) used when calculating distance information. Origin 1002 represents the point closest to illumination 1001, origin 1003 represents the center point of serum upper end 303, and origin 1004 represents the center point of serum lower end 304. Distance information 1005, distance information 1006, and distance information 1007 are examples of distance information when origin 1002, origin 1003, and origin 1004 are used as the origins when calculating distance information. The closer to black, the shorter the distance, and the closer to white, the longer the distance.

[0064] Thus, the state determination suitability information calculation unit 203 may calculate the distance from an arbitrary point in the image as the origin to each pixel or each small area as the state determination suitability information. Alternatively, the distance from a reference line, such as a horizontal line at the upper / lower end of the serum, to each pixel or each small area may be calculated instead of a point.

[0065] The state determination area selection unit 204 will be described.

[0066] The state determination area selection unit 204 uses the state determination appropriateness information (group) calculated by the state determination appropriateness information calculation unit 203 to determine and output one or more pixels or areas for state determination by the state determination unit 205, described later. Here, the state determination area selection method using edge information, color appropriateness information, and distance information, as described above with respect to the state determination appropriateness information calculation unit 203, will be described as an example.

[0067] Figure 5 A flowchart showing an example of a method for selecting a status determination area. Figure 5 In the example shown, edge information and color appropriateness information are first used to narrow down candidate states for determination on a pixel-by-pixel basis, and distance information is then used to determine the final state determination region. Alternatively, only a portion of edge information, color appropriateness information, or distance information may be referenced, or only one or two of these information types may be referenced. For example, only color appropriateness information, or both color appropriateness information and edge information or distance information may be referenced. By increasing the types of referenced information, more accurate determinations can be made.

[0068] In step S501, the variables x and y representing the coordinates of the pixels or small areas to be screened, are initialized to 0, and a candidate map is generated and initialized. The pixel or small area is a portion of the image of the serum area 308, i.e., a partial area. The candidate map is temporary information used to determine the state-facing determination area, and is a two-dimensional array that records whether the candidate is suitable as the state-facing determination area. Here, the candidate map is a two-dimensional array of binary values ​​(0 or 1) with the same width and height as the serum or plasma area image. 1 is stored in pixels or areas that are judged to be suitable as the state-facing determination area, and 0 is stored in pixels or areas that are judged to be inappropriate. As an initialization, all values ​​are set to 0.

[0069] In step S502, it is determined whether the edge information at the coordinates (x, y) is less than a predetermined threshold value Th_e. If true (S502: Yes), the process proceeds to step S504; if false (S502: No), the process proceeds to step S503. In this step, as explained by the state determination appropriateness information calculation unit 203, areas with large edge components and their surrounding areas are highly likely to be affected by labels, printing, foreign matter, etc., and therefore such areas are excluded from candidates for state determination areas.

[0070] In step S503 , the value of the candidate image at the coordinate (x, y) is set to 0.

[0071] In step S504, a determination is made as to whether the color appropriateness information at the coordinates (x, y) is greater than a predetermined threshold value Th_c. If true (S504: Yes), the process proceeds to step S505; if false (S504: No), the process proceeds to step S503. In this step, only pixels or small areas with color appropriateness information exceeding a certain threshold value are extracted as areas or pixels suitable for face state determination.

[0072] In step S505 , the value of the candidate image at the coordinate (x, y) is set to 1.

[0073] In step S506, it is determined whether all pixels or all small regions in the serum or plasma region image have been processed. If true (S506: Yes), the process moves to step S508. If false (S506: No), the process moves to step S507.

[0074] In step S507, the coordinates of the next pixel or small area to be filtered are obtained. This coordinate acquisition can be performed, for example, by scanning from the upper left to the lower right of the image. If new coordinates are obtained, the process returns to step S502 and repeats until step S506 is true.

[0075] In step S508, the region for the final state determination is determined. Step S509 outputs the coordinates of the selected region. By the time step S508 is executed, the candidate selection for each pixel has been completed, and the candidate map is complete. Here, as an example, we will describe how distance information is used to determine the region for the final state determination.

[0076] First, as distance information, the distance to the upper end of serum or plasma and the distance to the lower end of serum or plasma are input. Figure 10 In the example of the lighting position shown, the area near the upper end of the serum tends to be brighter and the area near the lower end tends to be darker. Therefore, among the candidate areas, the area midway between the upper and lower ends of the serum is selected as the area for final status determination. In this way, the area midway between the closest and farthest positions relative to the lighting position can be selected as the area for final status determination.

[0077] Therefore, in this step, the candidate image is calculated as 1 and the absolute difference between the distance relative to the upper end of the serum or plasma and the distance relative to the lower end of the serum or plasma is calculated, and the pixel or small area in the serum or plasma area image corresponding to the coordinate with the smallest absolute difference value is selected as the pixel or small area for the final state judgment.

[0078] Figure 8 The candidate graph 801 is an example of a state determination area. Figure 4A and Figure 4B The edge information 401 and color appropriateness information 402 described in the above are input to Figure 5 This is an example of a candidate image obtained by the process of S508. In candidate image 801, black represents 0 and white represents 1. Areas around the edges with low color suitability (black) are excluded, and only the serum area suitable for state determination is extracted as a candidate (white). State determination-oriented area 802 is an example of a state determination-oriented area selected according to the process of S508. State determination-oriented area 802 within the serum or plasma area image is output as the area for the final state determination.

[0079] Thus, the state determination region selection unit 204 uses the state determination appropriateness information (group) calculated by the state determination appropriateness information calculation unit 203 to select and output pixels or small regions (partial regions) for state determination. As described above, the state determination appropriateness information calculation unit 203 may also calculate new information by processing or combining the state determination appropriateness information to select pixels or small regions for state determination. Furthermore, for example, in step S508, multiple pixels or regions may be extracted, such as the region closest to the upper end of the serum, the region closest to the lower end of the serum, and the region between them, for which candidate image 1 is selected.

[0080] Figure 5 The flowchart is merely an example, and it is not necessary to follow the flow in the flowchart. It is particularly important to select one or more pixels or areas for state determination using state determination appropriateness information (group), and the selection process is not limited to the example shown here.

[0081] The state determination unit 205 will be described.

[0082] The state determination is performed using the image of the state determination area selected by the state determination area selection unit 204. The state determiner is, for example, a machine learning model such as a neural network, support vector machine (SVM), or a decision tree. The image 802 of the partial state determination area is input to the machine learning model. Alternatively, manually designed features may be extracted from the image 802 of the state determination area, and determination may be performed manually or using a machine learning model based on the extracted features.

[0083] The output of the state determiner can be a real-valued estimate of each LHI indicator, a coarser-scaled classification of the LHI degree, or a binary value indicating whether the LHI exceeds a predetermined threshold (positive or negative). Furthermore, in the case of a machine learning method, the state determiner is trained using a large number of combinations of input images and correct information on the sample state, and the state determination region selection unit 204 uses the trained (learned) state determiner.

[0084] When a single state determination region is selected by the state determination region selection unit 204, the state determination region selection unit 204 outputs the determination result for that region. When multiple state determination regions are used, the recognition result is calculated based on, for example, a majority decision of the determination results for the multiple regions or the sum of the recognition scores.

[0085] In addition, one or more images obtained by statistically processing and integrating multiple areas for state determination can be used to perform state determination. In this way, a more appropriate determination can be made. For example, an average image can be used to perform state determination. The state determiner can also determine the state based on the input average image. The state determination can also be performed using differential images of multiple areas for state determination. For example, the differential image can be the difference between the maximum and minimum values ​​of pixels at the same position in each of the multiple areas. The state determiner can also determine the state based on the input differential image and any one of the multiple images or their average image. The differential image is particularly effective when the turbidity of the sample is taken into consideration for identification.

[0086] When the state determination area selection unit 204 outputs a small area, the state determination unit 205 may also perform LHI determination based on each pixel, i.e., the color and / or brightness of each pixel, and use the determination results of each pixel in the small area or statistical values ​​such as the majority decision, sum, average, weighted sum of the determination scores to calculate the final state determination result.

[0087] Alternatively, statistical processing such as the sum, average, or weighted sum of the values ​​of multiple pixels in a region may be performed, and this value may be used as the input to the state determiner. Alternatively, by using an image of a small region, that is, information on the position and value of pixels, as the input to the determiner, texture information (spatial information on the value (color and / or brightness) of the pixel) can be used as a feature to calculate the state determination result. This allows for more accurate determination. In addition, as for the weight of the weighted sum, a portion of the state determination appropriateness information calculated by the state determination appropriateness information calculation unit 203, such as color appropriateness information, may be used.

[0088] The output unit 206 will be described.

[0089] The output unit 206 outputs the state determination result of the sample output by the state determination unit 205 and / or the position information within the image of the state determination area output by the state determination area selection unit 204 .

[0090] Figure 6 This is an example of a case where the output result is displayed on the screen. Window 601 is a window that prompts the user, the state determination result display section 602 displays the sample state determination result, and the state determination area display section 603 displays the position of the state determination area in the input image as an image. However, the output format is not limited to Figure 6For example, the position information facing the status determination area can be displayed not as an image but as a string of start and end coordinates or a combination of start coordinates, width, and height. Alternatively, instead of displaying each specimen individually, a format such as listing the ID, name, status determination results, and coordinate information (such as start and end coordinates) facing the status determination area for multiple specimens can be used.

[0091] Alternatively, these displays may be recorded once in the memory 112 and displayed in an arbitrary format according to an instruction from the user.

[0092] In the above, the details of the embodiment 1 are described by functional blocks. However, the embodiment of the present invention does not necessarily have to be composed of Figure 2 The functional block structure can be realized as long as the processing of the actions of each functional block can be realized. Figure 7 An example of a processing flow chart of Example 1 is shown. Each step is Figure 2 The elements of the functional block diagram shown correspond to each other.

[0093] The image input step S701 receives an image input from the interface 110 , which is an image of a specimen to be subjected to condition determination, taken from the side.

[0094] The serum (plasma) region detection step S702 detects a serum or plasma region from the specimen image received in the image input step S701.

[0095] The state determination adequacy information calculation step S703 calculates information (state determination adequacy information) for evaluating the adequacy of the state determination based on the serum or plasma area calculated in the serum (plasma) area detection step S702.

[0096] The state determination region selection step S704 selects one or more partial regions for state determination from the serum or plasma region based on the state determination suitability information calculated in the state determination suitability information calculation step S703.

[0097] In the state determination step S705 , the state (LHI) of the sample is determined using the partial region for state determination selected in the state determination region selection step S704 .

[0098] The output step S706 outputs the state determination result of the sample calculated in the state determination step S705 and / or the position information of the partial area for state determination selected in the state determination area selection step S704.

[0099] According to the sample state determination device described in Example 1, the state of the sample can be determined with high accuracy even when the image contains factors that affect the appearance of the serum or plasma region or when the serum or plasma region is non-uniform.

[0100] Example 2

[0101] Example 2 describes a sample analysis system that uses the sample state determination device described in Example 1 to determine the state of the sample and controls the transport path of the specimen based on the sample state determination result.

[0102] Figure 9 A system configuration diagram showing Example 2.

[0103] The sample analysis system 901 includes: a photographing device 120 for photographing the specimen 130 from the side, the sample state determination device 101 described in Example 1, a specimen transporting device 902 for transporting the specimen 130, a specimen obtaining device 903 for obtaining a specimen from the specimen transported by the specimen transporting device 902, an analyzing device 904 for analyzing the specimen obtained by the specimen obtaining device 903, and a display device 121 for prompting the user with the sample analysis results of the analyzing device 904 and the sample state determination results of the sample state determination device 101 and / or position information facing the state determination area.

[0104] The imaging device 120 has the same hardware components as those of Example 1, and therefore, description thereof will be omitted. Next, the specimen transport device 902 , the sample acquisition device 903 , the analysis device 904 , and the display device 121 will be described.

[0105] The specimen transport device 902 is a device for transporting specimens, such as a belt conveyor for transporting specimens (test tubes containing samples) fixed to a bracket. The specimen transport device switches the transport path of each specimen based on the specimen status determination result of the specimen status determination device 101 for each specimen. Specifically, specimens determined to have a negative LHI or an LHI level / index value below a predetermined threshold are transported to the specimen acquisition device 903, and other specimens are transported to a temporary storage space, for example. Specimens transported to the temporary storage space are not analyzed by the analysis device 904, and their processing is determined by the user, or they are transported to other devices.

[0106] The sample acquisition device 903 is a device for acquiring a sample in the specimen transported by the specimen transport device 902 and is a suction nozzle or the like.

[0107] The analyzing device 904 is a device that performs biochemical analysis or immunoassay using a reaction solution obtained by reacting the sample acquired by the sample acquiring device 903 with a reagent.

[0108] The display device 121 displays the analysis result of the analyzer 904 , the sample state determination result of the sample state determination device 101 , and / or position information of the state determination area.

[0109] In the case of a specimen with a positive LHI, any of the LHI values ​​may affect the analysis results. With the above configuration, specimens determined by the specimen status determination device 101 to have a positive or high LHI index value are not sampled by the specimen acquisition device 903 or analyzed by the analyzer 904. This allows analysis to be performed only on normal specimens, preventing waste of specimen samples containing LHI and saving analysis time and reagents.

[0110] <Modification>

[0111] While the serum (plasma) region detection unit 202 has been described as extracting and outputting a plasma or plasma region image as serum or plasma region information, it is also possible to output, for example, an input image received from the image input unit 201, along with rectangular information and mask information representing the serum or plasma region. Alternatively, rather than acquiring the entire serum or plasma region, it is also possible to detect only the top of the serum or plasma region, for example, and extract the region at a predetermined height from the top of the serum or plasma region as the serum or plasma region. Furthermore, when using a region segmentation method, regions such as labels and prints may be determined at this point in time and excluded from the candidate regions.

[0112] In the state determination appropriateness information calculation unit 203, the color appropriateness information is described, but for example, in the case where there are images (training images, etc.) used to construct the state determiner used in the state determination unit 205, these images can also be used as calibration data. In this case, it is expected that the higher the value of the color appropriateness information, the higher the reliability of the result of the state determiner. In addition, when using a color histogram, etc. to determine the color, it is also possible to prepare determination methods for serum and plasma respectively. For example, if it is a color histogram, it is also possible to prepare calibration data collected only for serum and calibration data collected only for plasma, and to create a histogram for serum and a histogram for plasma. In addition, when performing determinations such as color similarity, any color system such as RGB, HSV, HLS, etc. can be used.

[0113] In the state determination area selection unit 204, the area for state determination can also be selected based on the value of any state determination appropriateness information. For example, the pixel or area with the largest color appropriateness in the candidate area can be selected as the area for state determination. Figure 5 In the example of scanning all pixels, the number of scans may be reduced under certain conditions, such as by proceeding to the next step when a certain number of candidate areas or more are found.

[0114] In addition, in at least one of the serum (plasma) area detection unit 202, the state determination appropriate information calculation unit 203, and the state determination unit 205, the results of color and brightness standardization, correction, filtering, Retinex and other image processing applied to the image to be processed can be used to perform serum (plasma) area detection, state determination appropriate information calculation, state determination, etc.

[0115] One example of image processing involves formulating the characteristics of illumination and then normalizing and correcting the hue and brightness values ​​within the image based on these characteristics. The illuminance received by each pixel from the illumination can be formulated based on the distance and angle of each pixel relative to the illumination. Based on this formulated illuminance, for example, it is possible to correct the brightness so that areas with low illumination have the same brightness as pixels with the highest illumination.

[0116] Furthermore, when using multiple lighting sources, color temperature may vary depending on the lighting source. When determining the sample's condition based on color, uniform color is preferred. Therefore, based on the specifications of each lighting source or pre-measured color temperature, for example, the hue shift in each pixel is formulated. By correcting the hue in each pixel based on this formulated hue shift, hue variations caused by lighting variations can be mitigated.

[0117] Alternatively, component separation processing such as Retinex can be applied. Retinex is a process that separates an object's constant color and brightness components from those affected by illumination. By extracting information about an object's constant color and brightness using Retinex, a corrected image can be obtained that mitigates differences in color and brightness caused by illumination at different coordinates.

[0118] By using such image processing to perform serum (plasma) area detection, state determination appropriateness information calculation, state determination, etc., more accurate detection and determination can be performed.

[0119] Furthermore, the present invention is not limited to the above-described embodiments and encompasses various variations. For example, the above-described embodiments are examples described in detail to facilitate understanding of the present invention and are not necessarily limited to all of the described structures. Furthermore, a portion of the structure of one embodiment can be replaced with a structure of another embodiment, and a structure of another embodiment can be added to a structure of one embodiment. Furthermore, with respect to a portion of the structure of each embodiment, other structures can be added, deleted, or substituted.

[0120] Furthermore, the aforementioned structures, functions, processing units, etc. may be partially or entirely implemented in hardware, for example, by designing them using integrated circuits. Furthermore, the aforementioned structures, functions, etc. may be implemented in software by having a processor interpret and execute programs that implement the respective functions. Information such as programs, tables, and files that implement the respective functions may be stored in a storage device such as a memory, a hard disk, or an SSD (Solid State Drive), or a recording medium such as an IC card or an SD card.

[0121] In addition, the control lines and information lines are those considered necessary for explanation and do not necessarily represent all the control lines and information lines on the product. In fact, it can be assumed that almost all the components are connected to each other.

Claims

1. A sample state determination device for determining the state of a sample stored in a container, characterized in that: The sample state determination device comprises: an image input unit that receives an image obtained by photographing the container storing the sample from a side surface; a region detecting unit for determining a serum or plasma region from the image; a state determination adequacy information calculation unit for calculating state determination adequacy information for evaluating the adequacy of the state determination for the sample, for each partial region of the serum or plasma region determined by the region detection unit; a state determination region selection unit for selecting a partial region suitable for state determination of the sample based on the state determination appropriateness information; a state determination unit configured to determine the state of the sample using the partial area selected by the state determination area selection unit; as well as An output unit outputs a state determination result of the sample determined by the state determination unit and / or position information of the partial area selected by the state determination area selection unit within the image.

2. The sample state determination device according to claim 1, characterized in that: The state determination adequacy information calculation unit calculates information on an edge of each of the partial regions as information included in the state determination adequacy information.

3. The sample state determination device according to claim 1, wherein: The state determination adequacy information calculation unit calculates the adequacy of the color or brightness of each of the partial regions for sample state determination as information included in the state determination adequacy information.

4. The sample state determination device according to claim 1, wherein: The state determination adequacy information calculation unit calculates a distance from a predetermined origin to the partial area as information included in the state determination adequacy information.

5. The sample state determination device according to claim 1, characterized in that: At least one of the area detection unit, the state determination appropriateness information calculation unit, and the state determination unit applies at least one of normalization and correction, filtering, and component separation processing to the image.

6. The sample state determination device according to claim 1, characterized in that: The state determination unit determines the state of the sample using texture information of the partial region.

7. The sample state determination device according to claim 1, characterized in that: The state determination area selection unit extracts a plurality of partial areas. The state determination unit determines the state of the sample using an image obtained by integrating the plurality of partial regions through statistical processing.

8. A method for determining the state of a sample stored in a container, wherein: The device receives an image obtained by photographing a container storing a sample from the side, determines an area of ​​serum or plasma from the image, calculates information for evaluating the appropriateness of a state judgment for the sample, i.e., state judgment appropriateness information, based on a partial area of ​​the determined area, selects a partial area suitable for state judgment of the sample based on the state judgment appropriateness information, determines the state of the sample using the selected partial area, and outputs the result of the judgment and / or position information of the selected partial area within the image.

9. The method for determining the state of a sample according to claim 8, wherein: The device calculates information on the edge of each of the partial areas as information included in the state determination appropriateness information.

10. The method for determining the state of a sample according to claim 8, wherein: The device calculates the suitability of the color or brightness of each of the partial areas for the specimen state determination as information included in the state determination suitability information.

11. The method for determining the state of a sample according to claim 8, wherein: The device calculates a distance from a pre-specified origin of the serum or plasma region within the image to the partial region as information included in the state determination appropriateness information.

12. The method for determining the state of a sample according to claim 8, wherein: The device determines the state of the sample using texture information of the partial area.

13. The method for determining the state of a sample according to claim 8, wherein: The device extracts a plurality of partial regions, The device determines the state of the sample using an image obtained by integrating the plurality of partial regions through statistical processing.

14. A sample analysis system for determining the state of a sample stored in a container and controlling the transport of the container storing the sample based on the determination result, characterized in that: The sample analysis system comprises: a photographing device for acquiring an image of the container storing the sample from a side view; The sample state determination device according to any one of claims 1 to 7, which determines the state of the sample in the image; a transport device for controlling a transport path of a container storing the sample based on a determination result of the state of the sample by the sample state determination device; a sample obtaining device for obtaining the sample from the container transported by the transport device; an analyzing device that analyzes the sample obtained by the sample obtaining device; and A display device displays the analysis result of the analyzer, the determination result of the sample state determination device, and / or position information within an image of a partial region used for determination of the sample state.

Images