Optical information reading device and method

Abstract

To provide an optical information reader which makes the background have a distinctive pattern in order to accurately detect a cap attached to a transparent container in an inspection device and efficiently distinguishes the container from the cap. [Solution] A device for detecting a cap attached to a container containing a sample or reagent, comprising an imaging unit that acquires an image of the container along with a pattern placed in the background of the container, and a control unit that decodes a code affixed to the container from the image, wherein the control unit recognizes a specific pattern including the pattern from the image and detects the cap based on the presence or absence of the specific pattern.

Description

[Technical Field]

[0001] The present disclosure relates to optical information reading devices and methods. [Background technology]

[0002] Test tubes and reagent bottles containing samples used in blood testing equipment and other specimen testing devices (automated biochemical analyzers) are usually capped to prevent the contents from leaking during transportation. The specimen testing device has a tub-shaped container into which workers place the uncapped test tubes or reagent bottles to perform biochemical testing.

[0003] The specimen testing device uses a probe (suction tube) attached to the tip of the dispenser to sequentially aspirate liquid from test tubes or reagent bottles and dispenses it into a mixing reaction vessel, causing the specimen and reagent to react and perform biochemical testing.

[0004] The probe (aspiration tube) is made of metal and is structured like a thin pipette, so it is prone to breaking or bending. When an operator sets a test tube or reagent bottle without removing the cap, the probe can hit the cap and be damaged.

[0005] To solve this problem, techniques are known from Patent Documents 1 and 2 that detect the presence or absence of a cap on a test tube (specimen container or reagent bottle) by image recognition.

[0006] Patent Document 1 describes a system for identifying test tubes and caps. In the tube identification sub-process, an image captured by an optical imager is used, and contrast is used in a retroreflective target area with a very bright background to extract features of the test tube and cap shapes, thereby detecting the cap of the test tube.

[0007] Patent Document 2 describes a system for identifying the caps of specimen containers. A specimen container including a cap is illuminated with a light source of different wavelengths at multiple exposure times and photographed. From the multiple photographed images, pixels with appropriate exposure for each wavelength are selected, and the pixels in the image are classified as belonging to each area, such as the transparent part of the test tube, the cap, or the label. From the classified data, the shape, capacity, characteristics, etc. of each area are identified, and the size of the test tube, the type of cap, the cap color, etc. are identified.

[0008] Furthermore, Patent Document 3 describes an automatic analyzer that can detect specimen containers even if they do not have barcode labels attached. A two-dimensional code is placed on the rear surface of the specimen container mounting section, and the two-dimensional code is read by a sensor 16. Whether a specimen container is mounted on the mounting section is determined based on whether the contrast value of light reflected from the two-dimensional code is equal to or greater than a preset threshold.

[0009] Furthermore, Patent Document 4 describes an optical information reader that uses a patterned background plate to properly read a two-dimensional code displayed on a mobile phone. A light-receiving sensor captures an image of the mobile phone inserted between the background plate and the sensor. From the image of the light-receiving sensor, changes in the black-and-white output signal for each coordinate are detected, and the position and movement speed of the mobile phone are detected. From the detected position and movement speed of the mobile phone, the image capturing conditions for reading the two-dimensional code are set. [Prior art documents] [Patent documents]

[0010] [Patent Document 1] US Patent Application Publication No. 2009 / 0324032 [Patent Document 2] International Publication No. 2018 / 022280 [Patent Document 3] International Publication No. 2021 / 182038 [Patent Document 4] Japanese Patent Application Laid-Open No. 2007-279950 Summary of the Invention [Problem to be solved by the invention]

[0011] The above-described conventional detection methods have the following problems.

[0012] In Patent Document 1, since the retroreflection target area is fixed, the position of the cap that can be distinguished is limited, and it is not possible to detect the caps of specimen containers of various heights.

[0013] In Patent Document 2, a multi-wavelength light source is used, which requires a high-performance sensor to detect the cap, complicating the identification process, and making it impossible to distinguish between caps and labels if they have the same wavelength (color) as the light source.

[0014] In Patent Document 3, the presence or absence of a specimen container is detected by a sensor using the transparent portion of the specimen container, but the automatic analyzer does not take into consideration the detection of the cap.

[0015] Patent Document 4 discloses a technique for obtaining coordinates from a background pattern, but does not contribute to the technique for detecting the cap of a specimen container.

[0016] Sample containers come in a variety of heights and thicknesses, and it is desirable for image identification to be able to accommodate all types. Furthermore, sample container caps come in a variety of colors, from light to dark, such as white, brown, and purple, and it is desirable for detection to be possible regardless of the cap color.

[0017] An object of the present invention is to provide an optical information reading device, method, and program that can more easily and reliably detect the caps of test tubes (specimen containers and reagent bottles) of various sizes, and non-limiting examples of the present disclosure contribute to providing an optical information reading device, method, and program that can accurately and easily identify the caps of test tubes (specimen containers and reagent bottles) of various sizes. [Means for solving the problem]

[0018] An optical information reading device according to one embodiment of the present disclosure includes an imaging unit that captures an image of a container to which a code is affixed and a pattern placed in the background of the container, and acquires the image, and a control unit that decodes the code from the image and detects whether or not the container has a cap based on the state of the pattern in the image.

[0019] In an optical information reading method according to one embodiment of the present disclosure, an optical information reading device captures an image of a container to which a code is affixed and a pattern placed in the background of the container, acquires an image, decodes the code from the image, and detects whether or not the container has a cap based on the state of the pattern in the image.

[0020] An optical information reading program according to one embodiment of the present disclosure causes an optical information reading device to perform the steps of capturing an image of a container to which a code is affixed and a pattern placed in the background of the container, acquiring the image, decoding the code from the image, and detecting whether or not the container has a cap based on the state of the pattern in the image.

[0021] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. [Effects of the Invention]

[0022] The optical information reading device of the present disclosure can detect and notify if a cap has been left on a test tube (specimen container, reagent bottle), thereby effectively preventing damage to the probe (aspiration tube).Furthermore, the optical information reading device of the present disclosure can use a background label with a pattern in which a certain design is repeated, which makes it easier to distinguish by image recognition and greatly improves the accuracy of distinction, making it possible to more easily and accurately detect if a cap has been left on a test tube.

[0023] Further advantages and benefits of an embodiment of the present disclosure will become apparent from the specification and drawings. Such advantages and / or benefits may be provided by some of the embodiments and features described in the specification and drawings, respectively, but not necessarily all of them may be provided to obtain one or more identical features. [Brief explanation of the drawings]

[0024] [Figure 1] 1 is a diagram showing an example of a specimen testing device in which an optical information reading device according to an embodiment of the present disclosure is installed; [Figure 2] 1 is a diagram showing an example of a specimen testing device in which an optical information reading device according to an embodiment of the present disclosure is installed; [Figure 3] 1 is a block diagram illustrating an example of a configuration of an optical information reading device according to an embodiment of the present disclosure. [Figure 4] 10A and 10B are diagrams illustrating an example of an image of a test tube using a background label captured by an optical information reading device according to an embodiment of the present disclosure. [Figure 5] 10A and 10B are diagrams illustrating an example of an image of a test tube using a background label captured by an optical information reading device according to an embodiment of the present disclosure. [Figure 6] 10A and 10B are diagrams illustrating an example of an image of a test tube using a background label captured by an optical information reading device according to an embodiment of the present disclosure. [Figure 7] 10A and 10B are diagrams illustrating an example of an image of a test tube using a background label captured by an optical information reading device according to an embodiment of the present disclosure. [Figure 8] FIG. 10 is a diagram illustrating an example of a background label according to an embodiment of the present disclosure. [Figure 9] 1A and 1B are diagrams illustrating examples of images captured by an optical information reading device according to an embodiment of the present disclosure. [Figure 10] 1A and 1B are diagrams illustrating examples of recognition areas and recognized patterns of an image captured by an optical information reading device according to an embodiment of the present disclosure. [Figure 11] 3A and 3B are diagrams illustrating examples of recognition areas and recognized patterns (A and B) of an image captured by an optical information reading device according to an embodiment of the present disclosure. [Figure 12] FIG. 10 is a flowchart illustrating a detection process using image recognition according to an embodiment of the present disclosure. [Figure 13] FIG. 13 is a diagram showing a flowchart (continuation of FIG. 12) of a detection process using image recognition according to an embodiment of the present disclosure. [Figure 14] FIG. 10 is a diagram illustrating a flowchart of a detection process using a trained machine learning model according to an embodiment of the present disclosure. [Figure 15] 10A and 10B are diagrams illustrating an example of an image of a test tube using a background label captured by an optical information reading device according to an embodiment of the present disclosure. [Figure 16] 10A and 10B are diagrams illustrating an example of an image of a test tube using a background label captured by an optical information reading device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0025] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters or redundant explanation of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following explanation and to facilitate understanding by those skilled in the art.

[0026] The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims. The embodiments described below are examples, and the embodiments to which the present disclosure is applied are not limited to the following embodiments.

[0027] <Embodiment> [Configuration of sample testing equipment] 1 and 2 show an example of a medical specimen testing device 2 (automatic biochemical analyzer) in which an optical information reading device 1 according to an embodiment of the present disclosure is installed. A tub-shaped turntable container 21 is installed so that it can rotate in the specimen testing device 2. This tub-shaped turntable container 21 is provided with a cooling function to maintain the quality of the specimen 63, and is provided with a lid 22 that can be attached to keep it cool.

[0028] A rack 25 is provided on the tub-shaped turntable container 21, and the rack 25 is provided with a plurality of holders in the circumferential direction, configured to allow a plurality of test tubes 6 (specimen containers or reagent bottles) to be attached. In addition, labels printed with barcodes 62 or two-dimensional codes for identifying the contained specimens or reagents are affixed to the test tubes 6. Slits and windows are provided in the tub-shaped turntable container 21 and the rack 25, and the optical information reader 1 is configured to be able to photograph the test tubes 6 and the labels affixed to the test tubes 6 from outside the tub-shaped turntable container 21.

[0029] An operator places test tubes 6 with their caps 61 removed in holders on rack 25 of this tub-shaped turntable container 21, and performs biochemical testing using the specimen testing apparatus 2. When the specimen testing apparatus 2 is activated, it sequentially aspirates specimens 63 and reagents according to the test items from the test tubes 6 using a probe (aspiration tube) 24 attached to the tip of a dispenser 23, and dispenses them into a reaction vessel for mixing, causing the specimens 63 to react with the reagents. The specimen testing apparatus 2 then performs biochemical testing by measuring the absorbance and turbidity of the reaction products using a photometer, and quantifying the target components in the specimens 63.

[0030] [Functions of optical information reading device] 1 and 2, the optical information reading device 1 is a 2D imager installed in the specimen testing device 2. As will be described below, the optical information reading device 1 has the functions of (1) a barcode reader and (2) detecting whether a cap has been left on the specimen.

[0031] The optical information reading device 1 sequentially acquires images of the test tubes 6 set in the holders of the rack 25 of the tub-shaped turntable container 21, reads the barcodes 62, two-dimensional codes, etc. affixed to the test tubes 6, and decodes the contents of the barcodes 62, two-dimensional codes, etc. The optical information reading device 1 transfers the contents of the barcodes 62, two-dimensional codes, etc. of the test tubes 6 that it has read and decoded to the specimen testing device 2. The specimen testing device 2 identifies the specimen 63 to be tested from the contents of the barcodes 62, two-dimensional codes, etc. of the test tubes 6 that it has received.

[0032] As mentioned above, if an operator forgets to remove the cap 61 and sets the test tube 6 in the holder of the rack 25 of the tub-shaped turntable container 21, the probe (suction tube) 24 may hit the cap 61, causing the probe (suction tube) 24 to break or bend, resulting in damage.

[0033] In order to detect the case where the cap 61 has been left unremoved, which is the cause of such damage to the probe 24, and notify the operator, the optical information reading device 1 recognizes the acquired image of the test tube 6 and detects the presence or absence of the cap 61 on the test tube 6. When the optical information reading device 1 detects the cap 61 on the test tube 6, it transfers information about the cap detection to the specimen testing device 2 and notifies the operator using a buzzer, LED indicator, or the like. Upon receiving the notification, the operator can know that the cap 61 on the test tube 6 has been left unremoved. Furthermore, the specimen testing device 2 may be configured to stop the biochemical test when it receives information about the cap detection.

[0034] <Optical information reader> 3 is a block diagram showing an example of the optical information reader 1. This optical information reader 1 is made up of a camera unit 3, a processing unit 4, and a sub-board 5. This configuration will be described below.

[0035] [Camera section] The camera unit 3 includes an imaging unit 31, a control unit 32, an LED illumination drive circuit 33, and an LED illumination unit .

[0036] The imaging unit 31 is composed of an imaging lens 311 and a CMOS (Complementary Metal Oxide Semiconductor) image sensor 312. The CMOS image sensor 312 may be a grayscale CMOS image sensor or a color CMOS image sensor. The imaging lens 311 of the imaging unit 31 is, for example, an optical lens, and forms an image of an object on its optical axis on the imaging area of ​​the CMOS image sensor 312. This allows the CMOS image sensor 312 to acquire image data of the object to be imaged.

[0037] Control unit 32 is a general-purpose microprocessor that controls the entire camera unit 3 by controlling imaging unit 31 of camera unit 3 and LED illumination drive circuit 33 that drives LED illumination 34. Control unit 32 may be realized, for example, by an MCU (Micro Controller Unit) or the like that integrates peripheral functions such as a central processing unit (CPU) and a storage unit (memory) into one unit.

[0038] The LED illumination drive circuit 33 is a circuit that drives an LED illumination 34 for illuminating an object to be imaged.

[0039] The LED lighting 34 irradiates the object with light, illuminating the object, thereby enabling the imaging unit 31 to capture clear image data even in a dark, enclosed space.

[0040] [Processing section] The processing unit 4 is composed of a control unit 41, a storage unit 42, an indicator control circuit 43, and a status LED 44.

[0041] The control unit 41 is a processor, and controls the overall operation of the optical information reader 1. The control unit 41 may be realized by, for example, a central processing unit (CPU (Central Processing Unit)).

[0042] The control unit 41 operates as a barcode reader by loading software having the function of reading and decoding the barcode 62 from the storage unit 42. The control unit 41 reads the barcode 62 from the image data acquired by the camera unit 3, determines the coding system, and decodes the data of the barcode 62.

[0043] The control unit 41 also operates as a functional unit that detects, through image recognition, whether the cap 61 has been left on, by reading from the storage unit 42 software having a function of detecting, through image recognition, whether the cap 61 has been left on. Details of this process will be described later in the flowchart of the detection process using image recognition.

[0044] The storage unit 42 is composed of a main storage unit such as a dynamic random access memory (DRAM) for storing data and commands read by the control unit 41, and an auxiliary storage unit such as a flash memory that can stably store information such as programs even when the power is turned off. The programs stored in the storage unit 42 cause the control unit 41 to operate as a functional unit that runs software that realizes each function. The storage unit 42 also stores image data input from the camera unit 3, data necessary for image recognition processing and detection processing, and the like.

[0045] The indicator control circuit 43 is a control circuit for driving the LED of the status LED 44 as an indicator in accordance with the operating state of the optical information reading device 1 and the detection results of the image data acquired by the camera unit 3. For example, when it is detected from the image acquired by the camera unit 3 that the cap 61 of the test tube 6 has been left unremoved, the indicator control circuit 43 lights up the status LED 44 in a predetermined manner.

[0046] The status LED 44 is composed of multiple LEDs and is used to notify workers, etc. of the operating status of the optical information reading device 1 and the detection results of image recognition on image data acquired by the camera unit 3 (such as forgetting to remove the cap 61).

[0047] [Sub-board] The sub-board 5 includes a communication unit 51 and a buzzer 52 .

[0048] The communication unit 51 connects to and communicates with the sample testing apparatus 2 (host) via RS-232C, USB, or the like.

[0049] The buzzer 52 notifies the operator or the like of the operating state of the optical information reader 1 and the detection results of the image recognition acquired by the camera unit 3 (such as forgetting to remove the cap 61) by sounding a notification sound.

[0050] <Image acquisition using an optical information reader> The optical information reader 1 is installed in the specimen testing apparatus 2 with the optical axis of the imaging lens 311 facing upward, and acquires images of each test tube 6 set in a holder on the rack 25 of the tub-shaped turntable container 21 of the specimen testing apparatus 2 (see FIGS. 4 to 7). Each test tube 6 set in the holder is imaged one by one by the optical information reader 1, and a vertically long image visible through the gap in the holder is acquired (see FIG. 9).

[0051] Background Label When installing the optical information reading device 1, the installer attaches a background label 10 to the backside of the lid 22 of the tub-shaped turntable container 21 (see FIG. 4). With this background label 10, when the optical information reading device 1 captures an image of each test tube 6 at an elevation angle, the pattern of the background label 10 attached to the lid 22 is captured in the background of the captured image. For example, the background label 10 may be one with a checkered pattern printed on it, as shown in FIG. 8 (1). This background label 10 is used to improve the accuracy of detection processing using image recognition and to facilitate image recognition.

[0052] Next, following the examples shown in FIGS. 4 to 7, imaging patterns will be sequentially explained for the cases where the cap 61 is not attached to the test tube 6 and where the cap 61 is attached.

[0053] [If the test tube does not have a cap] 4 and 5 are diagrams showing examples of images of a test tube taken by the optical information reading device 1 when a cap 61 is not attached to the test tube 6. FIG.

[0054] The area indicated by A in Figure 4 is above the test tube 6, and is the area in which the background label 10 is imaged without any obstructions (see Figure 10(1)A and Figure 11A(a)). The background label 10 is imaged with a clear pattern of repeated patterns.

[0055] Next, the range indicated by B in Fig. 4 is the range in which the background label 10 is imaged after passing through the test tube 6 (see Fig. 10(1)B and Fig. 11B). In the range indicated by B, the light passing through the transparent glass or plastic of the test tube 6 is bent and refracted, so that the repeated pattern of the background label 10 is distorted (see Fig. 11B(c)), the contrast of the repeated pattern of the background label 10 is reduced (see Fig. 11B(b)), or part of the repeated pattern of the background label 10 is hidden by refracted light or reflected light (see Fig. 11B(d) and Fig. 11B(e)).

[0056] The area below that, indicated by C in Fig. 4, is the area in which the barcode 62 label attached to the test tube 6 is imaged (see Fig. 10(1)C). In other words, the area indicated by C is the area in which the repeated pattern of the background label 10 is completely blocked by the smooth label surface on which the barcode 62 and characters are printed.

[0057] When the test tube 6 is imaged by the optical information reading device 1 as shown in Figures 4 and 5, the image is captured as shown in (2), (4), (6), or (8) of Figure 9. The test tubes in (2) and (4) and (6) and (8) of Figure 9 are different in size. Note that (1) of Figure 10 is the same image as (4) of Figure 9.

[0058] [If the test tube has a cap attached] 6 and 7 are diagrams showing examples of images of a test tube 6 attached with a cap 61 by the optical information reader 1. FIG.

[0059] The range indicated by A in FIG. 6 is the range in which the background label 10 is imaged without any obstructions (see FIG. 10(2)A).

[0060] Next, the range indicated by D in Figure 6 is the range in which the cap 61 attached to the test tube 6 is imaged (see Figure 10(2)D). It can be seen that the cap 61 has grooves and surfaces on the gripping portion to prevent slipping. In other words, the range indicated by D is the range in which the cap 61 completely blocks the repeated pattern of the background label 10.

[0061] The area below that, indicated by B in FIG. 6, is the area in which the background label 10 is imaged through the test tube 6 (see FIG. 10(2)B).

[0062] Moreover, the range indicated by C in FIG. 6 is a range in which the label of the barcode 62 attached to the test tube 6 is imaged, but the background label 10 is not imaged (see FIG. 10(2)C).

[0063] When the test tube 6 is imaged by the optical information reading device 1 as shown in Figures 6 and 7, the image is captured as shown in (3), (5), (7), or (9) of Figure 9. The test tubes in (3) and (5) and (7) and (9) of Figure 9 are different in size. Note that (2) of Figure 10 is the same image as (5) of Figure 9.

[0064] [If no test tube is set] When no test tubes 6 are set in the holders of the rack 25, the image captured by the optical information reader 1 is as shown in (1) of Figure 9 or (3) of Figure 10. The image includes an area where the background label 10 is imaged from above without any obstructions (see Figure 10(3)A), an area where the inner surface of the rack 25 is imaged as a dark space (see the upper side of Figure 10(3)E), and an area where a small barcode is imaged (see the lower side of Figure 10(3)E). The small barcode shown at the bottom of Figure 10(3)E is a special barcode placed on the back of the rack 25 or the like, and indicates that "no test tubes are set."

[0065] Test tubes 6 (specimen containers) come in a variety of sizes, and many different types of caps 61 with different colors and shapes are also used. Furthermore, the same test tube 6 is imaged differently depending on whether the test tube 6 is set in a holder on the inner periphery of the rack 25 or in a holder on the outer periphery of the rack 25, due to the difference in distance from the imaging unit 31. In this way, the images of the test tube 6 that are imaged vary. As described above, by using the features of the regions classified as A to E, it is possible to easily determine whether or not the cap 61 is attached to the test tube 6 in various images.

[0066] [Summary of characteristics of areas classified as A to E] Based on the above, the characteristics of the areas classified as A to E can be summarized as follows: A: The range in which the background label 10 is imaged without any obstructions (the area in which the background label 10 is imaged with a clear pattern of repeated patterns). B: This is the range in which the background label 10 is imaged after passing through the test tube 6 (light passing through the transparent glass or plastic of the test tube 6 is bent and refracted, so the repeated pattern of the background label 10 becomes distorted, the contrast of the repeated pattern of the background label 10 becomes reduced, or part of the repeated pattern of the background label 10 is hidden by refracted light or reflected light in the image). C: This is the range in which the background label 10 is obscured by the label on which the barcode 62 is printed (this is the area in which the repeating pattern of the background label 10 is completely obscured by the smooth label surface on which the barcode 62 and characters are printed). D: The range in which the background label 10 is imaged when it is blocked by the cap 61 (the area in which the repeated pattern of the background label 10 is imaged when it is completely blocked by the cap 61). E: This is the range in which the background label 10 is not visible because it is outside the angle of view in which it is imaged, and the inner surface of the rack 25 is imaged as a dark space, or a special barcode indicating that "no test tubes are set" is imaged in the background (this is the range in which the background label 10 is outside the angle of view in which it is imaged, and the repeated pattern of the background label 10 is not imaged, and the inner surface of the rack 25 is imaged as a dark space in the background, or a small barcode is imaged in the background).

[0067] The smooth label surface with a barcode 62 or text printed on it in C and the cap 61 in D can be distinguished by whether or not text or a code is printed on it, as well as by the differences in the texture, smoothness, and shape of the surface. C and D can also be distinguished by whether or not a barcode is included in the area to be classified.

[0068] Furthermore, using a background label 10 with a repeating pattern as the background makes it easier to distinguish the features of the image compared to using a plain background. In particular, it is easier to distinguish unobstructed space (a feature of areas classified as A), curvature and refraction of light (a feature of areas classified as B), and obstruction by labels or caps (a feature of areas classified as C or D), and the accuracy of the distinction is greatly improved.

[0069] <Flowchart of detection process using image recognition> As described above, the control unit 41 performs image recognition on the image acquired by the camera unit 3, and detects the presence or absence of the cap 61 on the test tube 6 based on the features of each area recognized as A to E. These image recognition processes and detection processes will be explained below in order.

[0070] 12 and 13 are flowcharts of the detection process using image recognition executed by the optical information reading device 1. The optical information reading device 1 executes the process of the flowcharts shown in FIGS. 12 and 13 for each test tube 6 to be detected.

[0071] In S1 (step 1), the camera unit 3 captures an image of the test tube 6 to be detected. The control unit 41 reads (decodes) the barcode 62 from the image data acquired by the camera unit 3.

[0072] In S2, the control unit 41 starts scanning from the top of the image data acquired by the camera unit 3 (see FIG. 10).

[0073] In S3, the control unit 41 executes image recognition processing on the recognition area of ​​the image data and recognizes whether the recognition area is classified into A to E. A "recognition area" is a partial area of ​​an image for which image recognition processing is performed in one scan.

[0074] In S4, the control unit 41 stores the recognition results (any of A to E) for each recognition area as a pattern in the storage unit 42. A "pattern" is a set of the recognition results for each recognition area.

[0075] In S5, the control unit 41 moves the recognition area of ​​the image data to the next recognition area.

[0076] In S6, the control unit 41 determines whether the scanning has reached the bottom of the image data (whether the moved recognition area has reached the bottom of the image data).

[0077] In S6, if the control unit 41 determines that the bottom of the image data has been scanned (YES in S6), the flow proceeds to S7. On the other hand, in S6, if the control unit 41 determines that the bottom of the image data has not been scanned (NO in S6), the flow returns to S3, and the processes of S3 to S5 are continued.

[0078] In S7, control unit 41 reads the saved pattern from storage unit 42, and if there are multiple consecutive identical recognition results, performs a process of combining them into one. For example, as shown in (1) of Fig. 10, when image data is processed for each of a number of recognition areas, if the recognition results of the first 1st to 11th recognition areas are all A, the recognition results of the 12th to 22nd recognition areas are all B, and the recognition results of the 23rd to 51st recognition areas are all C, control unit 41 performs a process of combining the results of the 1st to 11th recognition areas (pattern: AA...A) into one A, the results of the 12th to 22nd recognition areas (pattern: BB...B) into one B, and the results of the 23rd to 51st recognition areas (pattern: CC...C) into one C. As a result of this process, the combined pattern becomes ABC.

[0079] In S8, the control unit 41 determines whether the patterns compiled in S7 are, from the top, ABC or BC.

[0080] In S8, if the control unit 41 determines that the patterns grouped in S7 are ABC or BC from top to bottom (YES in S8), the flow proceeds to S9. On the other hand, if the control unit 41 determines that the patterns grouped in S7 are neither ABC nor BC from top to bottom (NO in S8), the flow proceeds to S10.

[0081] In S9, the control unit 41 executes a process to notify that "test tube present, cap absent." For example, the control unit 41 may execute the process to notify that "test tube present, cap absent" by causing the indicator control circuit 43 to light up the status LED 44 in a predetermined manner, causing the buzzer 52 to emit a predetermined notification sound, or by causing the communication unit 51 to transfer information to the sample testing apparatus 2 and display it on the display means of the sample testing apparatus 2.

[0082] In S10, the control unit 41 determines whether the pattern collected in S7 is ADC, DC, DBC, or ADBC.

[0083] In S10, if the control unit 41 determines that the pattern summarized in S7 is ADC, DC, DBC, or ADBC ​​(YES in S10), the flow proceeds to S11. On the other hand, if the control unit 41 determines that the pattern summarized in S7 is not ADC, DC, DBC, or ADBC ​​(NO in S10), the flow proceeds to S12.

[0084] In S11, the control unit 41 executes a process of notifying “test tube present, cap present.” For example, the control unit 41 may execute a process of notifying “test tube present, cap present” in the same manner as in S9.

[0085] In S12, the control unit 41 determines whether the content of the barcode 62 read in S1 is data on the test tube 6 or not.

[0086] In S12, if the control unit 41 determines that the content of the barcode 62 read in S1 is data on the test tube 6 (YES in S12), the flow proceeds to S13. On the other hand, if the control unit 41 determines that the content of the barcode 62 read in S1 is not data on the test tube 6 (NO in S12), the flow proceeds to S15.

[0087] In S13, the control unit 41 executes a process of notifying “test tube present, cap unknown.” For example, the control unit 41 may execute a process of notifying “test tube present, cap unknown” in the same manner as in S9.

[0088] In S14, the control unit 41 determines the height of the test tube 6 based on the height of part A. Test tubes 6 used as specimen containers come in heights of 100 mm, 75 mm, and 66 mm, with small containers (test cups) having a height of 25 mm. Since the height of the test tube 6 can be selected from multiple heights, it is possible to determine the height of the test tube by calculating the proportion of the recognition area occupied by the area classified as A to the total recognition area of ​​a specified length.

[0089] In S15, the control unit 41 determines whether the patterns compiled in S7 include E. At this time, it may be determined that the pattern E is included in the image based on the fact that the content of the barcode 62 read in S1 is a special barcode indicating that "no test tube is set."

[0090] In S15, if the control unit 41 determines that the patterns compiled in S7 include E (YES in S15), the flow proceeds to S16. On the other hand, in S15, if the control unit 41 determines that the patterns compiled in S7 do not include E (NO in S15), the flow proceeds to S17.

[0091] In S16, the control unit 41 executes a process of notifying “no test tubes.” For example, the control unit 41 may execute a process of notifying “no test tubes” in the same manner as in S9.

[0092] In S17, the control unit 41 executes a process of notifying "unknown." For example, the control unit 41 may execute a process of notifying "unknown" in the same manner as in S9.

[0093] In S3, the image recognition process performed by the control unit 41 may be performed by having the manufacturer or user register a plurality of actual image data corresponding to each recognition result (any of A to E) in advance when the device is manufactured or before use, and the control unit 41 may compare the registered image data with the acquired image data to recognize the appearance of a feature (classified as any of A to E) from the image data. As the image recognition process, a known method such as a pattern matching method, a contrast comparison method, or a method using SSIM (Structural SIMilarity) may be used.

[0094] The CMOS image sensor 312 used in the imaging unit 31 of the optical information reading device 1 of the present disclosure can perform the discrimination process in the same way whether it is a grayscale CMOS image sensor or a color CMOS image sensor. Note that, if the CMOS image sensors have the same number of pixels, a grayscale CMOS image sensor can achieve higher resolution than a color CMOS image sensor.

[0095] The optical information reading device 1 of the present disclosure is an optical information reading device that has been provided with a function for detecting caps in a device that has been widely used as a barcode reader. Conventionally known barcode readers are not limited to those installed in specimen testing devices 2, but are highly versatile devices that are generally used in a variety of situations to read barcodes. As for barcodes, in addition to one-dimensional codes, two-dimensional codes such as stacked and matrix two-dimensional codes may also be used. In addition to barcodes, specific fonts and the like can also be read.

[0096] <Effects> According to the optical information reading device 1 of the present disclosure, damage to the probe 24 (aspiration tube) can be effectively prevented by detecting and notifying that a cap 61 of a test tube 6 (specimen container, reagent bottle) has been left unremoved. Furthermore, according to the optical information reading device 1 of the present disclosure, by using a background with a pattern in which a certain design is repeated as the background label 10, discrimination by image recognition can be facilitated and discrimination accuracy can be greatly improved, so that a cap 61 of a test tube 6 can be detected more easily and with higher accuracy. Furthermore, the optical information reading device 1 of the present disclosure is highly versatile and cost-effective because a function for detecting a cap can be easily implemented by simply changing the software without modifying the hardware of a device that has previously been used as a barcode reader.

[0097] <Variation 1> A trained machine learning model may be used as the image recognition functional unit of the control unit 41 included in the processing unit 4 in the above-described embodiment. In this modification 1, the trained machine learning model is configured to perform image recognition processing of the optical information reading device 1. Below, a description will be given of modification 1 using the trained machine learning model.

[0098] When using a trained machine learning model for image recognition processing, it is essential that the manufacturer or user train the machine learning model when manufacturing the device or before use. Therefore, the following describes (1) the machine learning model training process and (2) the detection process using the trained machine learning model.

[0099] (1) Learning process for machine learning models The learning of the machine learning model will be explained below.

[0100] The machine learning model is trained using an information processing device. As the information processing device, the processing unit 4 of the optical information reading device 1 may be used, or a commonly used computer may be used. In the training process, the information processing device trains the machine learning model using training data consisting of image data for each recognition area and a recognition result (any of A to E) for each recognition area. The image data for each recognition area and the recognition result (any of A to E) for each recognition area may be data obtained by the processes described in S1 to S6 in FIG. 12 described above. Through this training process, the machine learning model is trained so that it can estimate which area, A to E, the image data for each recognition area belongs to.

[0101] The machine learning model used here may be realized by any appropriate machine learning model, for example, a machine learning model such as a convolutional neural network, or a known machine learning model (object detection library) such as YOLO. When YOLO is used, the detection result is a class and a bounding box (coordinates of the recognition area), and learning is performed so that the output class is one of the above-mentioned A to E.

[0102] (2) Detection processing using trained machine learning models The detection process using the trained machine learning model obtained as described above will now be described.

[0103] (Flowchart of detection process using trained machine learning model) The control unit 41 operates the trained machine learning model by reading software that executes the trained machine learning model from the storage unit 42. Figure 14 is a flowchart of the detection process using the trained machine learning model executed by the optical information reading device 1. The control unit 41 executes the process shown in Figure 14 for each test tube 6 to be detected.

[0104] In S21, the camera unit 3 captures an image of the test tube 6 to be detected. The control unit 41 reads (decodes) the barcode 62 from the image data acquired by the camera unit 3. This step is the same as S1 in FIG. 12 described above.

[0105] In S22, the trained machine learning model recognizes the image for each recognition area in the image data acquired by the camera unit 3, and recognizes which of the areas classified as A to E each recognition area belongs to.

[0106] In S23, the control unit 41 sorts the recognition results for each recognition region obtained by the trained machine learning model in the height direction of the test tube 6.

[0107] In S24, the control unit 41 executes a process of consolidating multiple consecutive identical recognition results into one in a set (pattern) of recognition results sorted in the height direction of the test tubes 6. The consolidation method is the same as that in S7 of FIG.

[0108] S25 to S34 executed by the control unit 41 may be executed in the same manner as S8 to S17 in FIG. 13 described above.

[0109] When image recognition processing is used, classification (as one of A to E) depends on the position of the recognition area, whereas when a trained machine learning model is used, image recognition is performed and classification (as one of A to E) is performed regardless of the position of the recognition area, which makes it possible to achieve a higher classification rate than when image recognition processing is used.

[0110] <Variation 2> 15 shows an example of imaging when the optical information reader 1 is installed in a specimen testing device with the optical axis of the imaging lens 311 facing downward, and a background label 10 is affixed to the rotation axis 26 of the tub-shaped turntable container 21. The optical information reader 1 and background label 10 may be installed in this manner. With this installation, the area above the test tube 6 is clearly imaged.

[0111] <Variation 3> 16 shows an example of imaging when the optical information reader 1 is installed in the specimen testing device with the optical axis of the imaging lens 311 facing downward, and the background label 10 is affixed to the back of the rack 25 of the tub-shaped turntable container 21. The optical information reader 1 and background label 10 may be installed in this manner. With this installation, not only is the area above the test tube 6 clearly imaged, but the pattern of the background label 10 is also clearly imaged.

[0112] <Variation 4> As explained in the section [Background Label] above, the background label may be one printed with a checkered pattern as shown in (1) of Fig. 8, but other patterns such as a repeating pattern of a certain design may also be used as the background label as shown in (2) to (6) of Fig. 8. Note that any pattern in which a certain design is repeated may be used, and is not limited to those shown in (2) to (6) of Fig. 8.

[0113] Also, patterns that characteristically change depending on the location ((7) to (8) in FIG. 8) may be used. For example, the pattern of the background label may be configured so that the pattern on the side farther away from the imaging unit 31 is larger and the pattern on the closer side is smaller, so that the size of the pattern recognized by the imaging unit 31 is constant regardless of distance.

[0114] The pattern that changes characteristically depending on the location can be anything that can be said to change characteristically depending on the location, such as a pattern in which the darkness, density, size, etc. of the pattern changes continuously, or a pattern in which the pattern changes continuously and in stages to form a gradation as a whole, and is not limited to those shown in (7) to (8) of Figure 8.

[0115] Although the embodiments have been described above with reference to the drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications or alterations within the scope of the claims. It is understood that such modifications or alterations also fall within the technical scope of the present disclosure. Furthermore, the components in the embodiments may be combined in any manner without departing from the spirit of the present disclosure.

[0116] In the above-described embodiments, the notation "... part" used for each component may be replaced with other notations such as "... circuitry," "... assembly," "... device," "... unit," or "... module." Calculation may be read as computation.

[0117] The present disclosure can be realized by software, hardware, or software linked to hardware. Each functional block used in the description of the above embodiments may be partially or entirely realized as an LSI, which is an integrated circuit, and each process described in the above embodiments may be partially or entirely controlled by a single LSI or a combination of LSIs. The LSI may be composed of individual chips, or may be composed of a single chip that includes some or all of the functional blocks. The LSI may have data input and output. Depending on the degree of integration, the LSI may be called an IC, system LSI, super LSI, or ultra LSI.

[0118] The integrated circuit method is not limited to LSI, but may be realized by a dedicated circuit, a general-purpose processor, or a dedicated processor. Also, a field programmable gate array (FPGA) that can be programmed after LSI manufacturing, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells within the LSI, may be used. The present disclosure may be realized as digital processing or analog processing.

[0119] Furthermore, if an integrated circuit technology that can replace LSI emerges due to advances in semiconductor technology or other derivative technologies, it is natural that such technology may be used to integrate functional blocks. The application of biotechnology, etc. is also a possibility. [Industrial Applicability]

[0120] The present disclosure is effective for detecting the presence or absence of a cap on a container that contains a specimen or a reagent, for example. [Explanation of symbols]

[0121] 1. Optical information reading device 2. Sample testing equipment (automatic biochemistry analyzer) 3 Camera section 4 Processing section 5 Sub-board 6 test tubes 10 Background Label 21 Tub-shaped turntable container 22 Turntable container lid 23 Dispenser 24 Probe (suction tube) 25 racks 26 Rotation axis 31 Imaging unit 311 Imaging Lens 312 CMOS image sensor 32 Control unit (MCU) 33 LED lighting driver circuit 34 LED lighting 41 Control unit (CPU) 42 Memory section (main memory section, auxiliary memory section) 43 Indicator control circuit 44 Status LED 51 Communications Department 52 Buzzer 61 Cap 62 Barcode 63 specimens

Claims

1. an imaging unit that captures an image of the container to which the code is attached and a pattern arranged in the background of the container, and acquires the image; a control unit that detects the presence or absence of a cap on the container by decoding the code from the image and classifying each area according to the state of the pattern in the image; Equipped with Optical information reading device.

2. The control unit is Classifying the regions of the image according to the state of the pattern, and detecting the presence or absence of the cap of the container based on a pattern that is a combination of a plurality of the classified regions.

2. The optical information reader according to claim 1.

3. The classified regions are: a first region in which the pattern is clearly captured; a second region in which the pattern is imaged in a distorted, reduced contrast, or hidden by refracted or reflected light; a third area in which the pattern is not captured due to the label surface on which the code or characters are printed; A fourth area in which the pattern is not captured by the cap, or a fifth region in which the background is captured while the pattern is not captured because the pattern is outside the angle of view; Either 3. The optical information reader according to claim 2.

4. The control unit is detecting the presence of the cap on the container when the pattern includes the fourth region, the fourth region being located at the beginning of the pattern or located below the first region, and the fourth region being located above the second region or the third region; 4. The optical information reader according to claim 3.

5. An optical information reader, Taking an image of the container to which the code is attached and a pattern arranged in the background of the container, and acquiring the image; Decoding the code from the image; classifying each area of ​​the image according to the state of the pattern, thereby detecting the presence or absence of a cap on the container; Optical information reading method.

6. An optical information reading device, A step of capturing an image of a container to which a code is attached and a pattern disposed in the background of the container; decoding the code from the image; detecting the presence or absence of a cap on the container by classifying each region in the image according to the state of the pattern; Execute the Optical information reading program.

Images