Apparatus and method for reading strip type test kit

Abstract

The present invention relates to a method for reading a strip-type test device based on image analysis, comprising: (a) acquiring a target image captured of the strip-type test device having an inspection part and a handle part that react with a specific component of a sample to produce color; (b) identifying a grip area corresponding to the handle part from the target image; (c) aligning the target image using shape information of the grip area; (d) identifying a Region of Interest (ROI) area corresponding to the inspection part from the aligned target image using the shape information and generating a crop image including the ROI area; and (e) acquiring a reading result of the strip-type test device using the crop image.

Description

Technology Field

[0001] The present invention relates to a device and method for reading a strip-type test kit, and for example, the present invention relates to a technique for reading an image taken of an ovulation test kit. Background Technology

[0003] Generally, medical devices are primarily used by end consumers, such as hospitals and patients; however, some devices are also used at home under a hospital prescription. Additionally, there are cases where users purchase and use medical devices on their own as a means of managing their health, even without a hospital prescription.

[0004] As such, medical devices that users can use without a doctor's prescription include ovulation analyzers (ovulation test kits), blood glucose meters, blood pressure monitors, thermometers, height scales, physical therapy devices, and hearing aids, and these can also be used in hospitals.

[0005] Among these, ovulation testers are essential products not only for women seeking pregnancy but also for those wishing to avoid unwanted pregnancies. They eliminate the hassle of temperature checks or urine tests and offer rapid and economical advantages compared to methods such as hormone determination. Furthermore, because they allow for the simple and accurate measurement of ovulation, they are used when trying to conceive and are gaining popularity recently as they can be utilized without the inconvenience or additional ongoing costs associated with using contraceptives such as birth control pills or condoms.

[0006] The technology forming the background of the present invention, Korean Published Patent Application No. 10-2019-0000003, relates to an illuminance meter, an Internet of Things-based ovulation test device including the same, and a method for measuring the fertile period using the same. The problem to be solved

[0008] The present invention provides a method for reading a strip-type tester and a reading device for performing said reading method, in order to solve the problems of the aforementioned prior art.

[0009] However, the technical problems that the embodiments of the present invention aim to solve are not limited to the technical problems described above, and other technical problems may exist. means of solving the problem

[0011] The first aspect of the present invention relates to a method for reading a strip-type test device based on image analysis, comprising: (a) acquiring a target image taken of the strip-type test device having an inspection part and a handle part that react with a specific component of a sample to produce color; (b) identifying a grip area corresponding to the handle part from the target image; (c) aligning the target image using shape information of the grip area; (d) identifying a Region of Interest (ROI) area corresponding to the inspection part from the aligned target image using the shape information and generating a crop image including the ROI area; and (e) acquiring a reading result of the strip-type test device using the crop image.

[0012] According to one embodiment of the present invention, step (b) may identify the grip area by cropping the handle portion of the strip-type tester and masking the cropped handle portion, or identify the grip area by masking the handle portion of the target image, but is not limited thereto.

[0013] According to one embodiment of the present invention, the step (b) may include, but is not limited to, (b1) masking the handle portion from the target image; (b2) identifying a plurality of reference points corresponding to corner areas forming the handle portion; and (b3) identifying a grip area using coordinate information of the plurality of reference points.

[0014] According to one embodiment of the present invention, step (c) may include (c1) a step of rotating the target image using coordinate information of the reference point; and (c2) a step of applying a predetermined image processing to the rotated target image, but is not limited thereto.

[0015] According to one embodiment of the present invention, the step (c1) may include, but is not limited to, a step of rotating the target image so that the strip-type tester of the target image is horizontal.

[0016] According to one embodiment of the present invention, the step (c1) may include, but is not limited to, a step of rotating the target image so that the handle portion is placed on one side of the target image when the handle portion of the strip-type tester is placed on the other side of the target image.

[0017] According to one embodiment of the present invention, the step (c2) may include, but is not limited to, a step of processing the rotated target image through an algorithm selected from the group consisting of Contrast Limited Adaptive Histogram Equalization (CLAHE), Clipping and Stretching (CS), Histogram Equalization (HE), and combinations thereof.

[0018] According to one embodiment of the present invention, the step (c2) may further include, but is not limited to, a step of identifying a plurality of reference points from the rotated target image; and a step of identifying a grip area of ​​the rotated target image using the coordinates of the reference points.

[0019] According to one embodiment of the present invention, the step (e) may include (e1) a step of processing the cropped image; and (e2) a step of analyzing the control line and inspection line of the image-processed ROI area; but is not limited thereto.

[0020] According to one embodiment of the present invention, the step (e1) may include, but is not limited to, a step of processing the cropped image into grayscale.

[0021] According to one embodiment of the present invention, the step (e2) may include, but is not limited to: (e21) extracting a control line and a test line from the grayscale-processed ROI area; (e22) calculating the brightness values ​​of the control line and the test line; and (e23) returning the reading result through the difference between the brightness value of the control line and the brightness value of the test line.

[0022] The second aspect of the present invention relates to a reading device for a strip-type tester based on image analysis, comprising: an image acquisition unit for acquiring a target image captured of the strip-type tester, which has an inspection unit and a handle unit that react with a specific component of a sample to produce color; an object identification unit for identifying a grip area corresponding to the handle unit from the target image; an image alignment unit for aligning the target image using shape information of the grip area; an image cropping unit for identifying a Region of Interest (ROI) area corresponding to the inspection unit from the aligned target image using the shape information and generating a cropped image including the ROI area; and a reading execution unit for acquiring a reading result of the strip-type tester using the cropped image.

[0023] According to one embodiment of the present invention, the object identification unit may include a masking unit that masks the handle portion of the target image, a reference point identification unit that identifies a plurality of reference points corresponding to corner areas forming the handle portion from the target image, and a grip area identification unit that calculates coordinate information of the reference points to identify a grip area, but is not limited thereto.

[0024] According to one embodiment of the present invention, the image alignment unit may include a rotation unit that rotates the target image using shape information of the grip area; and an image processing unit that applies a predetermined image processing to the rotated target image, but is not limited thereto.

[0025] According to one embodiment of the present invention, the image cropping unit may include an ROI identification unit that identifies an ROI region from the aligned target image; and a crop image generation unit that generates a crop image for the ROI region, but is not limited thereto.

[0026] According to one embodiment of the present invention, the reading performing unit may include a preprocessing unit that processes the image before analyzing the cropped image; and an analysis unit that analyzes inspection lines and control lines from the image-processed cropped image, but is not limited thereto.

[0027] According to one embodiment of the present invention, the reading device may further include an output unit for displaying the reading result, but is not limited thereto.

[0028] The means for solving the problem described above are merely exemplary and should not be interpreted as intended to limit the present invention. In addition to the exemplary embodiments described above, additional embodiments may exist in the drawings and the detailed description of the invention. Effects of the invention

[0030] The reading method of a strip-type test device according to the present invention minimizes the influence of the brightness of the environment in which the strip-type test device is photographed on the photographed image of the test device by photographing the test device and analyzing the image after image processing.

[0031] In addition, the reading method of the strip-type tester involves identifying the handle portion of the strip-type tester from the captured image of the strip-type tester and then identifying the inspection portion. At this time, the position of the strip-type tester can be adjusted or rotated according to the position of the inspection portion in the image, thereby eliminating the need for additional shooting.

[0032] In addition, the reading method of a strip-type test kit according to the present invention can be generally applied to strip-type test kits that involve a color reaction, and can be utilized for urine tests, etc., in addition to ovulation test kits.

[0033] However, the effects obtainable from this invention are not limited to those described above, and other effects may exist. Brief explanation of the drawing

[0035] FIG. 1 is a schematic diagram showing a reading system of a strip-type tester according to one embodiment of the present invention. FIG. 2 is a flowchart of the operation of a reading method for a strip-type tester according to one embodiment of the present invention. FIGS. 3 to 6 are detailed operation flowcharts for each step of the reading method of a strip-type tester according to one embodiment of the present invention. FIG. 7 is a schematic diagram of the reading device of a strip-type tester according to one embodiment of the present invention. FIGS. 8 to 15 are drawings illustrating an exemplary user interface for displaying a reading result using a reading device of an ovulation test kit according to one embodiment of the present invention. FIGS. 16 to 21 are exemplary drawings illustrating an image converted or processed from a target image during the reading process of an ovulation test kit according to one embodiment of the present invention. Specific details for implementing the invention

[0036] Below, embodiments of the present invention are described in detail with reference to the attached drawings so that those skilled in the art can easily implement them.

[0037] However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly illustrate the present invention in the drawings, parts unrelated to the description have been omitted, and similar parts throughout the specification have been given similar reference numerals.

[0038] Throughout this specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected," but also cases where they are "electrically connected" with other elements interposed between them.

[0039] Throughout the entire specification, when a component is described as being located "on," "on top," "on top," "under," "on bottom," or "on bottom" of another component, this includes not only cases where the component is in contact with the other component but also cases where another component exists between the two components.

[0040] Throughout this specification, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0041] As used herein, terms of degree such as “about,” “substantially,” etc., are used to mean at or near the stated value when inherent manufacturing and material tolerances are presented in the stated meaning, and are used to prevent unscrupulous infringers from unfairly exploiting the disclosure in which precise or absolute values ​​are mentioned to aid in understanding the invention. Furthermore, throughout this specification, “a step of” or “a step of” does not mean “a step for”.

[0042] Throughout this specification, the term “combination thereof” included in the Markush-type expression means one or more mixtures or combinations selected from the group consisting of the components described in the Markush-type expression, and means including one or more selected from the group consisting of said components.

[0043] Throughout the entire specification, the description "A and / or B" means "A or B, or, A and B".

[0044] The present invention relates to a device and method for reading a strip-type test kit, and for example, the present invention relates to a technique for reading an image taken of an ovulation test kit.

[0045] Hereinafter, a reading method and reading device for a strip-type tester will be described in detail with reference to embodiments, examples, and drawings. However, the present invention is not limited to these embodiments, examples, and drawings.

[0046] The first aspect of the present invention relates to a method for reading a strip-type test device (200) based on image analysis, comprising: (a) acquiring a target image (300) taken of the strip-type test device (200) which has an inspection part (210) that reacts with a specific component of a sample to produce color and a handle part (220); (b) identifying a grip area corresponding to the handle part (220) from the target image (300); (c) aligning the target image (300) using shape information of the grip area; (d) identifying a Region of Interest (ROI) area corresponding to the inspection part (210) from the aligned target image using the shape information and generating a crop image including the ROI area; and (e) acquiring a reading result of the strip-type test device (200) using the crop image.

[0047] As will be described later, the reading method of the strip-type test device (200) according to the present invention can be explained using an ovulation test device as an example, but it can be applied to various strip-type test devices that include a color reaction, even if they are not ovulation test devices.

[0048] FIG. 1 is a schematic diagram showing a reading system of a strip-type tester according to one embodiment of the present invention, and FIGS. 2 to 6 are flowcharts of an operation for a reading method of a strip-type tester according to one embodiment of the present invention. Specifically, FIGS. 3 to 6 represent schematic diagrams specifically illustrating a specific step of FIG. 2, where FIG. 3 is a flowchart showing step (b), FIG. 4 is a flowchart showing step (c), and FIGS. 5 and 6 are flowcharts showing step (e).

[0049] As will be described later, the reading method of the strip-type test device (200) according to the present invention can be performed by the reading device (100) of the strip-type test device (200). The reading device (100) is a mechanism for interpreting the reading result of the strip-type test device (200) from an image of the strip-type test device (200), and in addition to the device depicted in FIG. 1, it may have various forms such as a computer, a mobile communication terminal, a mobile computer, a network device, a server, etc.

[0050] Meanwhile, in the description of the embodiments of the present invention, some of the operations performed by the reading device (100) may be partially performed at the mobile device end that captures the target image (300), and the detailed reading process for the target image (300) may be performed at the server end that receives the target image (300) from the mobile device. For example, at the mobile device end, the capture of the target image (300) and the detection of ROI within the target image (300) may be performed, and an image corresponding to the detected ROI may be transmitted to the server, and the server may perform a process of reading the image, but is not limited thereto.

[0051] According to one embodiment of the present invention, the reading device (100) may include, but is not limited to, a device for analyzing an acquired target image (300), a device for reading a result from the analyzed target image (300), and a device for outputting the read result. For example, steps (b), (c), (d), and (e) of the reading method may be performed in the reading device (100).

[0052] First, (a) a target image (300) is obtained by taking a test of the strip-type test device (200) which has a test section (210) and a handle section (220) that react with a specific component of the sample to produce color (S100).

[0053] The strip-type test device (200) according to the present invention refers to a device comprising a test portion (210) that reacts with a specific component of a sample to produce color and a handle portion (220). For example, the strip-type test device (200) may include an ovulation test device.

[0054] In this regard, the inspection unit (210) includes an inspection line that develops color according to the concentration of a specific component of the sample, and a control line that exists in a colored state regardless of the sample. That is, in a device in which a sample with a concentration of the specific component below a certain level is reacted with the inspection unit (210), an inspection line that does not develop color and a control line that develops color are observed, and in a device in which a sample with a concentration of the specific component above a certain level is reacted with the inspection unit (210), a inspection line that develops color and a control line that develops color are observed.

[0055] The handle portion (220) refers to a part for bringing the inspection portion (210) into contact with the sample or for transporting the strip-type tester (200).

[0056] The target image (300) captured with respect to the strip-type test device (200) may refer to an image captured by a shooting unit included in the reading device (100), or an image captured by a separate device including a shooting unit and transmitted to the reading device (100).

[0057] The target image (300) according to the present invention may have a rectangular shape and may have a long side and a short side.

[0058] According to one embodiment of the present invention, the target image (300) may be placed within an image placement area (400), but is not limited thereto.

[0059] If the target image (300) is not tilted, the image placement area (400) and the target image (300) may coincide. However, if the line constituting the target image (300) is tilted at a predetermined angle, the image placement area (400) may mean a non-tilted rectangle that includes four reference points of the target image (300) on its sides, and the target image (300) is placed inside the image placement area (400).

[0060] Next, a grip area corresponding to the handle portion (220) is identified from the target image (300) (S200).

[0061] In this regard, the handle portion (220) and the grip area may represent the same area in the strip-type test device (200) and the target image (300). However, the handle portion (220) refers to a part of the strip-type test device (200), and the grip area refers to the handle portion (220) identified in the target image (300). The grip area is an image containing an image of the handle portion (220), and may be processed to increase clarity or in black and white.

[0062] According to one embodiment of the present invention, step (b) may identify the grip area by cropping the handle portion (220) of the strip-type tester (200) and masking the cropped handle portion (220), or identify the grip area by masking the handle portion (220) of the target image (300), but is not limited thereto.

[0063] The above target image (300) is a strip-type test device (200) that has been photographed. In this regard, since the target image (300) is analyzed, read, and output results by the reading device (100), the device needs to recognize the strip-type test device (200) within the target image (300). Generally, since the strip-type test device (200) is a rectangle having a specific structure, when the reading device (100) reads the strip-type test device (200) within the target image (300), the handle portion (220) of the strip-type test device (200) can be cropped, and the cropped handle portion (220) can be masked.

[0064] However, if the reading device (100) does not recognize the strip-type test device (200) within the target image (300), the reading device (100) can identify the grip area by reading the pattern corresponding to the preset handle portion (220) and masking the read portion.

[0065] For example, as will be described later, the handle portion (220) may mean a colored background with the letters LH arranged on it.

[0066] According to one embodiment of the present invention, the step (b) may include, but is not limited to, (b1) masking the handle portion (220) from the target image (300); (b2) identifying a plurality of reference points corresponding to corner areas forming the handle portion (220); and (b3) identifying a grip area using coordinate information of the plurality of reference points.

[0067] When the reading device (100) masks the handle portion (220) from the target image (300), the reading device (100) cannot recognize the portion excluding the grip area (the inspection portion (210) of the strip-type test device (200) and the background) in the target image (300), and can recognize only the grip area (step (b1)).

[0068] For example, if the handle portion (220) has a rectangular shape, the grip area may have a rectangular shape with one side longer. In this case, the plurality of reference points may refer to four vertices identified in the masked handle portion (230), but the number of vertices may increase or decrease or increase depending on the shape of the handle portion (220). The following description is based on the case where there are four reference points.

[0069] A rectangle drawn with the above four vertices may have one side longer than the other. In this case, if all three conditions are satisfied—(1) the length of the longer side of the rectangle is longer than the length of the shorter side, (2) the length of the longer side of the rectangle is smaller than the value obtained by subtracting a predetermined value (e.g., 500 pixels) from the length of the longer side of the target image (300), and (3) the length of the shorter side of the rectangle is greater than a preset value (e.g., 30 pixels)—the area defined by the above four vertices can be determined as a grip area.

[0070] However, the above predetermined values ​​(500 pixels, 30 pixels, etc.) may vary depending on the size of the target image (300), the size of the strip-type tester (200), etc.

[0071] The above grip area may be the same as the area represented by the handle portion (220), or may have an area larger or smaller than the handle portion (220) while being geometrically similar to the handle portion (220) and having a center of gravity at the same point, but is not limited thereto.

[0072] The reference points for identifying the above grip area can also be used in the process of aligning the target image (300) described later.

[0073] Next, (c) the target image (300) is aligned using the shape information of the grip area (S300).

[0074] The step of aligning the target image (300) means adjusting the position of the strip-type tester (200) within the target image (300). The target image (300) is an image of the strip-type tester (200), but the strip-type tester (200) may be photographed in a tilted state. Since the result of the inspection unit (210) of the strip-type tester (200) may be misinterpreted if the strip-type tester (200) is analyzed in a tilted state during the process of analyzing the inspection unit (210) of the strip-type tester (200) described later, the target image (300) can be aligned using the shape information of the grip area.

[0075] By using the coordinate information of the reference points constituting the grip area, the strip-type tester (200) can be rotated or the target image (300) in which the strip-type tester (200) is captured can be rotated so that the strip-type tester (200) has a horizontal shape without tilting.

[0076] According to one embodiment of the present invention, step (c) may include (c1) a step of rotating the target image (300) using coordinate information of a plurality of reference points constituting the grip area; and (c2) a step of applying a predetermined image processing to the rotated target image (310), but is not limited thereto.

[0077] A plurality of reference points for rotating the target image (300) may mean four vertices for identifying the grip area.

[0078] According to one embodiment of the present invention, the step (c1) may include, but is not limited to, a step of rotating the target image (300) so that the strip-type tester (200) of the target image (300) is horizontal.

[0079] Specifically, the reference point of the grip area includes four vertices of the handle portion (220), the handle portion (220) may have a rectangular shape, and the strip-type tester (200) may be photographed in a tilted state within the target image (300). Accordingly, when the target image (300) is rotated so that the angle between the long side of the handle portion (220) (or the long side of the grip area) and the side in the width direction of the image area becomes 0 (i.e., the strip-type tester (200) and the target image are horizontal), the strip-type tester (200) within the target image (300) may be horizontal with the reference line outside the target image (300) (i.e., the side in the width direction of the image area). However, since the strip-type tester (200) may not be perfectly level when the target image (300) is rotated once, the rotation of the target image (300) may be performed at least once, and if the strip-type tester (200) is level with the reference line even after being performed two or more times, the step of rotating may be stopped.

[0080] According to one embodiment of the present invention, the step (c1) may include, but is not limited to, a step of rotating the target image (300) so that the handle portion (220) is placed on one side of the image placement area (400) when the handle portion (220) of the strip-type tester (200) is placed on the other side of the image placement area (400).

[0081] As previously mentioned, the image placement area (400) refers to an area where the target image (300) is placed. Since the one side and the other side of the target image (300) may also change when the target image (300) is rotated, the image placement area (400) is intended to establish a standard for determining the one side and the other side regardless of whether the target image (300) is rotated.

[0082] Since the image placement area (400) maintains a rectangle that is not tilted and only increases in size even when the target image (300) is rotated, the one side and the other side of the image placement area (400) may refer to directions independent of the rotation of the target image (300) even when the target image (300) is rotated. For example, the other side of the image placement area (400) may refer to the left side of the line that vertically bisects the width side of the image placement area (400), and the one side of the image placement area (400) may refer to the right side of the line that vertically bisects the width side of the image placement area (400).

[0083] When the handle portion (220) of the strip-type tester (200) is positioned on the other side of the target image (300), the reading device (100) may not recognize the inspection portion (210) of the strip-type tester (200), or even if it recognizes it, it may misread the inspection portion (210). To prevent this problem, the handle portion (220) must be positioned on one side of the image placement area (400). To this end, the angle for rotating the target image (300) may be up to 180 degrees.

[0084] According to one embodiment of the present invention, the rotated target image (310) and the underotated target image (300) may have the same center, but are not limited thereto. In this case, the center refers to the center of gravity.

[0085] The image placement area (400) may refer to a rectangle having a reference point of the untilted target image (300) as a vertex. In this regard, if the strip-type tester (200) within the untilted target image (300) is tilted, the strip-type tester (200) within the rotated target image (310) may be horizontal or vertical to one side of the image placement area (410) through the step of rotating the target image (300).

[0086] At this time, the image placement area (400) formed based on the untilted target image (300) and the image placement area (410) formed based on the rotated target image (310) may have different areas. However, since the two image placement areas include the same center and are formed to include both target images, and are formed for the identification of the grip area and rotation angle for the analysis of the strip-type tester (200) rather than for the analysis of the strip-type tester (200), the strip-type tester (200) is not misread due to the difference in size between the image placement area (400) of the unrotated target image (300) and the image placement area (410) of the rotated target image (310).

[0087] According to one embodiment of the present invention, the step (c2) may further include, but is not limited to, a step of identifying a plurality of reference points from the rotated target image (310); and a step of identifying a grip area of ​​the rotated target image (310) using the coordinates of the reference points.

[0088] When the target image (300) is rotated in step (c1) above, it is necessary to identify a new grip area in the rotated target image (310). To do this, as mentioned in step (b) above, the handle portion (220) in the rotated target image (310) is masked, and after identifying a plurality of reference points in the masked handle portion (230), the grip area in the rotated target image (310) can be identified through the reference points.

[0089] However, even if the above grip area is not identified, if four reference points satisfying the following conditions are confirmed, the rectangle formed by the reference points can be determined as the grip area.

[0090] For example, if the other side of the image placement area (400) means the left side of the image placement area (400) and the one side of the image placement area (400) means the right side of the image placement area (400), the x-coordinate of the leftmost point of the image placement area (400) can be set to 0, and the x-coordinate of the rightmost point of the image placement area (400) can be set to 100. At this time, among the reference points in the rotated target image (310), a reference point satisfying any one of the following cases can be determined as a grip area: (1) the coordinate of the upper left is placed on the left side of the image placement area (400) (i.e., the coordinate of the upper left among the reference points is 50 or less based on the x-coordinate of the image placement area (400)), (2) the coordinate of the upper left exceeds 70 based on the x-coordinate of the image placement area (400), or (3) the length of the shortest side among the sides formed by the four reference points is less than a preset value (e.g., 10 pixels).

[0091] According to one embodiment of the present invention, the step (c2) may include, but is not limited to, a step of homogenizing an image.

[0092] In this regard, when finding the rectangle with the longest horizontal length in the image with the handle portion (220) masked in the target image (300) and checking the four corner coordinates, the rectangle can be identified as a grip area only if all conditions are satisfied: the horizontal length of the rectangle is longer than the vertical length, but shorter than the horizontal length of the image placement area (400) by a predetermined length (e.g., 500 pixels), and the vertical length of the rectangle is greater than a certain value (e.g., 30 pixels). At this time, even if the above three conditions are not satisfied, if any one of the following conditions is satisfied among the reference points in the rotated target image (310)—(1) the coordinate of the top left is placed to the left of the image placement area (400), (2) the coordinate of the top left exceeds 70 based on the x-coordinate of the image placement area (400), or (3) the length of the shortest side among the sides formed by the four reference points is less than a preset value (e.g., 10 pixels)—then the process can proceed to step (d) even if the rectangle is not identified as a grip area. However, if any of the three conditions (1), (2), and (3) do not apply while the identification of the grip area has failed, steps (b) and (c) can be performed again.

[0093] Due to the above image processing, the inspection part (211) and the handle part (221) of the image-processed target image (not shown) may be visually different from the inspection part (210) and the handle part (220) of the initial target image (300).

[0094] When reading the above strip-type test device (200), if the image processed by step (c), more specifically step (c2), is used, the reading result may be misread. Therefore, step (c) is used only when aligning the target image after obtaining only the coordinates through the image processed image.

[0095] At this time, the above-mentioned image processed can be used to identify the grip area when the grip area in the target image (300) is not identified.

[0096] Next, (d) using the shape information, a Region of Interest (ROI) area (212) corresponding to the inspection unit (210) is identified from the aligned target image, and a crop image including the ROI area (212) is generated (S400). At this time, the aligned target image is an image that has been uniformized by step (c2), but the uniformized image is used to identify the ROI area, and the crop image including the identified ROI area may be generated using the unaligned target image, that is, a target image (310) that has been rotated so that the ovulation test kit is horizontal with the image placement area.

[0097] According to one embodiment of the present invention, the step of identifying the ROI region may include the step of moving the grip region of the aligned target image by the length of the ROI region and the step of identifying the inspection unit (210) having a preset pattern from the moved target image (300) as the ROI region.

[0098] The above-mentioned aligned target image includes a strip-type tester (200) that is horizontal to the image placement area (400). Since the reading device (100) identifies an area corresponding to the inspection section (210) of the strip-type tester (200) as an ROI area to read the result, it is necessary to adjust the position of the inspection section (210) in the image placement area (400) so that the reading device (100) can easily read the inspection section (210).

[0099] When the grip area of ​​the above-mentioned aligned target image is moved by the length of the ROI area, the handle portion (220) of the strip-type tester (200) moves by the length of the ROI area, and as a result, the inspection portion (210) adjacent to the handle portion (220) can also move.

[0100] For example, the length of the ROI area may mean half of the (horizontal length of the handle portion (220)) minus a preset value, but is not limited thereto. In this case, the horizontal length of the handle portion (220) may mean the longer side of the rectangle constituting the handle portion (220) of the strip-type tester (200), and the preset value may mean 28 pixels.

[0101] However, the ratio (1 / 2) of the horizontal length of the handle portion (220), which is a numerical value used to determine the length of the ROI area, and the preset numerical value (28 pixels) may be changed as needed.

[0102] The above preset value is intended to remove the shadow area of ​​the other side of the grip area (for example, if the grip area is located on the right side, which is one side of the image placement area (400), the other side of the grip area means the left side of the grip area).

[0103] The above shadow refers to the shadow formed when the grip area identified by the handle portion (220) protrudes more than the bottom area.

[0104] As described above, the inspection unit (210) may include an inspection line that develops color according to the concentration of a specific component of the sample, and a control line that develops color regardless of the concentration of the specific component of the sample. That is, the reading device (100) may be configured to identify a certain area as an ROI region based on the control line.

[0105] According to one embodiment of the present invention, the method may include a step of adjusting the size of the cropped image, but is not limited thereto.

[0106] The cropped image can be stored in the reading device (100) and transmitted to the reading execution unit (150) of the reading device (100). In this process, if the size of the cropped image is not constant, there is a possibility that the reading will be performed incorrectly, so it is necessary to adjust the size of the cropped image.

[0107] For example, the size of the resized cropped image may be 400 pixels * 80 pixels, but is not limited thereto.

[0108] Next, (e) a reading result of the strip-type tester (200) is obtained using the cropped image (S500).

[0109] In this regard, the above-mentioned reading result can be output and verified by a separate output device.

[0110] As will be described later, the above step (e) may be performed in a separate device from the above steps (b) to (d), but is not limited thereto.

[0111] According to one embodiment of the present invention, the step (e) may include (e1) a step of processing the cropped image; and (e2) a step of analyzing the control line and inspection line of the image-processed ROI area; but is not limited thereto.

[0112] According to one embodiment of the present invention, the step (e1) may include, but is not limited to, a step of processing the cropped image into grayscale.

[0113] Step (e1) above may be grayscaled after adjusting the size of the cropped image, but is not limited thereto.

[0114] According to one embodiment of the present invention, the step (e2) may include, but is not limited to: (e21) extracting a control line and a test line from the grayscale-processed ROI area; (e22) calculating the brightness values ​​of the control line and the test line; and (e23) returning the reading result through the difference between the brightness value of the control line and the brightness value of the test line.

[0115] In the ROI area of ​​the above grayscale-processed cropped image, inspection line and control line image areas can be separated. Specifically, the central x-coordinate of the ROI area can be extracted, and one side and the other side can be extracted as the inspection line and control line based on the central x-coordinate.

[0116] For example, based on the above-mentioned center x-coordinate, an area of ​​a predetermined length to the left can be returned to extract a check line, and an area of ​​a predetermined length to the right can be returned to extract a control line.

[0117] Since the above cropped image is processed into a grayscale by image processing and there is a difference in brightness, the intensity value of the colored line from the extracted test line and control line can be calculated, and the result of the strip-type test device (200) can be read based on the intensity value.

[0118] Specifically, linear data regarding the intensity values ​​of each pixel can be generated from the grayscale-processed ROI image, and the average brightness value of pixels on the same vertical line can be calculated using the linear data and stored in a list of average brightness values. The list of average brightness values ​​may be linear data.

[0119] The above list of average brightness values ​​may include a list of average brightness values ​​of the inspection line, a list of average brightness values ​​of the control line, and a list of average brightness values ​​of the background area of ​​the ROI area (an ROI area other than the inspection line and the control line).

[0120] Next, the values ​​of the above list of average brightness values, preferably the list of average brightness values ​​of the background area that is not the inspection line and the control line, are sorted in descending order, and a certain number of ranges are taken from the sorted list to calculate the average brightness value of only the background area.

[0121] Then, if the average brightness value of the background area is subtracted from the brightness values ​​of the list of average brightness values ​​of the inspection line and the list of average brightness values ​​of the control line, only the brightness values ​​of the non-background area remain in the list of average brightness values ​​of the inspection line and the list of average brightness values ​​of the control line, which represents the brightness values ​​of the inspection line and the control line.

[0122] Next, if a certain number of ranges are taken from the list of average brightness values ​​of the inspection line and the list of average brightness values ​​of the control line, the average brightness value at the inspection line and the average brightness value at the control line can be returned.

[0123] In this regard, the description of the lightness value may be used interchangeably with the description of the "intensity value."

[0124] Based on the average brightness value of the returned test line and the average brightness value of the control line, the intensity value of the test line and the intensity value of the control line can be calculated. In this regard, since the control line develops color regardless of the concentration of a specific component in the sample, if the intensity value of the control line does not satisfy a predetermined condition, the reading device (100) cannot read the strip-type test device (200). However, if the intensity value of the control line satisfies a certain condition, the strip-type test device (200) can be read through the intensity value of the test line.

[0125] The intensity value of the above control line may exceed 0, but is not limited thereto. If the intensity value of the above control line is measured to be 0 or less, that is, if the intensity value of the control line is smaller than the intensity value of the background area, or if the intensity value of the control line and the intensity value of the background area are the same, it means that no reaction occurs between the sample and the inspection part because there is no color development of the above control line.

[0126] The above cropped image can be analyzed even in a colored state without grayscale processing. However, in this case, the color of the background area needs to be clearly distinguishable from the colors of the inspection line and the control line so that the color of the background area does not affect the analysis of the inspection line and the control line.

[0127] According to one embodiment of the present invention, the reading method of the strip-type tester (200) may further include the step of outputting the acquired reading result through an output device, but is not limited thereto.

[0128] According to one embodiment of the present invention, the sample may comprise a body fluid selected from the group consisting of urine, blood, sweat, and combinations thereof, but is not limited thereto.

[0129] Preferably, the sample may contain urine.

[0130] According to one embodiment of the present invention, the strip-type test device (200) may include an ovulation test device or a urine test device, but is not limited thereto.

[0131] Generally, the above ovulation test kit is intended to detect whether ovulation has occurred through the concentration of luteinizing hormone (LH) in urine. Specifically, referring to a graph of hormonal changes according to the menstrual cycle, if the concentration of luteinizing hormone rises sharply just before ovulation (LH surge), ovulation is confirmed 12 to 36 hours after the point of increase. In other words, the above ovulation test kit is designed to predict the timing of ovulation by predicting the LH surge through the concentration of LH.

[0132] FIG. 7 is a schematic diagram of the reading device of a strip-type tester (200) according to one embodiment of the present invention.

[0133] Referring to FIG. 7, the second aspect of the present invention relates to a reading device (100) of a strip-type test device (200) based on image analysis, wherein the reading device (100) comprises: an image acquisition unit (110) for acquiring a target image (300) captured on the strip-type test device (200), which has an inspection unit (210) that reacts with a specific component of a sample to produce color and a handle unit (220); an object identification unit (120) for identifying a grip area corresponding to the handle unit (220) from the target image (300); an image alignment unit (130) for aligning the target image (300) using shape information of the grip area; and an image cropping unit (140) for identifying a Region of Interest (ROI) area corresponding to the inspection unit (210) from the aligned target image using the shape information and generating a cropped image including the ROI area. The invention relates to a reading device (100) of a strip-type tester (200), comprising a reading execution unit (150) that obtains a reading result of the strip-type tester (200) using the cropped image.

[0134] Regarding the reading device (100) of the strip-type tester (200) according to the second aspect of the present invention, detailed descriptions of parts that overlap with the first aspect of the present invention have been omitted, but even if such descriptions are omitted, the contents described in the first aspect of the present invention can be applied identically to the second aspect of the present invention. Specifically, the reading method of the first aspect can be performed through the reading device (100) of the second aspect.

[0135] The above reading device (100) is a mechanism for interpreting the reading result of a strip-type test device (200) from an image of the strip-type test device (200), and in addition to the device depicted in FIG. 1, it may have various forms such as a computer, mobile communication terminal, mobile computer, network device, server, application, program, etc.

[0136] Throughout the entire specification, the reading method and the reading device (100) are described by assuming that the reading device (100) is a smartphone that includes a shooting unit and can be connected to an external server, but the reading device (100) may be defined as the sum of devices capable of performing each part.

[0137] According to one embodiment of the present invention, the reading device (100) may include a communication unit capable of transmitting and receiving data between an image collection unit (110), an object identification unit (120), an image alignment unit (130), an image cropping unit (140), and a reading execution unit (150), but is not limited thereto. The communication unit may also be responsible for transmitting and receiving data between the reading device (100) and an external device.

[0138] According to one embodiment of the present invention, the image acquisition unit (110) may include a shooting unit included in the reading device (100) or a shooting device existing as a device independent of the reading device (100), but is not limited thereto.

[0139] If the image acquisition unit (110) includes the shooting device, the target image (300) captured by the shooting device can be transmitted to the image acquisition unit (110) via the communication unit. In this regard, the target image (300) is placed inside the image placement area (400).

[0140] The target image (300) stored in the image collection unit (110) is transmitted to the object identification unit (120), and the object identification unit (120) can identify a grip area in the target image (300).

[0141] According to one embodiment of the present invention, the object identification unit (120) may include a masking unit (121) that masks the handle portion (220) of the target image (300), a reference point identification unit (122) that identifies a plurality of reference points corresponding to corner areas forming the handle portion (220) from the target image (300), and a grip area identification unit (123) that identifies a grip area by calculating coordinate information of the reference points, but is not limited thereto.

[0142] The grip area identified by the grip area identification unit (123) can be rotated by a plurality of reference points identified by the reference point identification unit (122).

[0143] According to one embodiment of the present invention, the image alignment unit (130) may include a rotation unit (131) that rotates the target image (300) using shape information of the grip area; and an image processing unit (132) that applies a predetermined image processing to the rotated target image (310), but is not limited thereto.

[0144] As described above, the target image (300) is placed inside the image placement area (400), and when the target image (300) is rotated in the rotation unit (131), the image placement area (400) may be expanded or reduced. Specifically, since the sides of the rectangle constituting the image placement area (400) include reference points of the target image (300), when the target image (310) rotated in the rotation unit (131) is rotated, the reference points of the top-left, bottom-left, top-right, and bottom-right constituting the grip area of ​​the target image (300) move, and a new image placement area (400) may be formed by the moved reference points of the top-left, bottom-left, top-right, and bottom-right. However, the area of ​​the image placement area (400) formed based on the untilted target image (300) and the area of ​​the image placement area (400) formed based on the rotated target image (310) may be different. However, since the two image placement areas (400) are formed to include the same center and both rotated target images (310), and are formed for the identification of grip areas and rotation angles for the analysis of the strip-type tester (200) rather than for the analysis of the strip-type tester (200), the strip-type tester (200) is not misread due to the difference in size between the image placement area (400) of the unrotated target image (300) and the image placement area (400) of the rotated target image (310).

[0145] According to one embodiment of the present invention, the image cropping unit (140) may include an ROI identification unit (141) for identifying an ROI region from the aligned target image; and a cropped image generation unit (142) for generating a cropped image for the ROI region, but is not limited thereto.

[0146] The above ROI identification unit (141) is intended to define the area corresponding to the inspection unit (210) of the strip-type tester (200) identified in the aligned target image as the ROI area.

[0147] The above crop image generating unit (142) can crop an image including the ROI area.

[0148] According to one embodiment of the present invention, the reading performing unit (150) may include a preprocessing unit (151) for image processing before analyzing the cropped image; and an analysis unit (152) for analyzing inspection lines and control lines from the image-processed cropped image, but is not limited thereto.

[0149] The above preprocessing unit (151) may include a unit for reducing or enlarging the area of ​​a cropped image including the ROI region and a unit for processing the cropped image into grayscale, but is not limited thereto. However, the above preprocessing unit (151) may not include a unit for processing the cropped image into grayscale.

[0150] The analysis unit (152) is intended to analyze the brightness or intensity value of the inspection line and the control line. At this time, if the intensity value of the control line is below a certain value, the analysis unit (152) may not perform the step of analyzing the inspection line, and only when the intensity value of the control line is above a certain value can the intensity value of the control line and the intensity value of the inspection line be analyzed to read the result of the strip-type test device (200).

[0151] According to one embodiment of the present invention, the reading device (100) may further include an output unit for displaying the reading result, but is not limited thereto.

[0152] The above output unit is intended to output the analysis results of the above inspection line and control line, and to display the analysis results so that they can be accessed through various channels such as sight or hearing.

[0153] In the following, each function is explained based on the case where the reading device is an application applied to a smartphone and the strip-type test device is an ovulation test device, that is, an application that analyzes an ovulation test device. In this case, the reading device can photograph the strip-type test device through the camera unit of the smartphone.

[0154] FIGS. 8 to 15 are drawings illustrating an exemplary user interface for displaying reading results using a reading device of an ovulation test kit according to one embodiment of the present invention. Specifically, FIG. 8 lists each menu of the reading device, and FIGS. 9 to 15 illustrate exemplary screens for each menu.

[0155] Referring to FIGS. 8 to 15, the reading device according to the present invention can check the current status of the user through a home screen, etc., and can notify the user of the status based on the reading results of an ovulation test kit through a notification history. In addition, the current status can be entered and checked through a calendar built into the reading device.

[0156] FIGS. 11 to 15 illustrate a method for reading an ovulation test kit using the reading device. First, necessary items can be prepared through the test guide, and time can be measured through the timer setting section so that the ovulation test kit can be read accurately. Subsequently, when the ovulation test kit in contact with the sample is photographed using a smartphone camera, the reading method described above (e.g., steps (b) to (d)) can be performed on the captured image to identify the ROI area. At this time, since the test section of the ovulation test kit may change over time, the ROI area can be identified before the optimal time (e.g., 5 minutes) has passed through the timer setting. By analyzing the identified ROI image by performing the reading method described above (e.g., step (e)), the analysis result of the ovulation test kit can be obtained.

[0157] At this time, the process of reading the ROI image can be performed on a smartphone, or by an external device connected to the smartphone via wired or wireless means.

[0158] The above-mentioned results can be verified by the user through the test report and physiological report items.

[0159] The present invention is to be explained in more detail through the following examples, but the following examples are for illustrative purposes only and are not intended to limit the scope of the present invention.

[0160] [Example]

[0161] First, urine was applied to the test area of ​​the ovulation test kit. Next, the ovulation test kit reader installed on the smartphone was launched to place the ovulation test kit within a specific area, and the ovulation test kit was photographed using the smartphone camera (target image).

[0162] Next, the handle portion of the ovulation test kit was masked in the target image, and after identifying the grip area, the captured image of the smartphone was rotated by a predetermined angle so that the ovulation test kit would be in a horizontal state without tilting.

[0163] Next, after confirming whether the rotated ovulation test device is in a horizontal state, the target image is transformed using the CLAHE algorithm. At this time, if the ovulation test device is not horizontal, the step of identifying the grip area and rotating it by a predetermined angle can be performed again.

[0164] Next, the length of the ROI region was calculated from the image transformed by the above CLAHE algorithm, and the position of the ovulation test kit within the target image before transformation by the above CLAHE algorithm was adjusted, and the ROI region was cropped.

[0165] Subsequently, the cropped ROI image was transmitted to an external server or analyzed on the smartphone. Specifically, the size of the cropped ROI image was adjusted to 400 pixels * 80 pixels, the top and bottom of the ROI image were cropped to a certain length, and then grayscaled. Subsequently, a certain area on the left of the ROI image was returned as the inspection line area based on the center x-coordinate of the cropped ROI image, and a certain area on the right of the ROI image was returned as the control line area. At this time, the inspection line area and the control line area may each include independently colored lines, but may not include them.

[0166] Next, the average brightness value of pixels vertically aligned in the grayscale-processed ROI area is calculated and stored as a list of average brightness values, and then the brightness values ​​are sorted in descending order. A certain number of pixels are selected from the list of average brightness values ​​sorted in descending order to calculate the average brightness value of the background area, and by subtracting the average brightness value of the background area from the list of average brightness values ​​sorted in descending order, the brightness values ​​of the inspection line and the control line can be verified from the list of average brightness values.

[0167] Subsequently, if the intensity value (brightness value) of the above-mentioned control line is below a certain value, a reading failure result is returned, and if the intensity value of the above-mentioned control line is above a certain value, a reading result according to the intensity value of the inspection line is returned.

[0168] FIGS. 16 to 21 are exemplary drawings illustrating an image converted or processed from a target image during the reading process of an ovulation test kit according to one embodiment of the present invention. Specifically, FIG. 16 is a step of photographing an ovulation test kit, FIG. 17 is a target image in which the ovulation test kit is photographed, and FIG. 18 is a masked handle portion in the target image.

[0169] At this time, the grip area can be identified through four reference points in FIG. 18, and the target image can be rotated as in FIG. 19 so that the rectangle formed by the reference points is parallel to the image placement area (400). In this regard, FIG. 20 is a uniformized version of the target image in FIG. 17 when the grip area cannot be identified, and the grip area can be identified and then rotated by masking the handle part (221) in FIG. 20.

[0170] FIG. 21 is a cropped image of the ROI area identified in FIG. 19, and the length of the ROI area can be obtained by uniformizing the image of FIG. 19.

[0171] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical concept or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0172] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and the concept of equivalents thereof should be interpreted as being included within the scope of the present invention. Explanation of the symbols

[0174] 100 : Reader 200 : Strip tester 210 : Inspection Department 211 : Image-processed inspection unit 212 : ROI Area 220 : Handle part 221 : Image-processed handle 300, 310, 320 : Target image 400, 410: Image placement area

Claims

Claim 1 A method for reading a strip-type test device based on image analysis comprises: (a) acquiring a target image captured of the strip-type test device having an inspection part and a handle part that react with a specific component of a sample to produce color; (b) identifying a grip area corresponding to the handle part from the target image; (c) aligning the target image using shape information of the grip area; (d) identifying a Region of Interest (ROI) area corresponding to the inspection part from the aligned target image using the shape information and generating a cropped image including the ROI area; and (e) acquiring a reading result of the strip-type test device using the cropped image, wherein step (b) involves cropping the handle part of the strip-type test device and masking the cropped handle part to identify the grip area, or masking the handle part of the target image to identify the grip area, comprising: (b1) masking the handle part from the target image; and (b2) identifying a plurality of reference points corresponding to the corner area forming the handle part. and (b3) a step of identifying a grip area using coordinate information of a plurality of reference points; wherein step (c) includes (c1) a step of rotating the target image using coordinate information of a plurality of reference points constituting the grip area; and (c2) a step of applying a predetermined image processing to the rotated target image, wherein step (c1) includes a step of rotating the target image such that the long side of the grip area and the side in the width direction of the target image are parallel. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 delete Claim 6 A method for reading a strip-type tester according to claim 1, wherein step (c1) includes the step of rotating the target image so that the handle portion is placed on one side of the target image when the handle portion of the strip-type tester is placed on the other side of the target image. Claim 7 A method for reading a strip-type tester according to claim 1, wherein the step (c2) includes a step of homogenizing the image. Claim 8 A method for reading a strip-type tester according to claim 1, wherein the step (c2) further comprises: a step of identifying a plurality of reference points from the rotated target image; and a step of identifying a grip area of ​​the rotated target image using the coordinates of the reference points. Claim 9 A method for reading a strip-type test device according to claim 1, wherein step (e) comprises: (e1) a step of processing the cropped image; and (e2) a step of analyzing the control line and the inspection line of the image-processed ROI area. Claim 10 A method for reading a strip-type test device according to claim 9, wherein step (e2) comprises: (e21) a step of extracting a control line and a test line from the image-processed ROI area; (e22) a step of calculating the brightness values ​​of the control line and the test line; and (e23) a step of returning the reading result through the difference between the brightness value of the control line and the brightness value of the test line. Claim 11 A reading device for a strip-type tester based on image analysis comprises: an image acquisition unit for acquiring a target image captured of the strip-type tester, which is equipped with an inspection unit and a handle unit that react with a specific component of a sample to produce color; an object identification unit for identifying a grip area corresponding to the handle unit from the target image; an image alignment unit for aligning the target image using shape information of the grip area; an image cropping unit for identifying a Region of Interest (ROI) area corresponding to the inspection unit from the aligned target image using the shape information and generating a cropped image including the ROI area; and a reading execution unit for acquiring a reading result of the strip-type tester using the cropped image, wherein the object identification unit identifies the grip area by cropping the handle unit of the strip-type tester and masking the cropped handle unit, or identifies the grip area by masking the handle unit of the target image, and comprises: a masking unit for masking the handle unit of the target image; and a reference point identification unit for identifying a plurality of reference points corresponding to a corner area forming the handle unit from the target image. A reading device for a strip-type tester, comprising: a grip area identification unit that identifies a grip area by calculating coordinate information of the reference point; wherein the image alignment unit includes a rotation unit that rotates the target image using shape information of the grip area and an image processing unit that applies a predetermined image processing to the rotated target image, wherein the rotation unit rotates the target image such that the long side of the grip area and the side in the width direction of the target image are parallel. Claim 12 delete Claim 13 delete Claim 14 A reading device for a strip-type tester according to claim 11, wherein the image cropping unit comprises: an ROI identification unit that identifies an ROI region from the aligned target image; and a crop image generation unit that generates a crop image for the ROI region. Claim 15 A reading device for a strip-type tester according to claim 11, wherein the reading performing unit comprises: a preprocessing unit that processes the image before analyzing the cropped image; and an analysis unit that analyzes the inspection line and the control line from the image-processed cropped image. Claim 16 A reading device of a strip-type tester according to claim 11, wherein the reading device further comprises an output unit for displaying the reading result.

Images