Position matching method of scalp images and hair loss diagnosis method using same

Abstract

The present invention relates to a position matching method of scalp images. The position matching method of scalp images includes the steps of: detecting feature points in images captured of a scalp; applying a Delaunay triangulation algorithm to the detected feature points; and calculating similarity by converting into a relative coordinate system. The formula for calculating similarity is [Formula 1]. f is a feature vector for each of three vertices p1, p2, and p3 of a triangle, and only triangles having the greatest similarity among feature vectors for all triangles included in the two images IA and IB are selected, and the sum of the selected triangles is calculated. The present invention has the effect that position differences in scalp images can be easily determined by deriving similarity after applying a Delaunay triangulation algorithm to feature points detected in the scalp images.

Description

Scalp image positioning method and hair loss diagnosis method using the same

[0001] The present invention relates to a method for aligning the positions of scalp images, and more particularly, to a method for aligning the positions of scalp images to match the positions of hair loss diagnosis.

[0002] Generally, hair loss is defined as the absence of hair in an area where hair should be present, and more specifically, it refers to the phenomenon in which hair originally present on the scalp gradually falls out over time.

[0003] Hair loss is known to be caused by genetic factors and aging, but recently, hair loss due to acquired causes such as environmental pollution, stress, and hormonal imbalances due to changes in diet has also been increasing, and these are acting as factors that accelerate the rate of hair loss.

[0004] Since hair loss has a significant impact on one's appearance, various hair loss-related technologies are being developed, and among them, checking and monitoring the progress of hair loss is very important.

[0005] Various technologies for analyzing scalp images to diagnose hair loss are being developed, and recently, along with the use of artificial intelligence through machine learning in the field of medical diagnosis, including medical image analysis, technologies for applying artificial intelligence to the analysis of scalp images (Korean Patent No. 10-2536416) are being developed.

[0006] Meanwhile, when analyzing hair loss, it's crucial to identify and analyze changes over time. Comparing images taken at different times can be problematic. This is because comparing multiple images of the human scalp often presents challenges due to differences in shooting ranges and other factors, making it difficult to compare images of the same location. To address this issue, a technology has been developed (Korean Patent No. 10-2303429) that extracts and compares characteristic information for each pore in scalp images. However, the process of extracting characteristic information for each pore is too complex.

[0007] The present invention is intended to solve the problems of the above-mentioned prior art and to provide a new method for aligning the positions of different images taken of the scalp.

[0008] The method for positioning a scalp image according to the present invention for achieving the above purpose includes the steps of: detecting feature points for an image of a scalp; applying a Delaunay triangulation algorithm to the detected feature points; and calculating similarity by converting to a relative coordinate system, and the formula for calculating similarity is:

[0009]

[0010]

[0011]

[0012] , f is a feature vector for each of the three vertices p1, p2, and p3 of the triangle, and two images I A Wow I B It is characterized by selecting only the triangles with the most similarity among the feature vectors for all triangles included and calculating their sum.

[0013] It is preferable that the above-mentioned characteristic points are hair follicles detected in an image of the scalp.

[0014] If the above similarity is below a certain value, it can be determined that it is the same location.

[0015] The step of detecting feature points can be performed by an artificial neural network trained based on an object detection methodology.

[0016] According to another aspect of the present invention, a hair loss diagnosis method comprises the steps of: generating a reference index of a scalp image by stitching together scalp images photographed at different locations in a panoramic form; photographing a current scalp image for hair loss diagnosis; aligning a position photographed in the current scalp image with respect to the reference index to derive a position corresponding to the current scalp image from the reference index; and comparing the number or thickness of hair in the reference index and the current scalp image, wherein the position matching method comprises the steps of: detecting feature points in an image in which the scalp is photographed; applying a Delaunay triangulation algorithm to the detected feature points; and calculating a similarity by converting to a relative coordinate system.

[0017] The formula for calculating similarity is:

[0018]

[0019]

[0020] , f is a feature vector for each of the three vertices p1, p2, and p3 of the triangle, and two images I A Wow I B It is characterized by selecting only the triangles with the most similarity among the feature vectors for all triangles included and calculating their sum.

[0021] You can compare the number or thickness of hairs connected to hair follicles located within the comparison range.

[0022] It is preferable that the above-mentioned characteristic points are hair follicles detected in an image of the scalp.

[0023] If the above similarity is below a certain value, it can be determined that it is the same location.

[0024] The step of detecting feature points can be performed by an artificial neural network trained based on an object detection methodology.

[0025] The present invention, configured as described above, has the effect of easily determining the difference in location of a scalp image by applying the Delaunay triangulation algorithm to feature points detected from a scalp image and then deriving similarity.

[0026] FIG. 1 is a drawing for explaining a method for positioning a scalp image according to an embodiment of the present invention.

[0027] FIG. 2 is an example of labeling hair follicles in scalp images captured by three hardware in a scalp image position alignment method according to an embodiment of the present invention.

[0028] Figure 3 is a diagram comparing the results of hair follicle exploration before and after NMS application.

[0029] FIG. 4 is a drawing for explaining a method for positioning a scalp image according to an embodiment of the present invention.

[0030] Figure 5 is a drawing for explaining a hair loss diagnosis method according to the first embodiment of the present invention.

[0031] FIG. 6 is a drawing for explaining a hair loss diagnosis method according to a second embodiment of the present invention.

[0032] An embodiment of the present invention is described in detail with reference to the attached drawings.

[0033] However, the embodiments of the present invention may be modified in various other forms, and the scope of the present invention is not limited to the embodiments described below. The shapes and sizes of elements in the drawings may be exaggerated for clearer explanation, and elements indicated by the same symbols in the drawings are the same elements.

[0034] Throughout the specification, when a part is said to be "connected" to another part, this includes not only the cases where it is "directly connected" but also the cases where it is "electrically connected" with another element in between. Furthermore, when a part is said to "include" or "comprise" a component, this does not mean that it excludes other components, but rather that it can include or comprise other components, unless otherwise specifically stated.

[0035] Additionally, terms such as "first," "second," etc. are intended to distinguish one component from another and should not be construed as limiting the scope of the rights. For example, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component.

[0036]

[0037] FIG. 1 is a drawing for explaining a method for positioning a scalp image according to an embodiment of the present invention.

[0038] First, in this embodiment, a hair follicle is used as a feature point for position alignment of a scalp image.

[0039] A hair follicle is a skin organ that produces hair. It surrounds and nourishes the hair root within the dermis. Also known as a hair pouch or hair sheath, hair grows from the scalp. One or more hairs grow from a single follicle. In this example, it is estimated that one to five hairs can grow from a single follicle.

[0040] The hair follicles used in this embodiment can also be changed to pores. Generally, a pore refers to the area where hair emerges from the scalp, and multiple hairs growing from the same hair follicle pass through the same pore. In the present invention, when a pore is used as a characteristic feature, it refers to the pores of hairs growing from the same follicle.

[0041]

[0042] As hair loss progresses, the number of hairs growing from a single follicle may decrease. Therefore, hair follicles exhibit relatively minimal or slowest changes compared to the number and thickness of hairs captured in scalp images. Therefore, utilizing hair follicles as landmarks can improve the efficiency of position matching.

[0043] The process of detecting feature points using hair follicles in a scalp image can be performed by an artificial neural network trained based on an object detection methodology. In this embodiment, an artificial neural network model with an optimized structure was constructed by modifying yolov8 (You look only once), but is not limited thereto.

[0044] The trained artificial neural network extracts the relative coordinates of all candidate hair follicles in a scalp image. Here, a follicle-based feature point represents the central point where hair clusters emerge from a specific follicle. A hair follicle can contain between one and five hairs, including black, white, dyed, and bleached hairs. By recognizing the number of hairs as an object class, the locations of hair follicles with m hairs can be extracted as feature points. If a hair follicle is not present in the image, the image is recognized as not capturing the scalp or skin, and a re-capture may be requested.

[0045] The training data for the artificial neural network used to detect these features consisted of magnified scalp images taken using three different hardware and from individuals of Asian and Western ethnicities. Additionally, negative samples (data not taken from the scalp) were added to increase reproducibility. The labeling rules are as follows:

[0046] - One hair follicle can have at least 1 and up to 5 hairs, including black hair, white hair, dyed hair, and bleached hair.

[0047] - When a single hair follicle contains multiple hair follicles, the center point of the multiple hair follicles should be the center of the object recognition range.

[0048] - The object recognition range must be larger than the hair follicle, and the starting point of every hair on the scalp must be observed.

[0049] - The object recognition ranges of different hair follicles should never overlap.

[0050] - If the center point of the hair follicle is ambiguous and difficult to define, label the hair follicles as separate hair follicles in an area where the object recognition ranges do not overlap.

[0051] FIG. 2 is an example of labeling hair follicles in scalp images captured by three hardware in a scalp image position alignment method according to an embodiment of the present invention.

[0052] As shown, when labeling pores, it can be confirmed that parts that would be labeled as separate pores are labeled as one hair follicle by labeling hair follicles.

[0053] However, the hair follicles detected by the learned artificial neural network may be detected with unclear locations for hair follicles from which multiple hairs grow, and all hair follicle candidates obtained from the artificial neural network can be confirmed as one hair follicle through a specific threshold value and NMS (non-maximum-suppression) algorithm.

[0054] Figure 3 is a diagram comparing the results of hair follicle exploration before and after NMS application.

[0055] Applying the Delaunay triangulization algorithm to the feature points of the scalp image selected through the above process creates triangles based on the feature points and connects them. The graph is then converted to a relative coordinate system, which is then used to calculate the similarity between the two images. The formula for calculating the similarity is as follows.

[0056]

[0057]

[0058] In the above formula, f is a feature vector for each of the three vertices p1, p2, and p3 of the triangle (hereinafter, 'triangle') generated by applying the Delaunay triangulation algorithm and then converting to a relative coordinate system, and two images I A Wow I B Compute the similarity of feature vectors for all triangles included in .

[0059] At this time, the number of triangles included in the two images may not be the same, but since only the triangles with the most similarity among the triangle feature vectors included in the two images are selected and the sum is calculated as the similarity between the images, the similarity can be calculated even when the number of triangles included in the two images is not the same.

[0060] FIG. 4 is a drawing for explaining a method for positioning a scalp image according to an embodiment of the present invention.

[0061] In order to verify the effectiveness of the above-mentioned similarity formula, first, a reference image was taken, and then multiple comparison images were taken while slightly moving the shooting position, and the similarity was calculated for each comparison image with respect to the reference image through the above-mentioned process, and is shown in Table 1.

[0062] [Table 1]

[0063]

[0064] And for comparison, multiple comparison images were taken with the shooting position moved a lot from the reference image, and the similarity was calculated for each comparison image with respect to the reference image through the above process, and is shown in Table 2.

[0065] [Table 2]

[0066]

[0067]

[0068] Table 1 shows images taken while moving only within a range where the same hair follicles could be overlapped and captured, and the calculated similarity values ​​showed low values.

[0069] In contrast, in Table 2, for images taken by moving so far that the same hair follicle was not captured, it can be confirmed that the calculated similarity value has a much larger value than in Table 1.

[0070] Accordingly, according to the embodiment of the present invention, if the similarity applied is calculated to be large, the identity between the two images can be evaluated as low, and if the similarity is calculated to be small, the identity between the two images can be evaluated as high. Furthermore, if the similarity is calculated to be smaller than a predetermined value, it can be determined that the positions of the captured images are identical enough to be applied to hair loss diagnosis. For example, in the present embodiment, if the calculated similarity is 500 or less, it is determined that the positions of the captured images are identical enough to be applied to hair loss diagnosis. Since the similarity value varies depending on the magnification and image resolution, the similarity value for determining identity can vary depending on the magnification and image resolution.

[0071] Using the above method, the shooting positions of scalp images taken at different times can be compared and aligned.

[0072] First, the process of aligning the positions can be applied by finding a comparison image with high similarity to a reference image fixed at a predetermined position, as shown in the table above, and this can be used in the form of taking the latest scalp image for hair loss diagnosis at the same position as the scalp image taken earlier in time series.

[0073] Conversely, after taking the latest scalp image for hair loss diagnosis, it can be used to find the location corresponding to the latest scalp image in the images that were taken first in time series and were taken relatively widely.

[0074]

[0075] Figure 5 is a drawing for explaining a hair loss diagnosis method according to the first embodiment of the present invention.

[0076] The hair loss diagnosis method of the present invention is a method of diagnosing hair loss by comparing images of the scalp taken in time series, and in this embodiment, hair loss can be diagnosed through changes in the number of hairs over time.

[0077] First, a baseline index of the scalp image that serves as the basis for comparison is created.

[0078] The reference index of this embodiment is generated by combining scalp images that were first captured in time series, and by stitching together scalp images captured at different locations in a panoramic form, a reference index having a larger area than the area of ​​the scalp included in a single captured image is generated.

[0079] At this time, it is desirable for the combined scalp images to be captured at the same or similar times, and images captured within a given period of time can be used. Alternatively, images captured from videos of the scalp captured while moving can be used.

[0080] There is no particular limitation on the method for stitching scalp images into a panorama, but in this embodiment, the scalp images are stitched into a panorama using a feature point matching algorithm.

[0081] Next, an image of the current scalp is taken to diagnose hair loss.

[0082] And, by applying the above-described scalp image position matching method, the shooting position of the currently captured scalp image is aligned with respect to the reference landmark. In order to compare the change in hair loss based on a specific image, shooting at the same position as the specific image is required, but it is difficult to shoot the same position when shooting the scalp using a magnifying glass, and this is especially true when the subject of the diagnosis directly shoots the scalp. However, in this embodiment, since the position of the currently captured scalp image is aligned with the reference landmark that is stitched together in a panoramic form, the position of the reference landmark corresponding to the currently captured scalp image can be found and compared, and this can solve the problem of performing repeated shooting to shoot the same position as the shooting position of a specific image.

[0083] And, in the reference index, the number of hairs in the part corresponding to the current scalp image is compared with the number of hairs in the current scalp image.

[0084] At this time, the number of hairs to be compared is the number of hairs connected to the hair follicles within the comparison range, and by comparing the number of hairs based on this standard, the number of hairs can be compared without being affected by the direction of the hair, etc.

[0085] The process of comparing the number of hairs is not particularly limited, but an artificial neural network trained to detect objects can be used, and specifically, an artificial neural network that detects hair as an object around a hair follicle can be used.

[0086] According to the above method, hair loss can be diagnosed by comparing the number of hairs growing from the same location over time.

[0087]

[0088] FIG. 6 is a drawing for explaining a hair loss diagnosis method according to a second embodiment of the present invention.

[0089] The hair loss diagnosis method of the present invention is a method of diagnosing hair loss by comparing images of the scalp taken in time series, and in this embodiment, hair loss can be diagnosed through changes in hair thickness over time.

[0090] In this embodiment, the details of shooting and position alignment of the reference index and the current image are the same as in the first embodiment, so they are omitted.

[0091] And by applying the above-mentioned scalp image position matching method, the hair thickness of the part corresponding to the current scalp image is compared with the hair thickness of the current scalp image in the reference index.

[0092] At this time, the thickness of the hair to be compared is the thickness of the hair connected to the hair follicle within the comparison range, and if the thickness of the hair is compared based on this standard, the thickness of the hair can be compared without being affected by the direction of the hair, etc.

[0093] The process of comparing hair thickness is not particularly limited, but an artificial neural network trained to detect objects can be used, and specifically, an artificial neural network trained based on an image segmentation methodology can be used to crop the area around the hair follicle and compare the thickness.

[0094] According to the above method, hair loss can be diagnosed by comparing the thickness of hair growing from the same location in a time series manner.

[0095]

[0096] The present invention has been described above through preferred embodiments. However, the above-described embodiments are merely illustrative of the technical idea of ​​the present invention. Those skilled in the art will understand that various changes may be made without departing from the technical idea of ​​the present invention. Therefore, the scope of protection of the present invention should be interpreted not by specific embodiments, but by the matters described in the claims, and all technical ideas within a scope equivalent thereto should be interpreted as being included within the scope of the rights of the present invention.

Claims

1. A step of detecting feature points from an image of a scalp; A step of applying the Delaunay triangulation algorithm to the detected feature points; and It includes a step of calculating similarity by converting to a relative coordinate system, The formula for calculating similarity is: , and f is one of the three vertices p of the triangle. 1 , p 2 , p 3 is a feature vector for each of the two images I A Wow I B A scalp image alignment method characterized by selecting only the triangles with the most similarity among the feature vectors for all triangles included in and calculating their sum.

2. In claim 1, A method for aligning the position of a scalp image, characterized in that the above-mentioned feature points are hair follicles detected from an image of the scalp.

3. In claim 1, A method for aligning scalp images, characterized in that it is determined that the positions are the same when the similarity is less than a predetermined value.

4. In claim 1, A method for position alignment of a scalp image, characterized in that the step of detecting feature points is performed by an artificial neural network learned based on an object detection methodology.

5. A step of creating a reference index of scalp images by stitching together scalp images taken at different locations in a panoramic form; Steps to take current scalp images for hair loss diagnosis; A step of aligning the location captured in the current scalp image with respect to a reference index and deriving a location corresponding to the current scalp image from the reference index; and Comprising a step of comparing the number or thickness of hairs in a reference index and a current scalp image, The above position alignment method, A step of detecting feature points in an image of a scalp; A step of applying the Delaunay triangulation algorithm to the detected feature points; and It includes a step of calculating similarity by converting to a relative coordinate system, The formula for calculating similarity is: , and f is one of the three vertices p of the triangle. 1 , p 2 , p 3 is a feature vector for each of the two images I A Wow I B A hair loss diagnosis method characterized in that only the triangles with the most similarity are selected from among the feature vectors for all triangles included in and their sum is calculated.

6. In claim 5, A hair loss diagnosis method characterized by comparing the number or thickness of hairs connected to hair follicles located within a comparison range.

7. In claim 5, A hair loss diagnosis method characterized in that the above-mentioned characteristic points are hair follicles detected in an image of the scalp.

8. In claim 5, A hair loss diagnosis method characterized in that it is determined that the location is the same when the similarity is below a predetermined value.

9. In claim 5, A hair loss diagnosis method, characterized in that the step of detecting feature points is performed by an artificial neural network learned based on an object detection methodology.

Images