A method and system for combined detection of hair follicle location and growth direction of hair

Abstract

The application discloses a kind of hair follicle position and growth direction combined detection method and system of hair, belong to hair detection technical field, comprising: obtaining a plurality of hair mirror images and all hair follicle points are single point marked to construct data set;Data set is divided into training set and test set, and the image in training set is carried out random data enhancement;The training set after random data enhancement is used to train hair follicle point prediction model and test set is used to test, obtain the coordinate of all hair follicle points in the hair mirror image to be measured that is predicted by trained hair follicle point prediction model;According to the coordinate of each hair follicle point in the hair mirror image to be measured, based on local skeleton constraint is iteratively tracked to obtain the final hair growth direction of corresponding hair follicle point;The coordinate of each hair follicle point in the hair mirror image to be measured and final hair growth direction are used as the combined detection result of the hair mirror image to be measured. Training data labeling cost can be reduced, avoid growth direction semantic ambiguity, and detection accuracy is high.

Description

Technical Field

[0001] This invention belongs to the field of hair detection technology, specifically relating to a method and system for the joint detection of hair follicle location and growth direction. Background Technology

[0002] In the field of computer vision, the accurate detection and localization of slender objects (such as hair) has significant practical application value, especially in biomedical imaging. Since scalp examinations often require manual counting of hair quantity and growth direction using a trichoscope, this method is highly dependent on human experience and is particularly time-consuming and labor-intensive in complex scenarios such as high-density areas or where hair cross-occlusion occurs. Furthermore, current technologies typically use deep learning models to estimate the severity and morphology of hair follicle loss.

[0003] However, existing methods still have the following limitations when facing the requirements of high accuracy and robustness: First, they usually require a large amount of high-quality labeled data for training to meet the requirements of intraoperative applications. However, related labeling methods, such as rotating bounding boxes, are cumbersome, and the consistency of labeling is difficult to guarantee in complex scenarios such as dense hair, occlusion, and curvature, resulting in high data labeling costs. In addition, besides hair follicle localization, it is also necessary to obtain the growth direction starting from the hair follicle to support angle planning and research. However, the directional expression of rotating bounding box annotations is prone to semantic ambiguity of uncertain directions, which in turn affects the reliability of angle planning and research. Summary of the Invention

[0004] The purpose of this invention is to address the above-mentioned problems by proposing a method and system for joint detection of hair follicle location and growth direction, which can obtain more reliable information on hair follicle location and hair growth direction, reduce the cost of training data annotation, avoid semantic ambiguity of growth direction, and achieve high detection accuracy.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] The present invention proposes a method for jointly detecting the location and growth direction of hair follicles, comprising the following steps:

[0007] S1. Obtain several hair follicle images and perform single-point annotation on all hair follicle points in each hair follicle image to construct a dataset;

[0008] S2. Divide the dataset into a training set and a test set, and perform random data augmentation on the images in the training set;

[0009] S3. The hair follicle point prediction model is trained using a training set with random data augmentation and tested using a test set to obtain a trained hair follicle point prediction model. The trained hair follicle point prediction model is used to predict the coordinates of all hair follicle points in the hair microscope image to be tested. The hair follicle point prediction model includes an encoder, a decoder, a detection head, and a post-processing module connected in sequence.

[0010] S4. Based on the coordinates of each hair follicle point in the hair microscope image to be tested, perform iterative tracking based on local skeleton constraints to obtain the final hair growth direction of the corresponding hair follicle point.

[0011] S5. The coordinates of each hair follicle point in the hair microscopic image to be tested and the final hair growth direction are used as the joint detection result of the hair microscopic image to be tested.

[0012] Preferably, the encoder includes four sequentially connected encoding layers, and the decoder includes four sequentially connected decoding layers, wherein the first to fourth encoding layers of the encoder are sequentially connected to the fourth to first decoding layers of the decoder in a one-to-one correspondence, wherein:

[0013] The encoding layer comprises a first convolutional module, an attention module, and a max pooling layer connected in sequence. The first convolutional module includes a convolutional layer, a normalization layer, a ReLU activation function, another convolutional layer, a normalization layer, and a ReLU activation function connected in sequence. The attention module includes a channel attention module and a spatial attention module connected in sequence. The channel attention module includes a first pooling unit, a convolutional layer, a ReLU activation function, a second pooling unit, a first addition operation, a sigmoid activation function, and a first multiplication operation connected in sequence. Both the first and second pooling units include parallel global average pooling layers and global max pooling layers. The first addition operation is used to multiply the second... The output features of the global average pooling layer and the output features of the global max pooling layer of the pooling unit are added together. The first multiplication operation is used to multiply the output features of the sigmoid activation function of the channel attention module and the input features of the first pooling unit to obtain channel enhancement features. The spatial attention module includes a third pooling unit, a convolutional layer, a sigmoid activation function and a second multiplication operation connected in sequence. The third pooling unit includes a global average pooling layer and a global max pooling layer connected in sequence. The second multiplication operation is used to multiply the output features of the sigmoid activation function of the spatial attention module and the input features of the third pooling unit to obtain spatial enhancement features.

[0014] The decoding layer includes an upsampling module, a second addition operation, and a second convolution module connected in sequence. The upsampling module includes an upsampling layer, a convolutional layer, a normalization layer, and a ReLU activation function connected in sequence. The second addition operation is used to skip-connect the output features of the upsampling module and the output features of the attention module of the corresponding encoding layer. The second convolution module is the same as the first convolution module.

[0015] The detection head consists of a convolutional layer, a normalization layer, a ReLU activation function, another convolutional layer, another convolutional layer, a normalization layer, a ReLU activation function, another convolutional layer, and a Sigmoid activation function connected in sequence.

[0016] The post-processing module is used to extract features from the output features of the detection head using a non-maximum suppression algorithm to obtain the coordinates of the hair follicle points in the corresponding hair mirror image.

[0017] Preferably, based on the coordinates of each hair follicle point in the hair microscopic image to be tested, iterative tracking is performed based on local skeleton constraints to obtain the final hair growth direction of the corresponding hair follicle point, as follows:

[0018] S41. After performing grayscale and binarization on the hair mirror image to be tested, the global skeleton map is obtained by Zhang-Suen thinning algorithm. Based on the coordinates of each hair follicle point in the hair mirror image to be tested, the hair shaft skeleton path of the corresponding hair follicle point in the global skeleton map is extracted respectively.

[0019] S42. Treat the hair shaft skeleton path of each hair follicle point as the local skeleton constraint of the corresponding hair follicle point and iteratively track it to obtain the final hair growth direction of the corresponding hair follicle point. The final hair growth direction of a hair follicle point is obtained as follows:

[0020] S421. Construct a rotating window with the hair follicle point as the midpoint of the bottom edge of the rectangle and calculate the grayscale response and coverage response of the rotating window at several rotation angles centered on the coordinates of the hair follicle point. Weight the grayscale response and coverage response of the corresponding rotation angle to obtain the comprehensive response of the corresponding rotation angle. Take the rotation angle corresponding to the maximum comprehensive response of the rotation angle as the prior direction angle.

[0021] S422. Perform principal component analysis on the neighborhood of each hair follicle point, and use the prior direction angle to correct the principal component direction obtained by the principal component analysis to obtain the initial growth direction as the starting direction. The neighborhood of the hair follicle point is the rotation window corresponding to the prior direction angle.

[0022] S423. Iteratively track along the corresponding hair shaft skeleton path from the starting direction of the hair follicle point with a preset step size to generate a candidate direction set for the corresponding hair follicle point, and select the candidate direction with the highest consistency with the historical growth direction set as the growth direction of the current iteration step.

[0023] S424. Update the growth direction of the current iteration step of the corresponding hair follicle point to the starting direction of the corresponding hair follicle point, and return to execute step S423 until the iteration tracking terminates when the iteration step reaches the preset threshold or the end point of the hair shaft skeleton path is reached. Take the point at which the iteration tracking terminates as the termination point, and calculate the direction vector from the hair follicle point to the termination point as the final hair growth direction of the corresponding hair follicle point.

[0024] Preferably, the hair shaft skeleton path of each hair follicle point in the global skeleton map is extracted based on the coordinates of each hair follicle point in the hair microscope image to be tested, as follows:

[0025] S411. Within a rectangular neighborhood of a preset size for the coordinates of each hair follicle point in the hair microscope image to be tested, calculate the distance between the hair follicle point and the pixel point of the skeleton in the global skeleton image, and select the pixel point of the skeleton in the global skeleton image that is closest to the hair follicle point as the starting point for tracking the hair growth direction of the corresponding hair follicle point.

[0026] S412. Determine the 8-neighbor connectivity of the starting point of each hair follicle point, that is, check whether there are pixels of the skeleton in the global skeleton graph in the 8-neighborhood of the starting point of each hair follicle point. If they exist and have not been added to the hair shaft skeleton path of the corresponding hair follicle point, add the pixels of the skeleton to the hair shaft skeleton path of the corresponding hair follicle point and use it as a new starting point to repeat step S412 until there are no new pixels of the skeleton. Extract the hair shaft skeleton path of the corresponding hair follicle point.

[0027] Preferably, the grayscale response and coverage response of the rotating window are calculated at several rotation angles centered on the coordinates of the hair follicle point. Then:

[0028] Rotation angle is grayscale response at time The formula is as follows:

[0029]

[0030] in, Indicates the rotation angle as The set of pixels covered by the rotating window at that time. for The number of pixels, For the first The grayscale value of each pixel To preset the grayscale threshold, For correction functions;

[0031] Rotation angle is Coverage response time The formula is as follows:

[0032]

[0033] in, For the first The mask value for each pixel.

[0034] Preferably, the initial growth direction is given by the following formula:

[0035]

[0036] in, The initial growth direction, For symbolic functions, For transpose, The principal direction vector is formed after normalizing the principal component directions. The prior direction and angle.

[0037] Preferably, the candidate direction set for hair follicle points is obtained as follows:

[0038] Centered on the growth direction of the previous iteration, several candidate angles are generated according to different offset angles to form a candidate angle set. For each candidate angle, a corresponding rotation window is generated and principal component analysis is performed to obtain the principal component direction of the hair at the corresponding hair follicle point as the candidate direction. All candidate directions of a hair follicle point constitute the candidate direction set of the corresponding hair follicle point.

[0039] Preferably, the candidate direction with the highest consistency with the historical growth direction set is selected from the candidate direction set as the growth direction for the current iteration step. Next candidate angle is Consistency between the candidate direction and the set of historical growth directions The calculation is as follows:

[0040]

[0041] in, The current iteration step Next candidate angle The principal direction vector is obtained by principal component analysis using a rotating window. , The current iteration step The following is the candidate angle set. Number of iterations The growth direction corresponding to the hair follicle point. Number of iterations The weights below, It is an absolute value.

[0042] Preferably, the test set is used for testing, specifically by calculating the average Euclidean distance between the coordinates of the hair follicle points predicted by the trained hair follicle point prediction model and the coordinates of the actual hair follicle points, and taking the trained hair follicle point prediction model with the smallest average Euclidean distance as the trained hair follicle point prediction model.

[0043] A hair follicle location and growth direction joint detection system includes a processor and a memory, the memory being used to store a computer program, which, when executed by the processor, implements the hair follicle location and growth direction joint detection method as described above.

[0044] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0045] 1) Reduce annotation costs: Using single-point annotation for hair follicle localization reduces reliance on high-cost annotation methods such as bounding boxes, thus reducing the annotation costs of training data from the source.

[0046] 2) Avoid semantic ambiguity in hair growth direction: Accurate modeling is performed based on the neighborhood information of hair follicles, and the hair growth direction is estimated by combining local skeleton constraints. This avoids the uncertainty of forward and reverse directions in the traditional rotating box annotation method, thereby eliminating semantic ambiguity in direction and ensuring the reliability and consistency of hair growth direction. Attached Figure Description

[0047] Figure 1 This is a flowchart of the method for jointly detecting the hair follicle location and growth direction of hair according to the present invention;

[0048] Figure 2 This is a schematic diagram of the structure of the hair follicle point prediction model of the present invention;

[0049] Figure 3 This is a single-point annotation result diagram of hair follicle points in different hair microscope images in step S1 of the present invention;

[0050] Figure 4 The figures shown are grayscale images, binary images, and global skeleton diagrams of the hair microscope image to be tested according to the present invention. Figure (a) is the grayscale image of the hair microscope image to be tested, Figure (b) is the binary image of the hair microscope image to be tested, and Figure (c) is the global skeleton diagram of the hair microscope image to be tested.

[0051] Figure 5 These are schematic diagrams illustrating the construction of the rotating window at different rotation angles and the construction of candidate angles according to the present invention.

[0052] Figure 6 This is a diagram showing the final growth direction detection results of hair follicle points in different hair microscope images in step S1 of the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0054] It should be noted that, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application.

[0055] Example 1:

[0056] like Figures 1-6 As shown, a method for jointly detecting the location and growth direction of hair follicles includes the following steps:

[0057] S1. Obtain several hair follicle images and perform single-point annotation on all hair follicle points in each hair follicle image to construct a dataset.

[0058] Specifically, in this embodiment, a dataset is constructed by single-point annotation of all hair follicle points in 774 hair microscope images. Single-point annotation can be performed using existing annotation tools such as LabelMe to annotate each hair follicle point in the hair microscope images and output the annotated hair microscope images. Some of the annotated hair microscope images in the dataset are shown below. Figure 3 As shown in Figures (a), (b), and (c), the yellow dots represent the locations of the hair follicle points marked as single points.

[0059] S2. Divide the dataset into a training set and a test set, and perform random data augmentation on the images in the training set.

[0060] Specifically, in this embodiment, the dataset (input data) is divided into a training set and a test set in an 8:2 ratio. Random data augmentation with a 50% probability is applied to the single-point labeled hair mirror images in the training set. This random data augmentation includes random vertical and horizontal flipping and brightness transformation. Therefore, the training set after random data augmentation includes 50% of the single-point labeled hair mirror images from the training set and 50% of the single-point labeled hair mirror images from the training set after random data augmentation. Vertical and horizontal flipping means performing a vertical flip first, followed by a horizontal flip.

[0061] S3. The hair follicle point prediction model is trained using a training set with random data augmentation and tested using a test set to obtain a trained hair follicle point prediction model. The trained hair follicle point prediction model is then used to predict the coordinates of all hair follicle points in the hair microscope image to be tested. The hair follicle point prediction model includes an encoder, a decoder, a detection head, and a post-processing module connected in sequence.

[0062] In one embodiment, the encoder includes four sequentially connected encoding layers, and the decoder includes four sequentially connected decoding layers, wherein the first to fourth encoding layers of the encoder are sequentially connected to the fourth to first decoding layers of the decoder in a one-to-one correspondence, wherein:

[0063] The encoding layer comprises a first convolutional module, an attention module, and a max pooling layer connected in sequence. The first convolutional module includes a convolutional layer, a normalization layer, a ReLU activation function, another convolutional layer, a normalization layer, and a ReLU activation function connected in sequence. The attention module includes a channel attention module and a spatial attention module connected in sequence. The channel attention module includes a first pooling unit, a convolutional layer, a ReLU activation function, a second pooling unit, a first addition operation, a sigmoid activation function, and a first multiplication operation connected in sequence. Both the first and second pooling units include parallel global average pooling layers and global max pooling layers. The first addition operation is used to multiply the second... The output features of the global average pooling layer and the output features of the global max pooling layer of the pooling unit are added together. The first multiplication operation is used to multiply the output features of the sigmoid activation function of the channel attention module and the input features of the first pooling unit to obtain channel enhancement features. The spatial attention module includes a third pooling unit, a convolutional layer, a sigmoid activation function and a second multiplication operation connected in sequence. The third pooling unit includes a global average pooling layer and a global max pooling layer connected in sequence. The second multiplication operation is used to multiply the output features of the sigmoid activation function of the spatial attention module and the input features of the third pooling unit to obtain spatial enhancement features.

[0064] The decoding layer includes an upsampling module, a second addition operation, and a second convolution module connected in sequence. The upsampling module includes an upsampling layer, a convolutional layer, a normalization layer, and a ReLU activation function connected in sequence. The second addition operation is used to skip-connect the output features of the upsampling module and the output features of the attention module of the corresponding encoding layer. The second convolution module is the same as the first convolution module.

[0065] The detection head consists of a convolutional layer, a normalization layer, a ReLU activation function, another convolutional layer, another convolutional layer, a normalization layer, a ReLU activation function, another convolutional layer, and a Sigmoid activation function connected in sequence.

[0066] The post-processing module is used to extract features from the output features of the detection head using a non-maximum suppression algorithm to obtain the coordinates of the hair follicle points in the corresponding hair mirror image.

[0067] In one embodiment, testing with a test set specifically involves calculating the average Euclidean distance between the coordinates of the hair follicle points predicted by the trained hair follicle point prediction model and the coordinates of the actual hair follicle points, and taking the trained hair follicle point prediction model that has the smallest average Euclidean distance as the trained hair follicle point prediction model.

[0068] Specifically, the hair follicle prediction model is trained using a training set augmented with random data. The hair follicle prediction model is as follows: Figure 2 As shown, it includes an encoder, a decoder, a detection head, and a post-processing module connected in sequence. The encoder includes four coding layers (denoted as coding layer 1, coding layer 2, coding layer 3, and coding layer 4 in sequence), as shown in the coding layer diagram. ( The first convolutional module consists of a 3×3 convolutional layer, a normalization layer, and a ReLU activation function connected in sequence. Figure 2 The "middle convolution + normalization + R" refers to a convolutional layer, a normalized layer, and a ReLU activation function. After downsampling in the first convolutional module, the data enters the attention module. The channel attention module then processes the feature map... (for the coding layer) The input features of the channel attention module are subjected to global average pooling and global max pooling respectively, resulting in two 1×1 channel vectors. The two channel vectors share a multilayer perceptron consisting of 1×1 convolutional layers, ReLU activation functions, and 1×1 convolutional layers. After passing through the second pooling unit, the outputs of the two branches are added and then subjected to a Sigmoid activation function to obtain the channel weights. These channel weights are then combined with the input features (feature maps). Channel enhancement features are obtained by multiplying each channel sequentially. Spatial attention submodule enhances channel features Global average pooling and global max pooling are performed along the channel dimension respectively, and the results are concatenated along the channel dimension. Then, spatial weights are obtained by passing a 7×7 convolutional layer and a sigmoid activation function. These spatial weights are then combined with the channel enhancement features. Spatial augmentation features are obtained by pixel-wise multiplication. The decoder consists of four decoding layers (decoded as decoding layer 1, decoding layer 2, decoding layer 3, and decoding layer 4). Each decoding layer is sequentially connected to an upsampling layer, a 3×3 convolutional layer, a normalization layer, and a ReLU activation function. The output features of the corresponding encoding layer are then concatenated via skip connections before being connected to a second convolutional module. This second convolutional module consists of sequentially connected 3×3 convolutional layers, a normalization layer, and a ReLU activation function. The detection head consists of sequentially connected convolutional layers, normalization layers, ReLU activation functions, convolutional layers, convolutional layers, normalization layers, ReLU activation functions, convolutional layers, and a Sigmoid activation function, used to output a single-channel hair follicle point probability map. The post-processing module performs non-maximum suppression on the hair follicle point probability map to extract the coordinates of the corresponding hair follicle points in the trichome image.

[0069] During training, a cosine annealing decay strategy was used to dynamically adjust the learning rate, with an initial learning rate of 0.005. The Adam optimizer was used for parameter updates, with a batch size of 4 and a maximum of 800 training iterations. After the training phase, the convergence performance of the hair follicle prediction model was evaluated on the test set using the average Euclidean distance between the predicted and actual hair follicle coordinates as the evaluation metric. The model with the smallest average Euclidean distance was selected, thus obtaining the well-trained hair follicle prediction model. Finally, the trained hair follicle prediction model was used to perform forward inference on the test hair microscope image, outputting the pixel-level coordinates of the corresponding hair follicles.

[0070] S4. Based on the coordinates of each hair follicle point in the hair microscope image to be tested, perform iterative tracking based on local skeleton constraints to obtain the final hair growth direction of the corresponding hair follicle point.

[0071] In one embodiment, based on the coordinates of each hair follicle point in the hair microscopic image to be tested, iterative tracking is performed based on local skeleton constraints to obtain the final hair growth direction of the corresponding hair follicle point, as follows:

[0072] S41. After performing grayscale and binarization on the hair microscope image to be tested, the global skeleton map is obtained by the Zhang-Suen thinning algorithm. Based on the coordinates of each hair follicle point in the hair microscope image to be tested, the hair shaft skeleton path of the corresponding hair follicle point in the global skeleton map is extracted.

[0073] In one embodiment, the hair shaft skeleton path of each hair follicle point in the global skeleton map is extracted based on the coordinates of each hair follicle point in the hair microscope image to be tested, as follows:

[0074] S411. Within a rectangular neighborhood of a preset size for the coordinates of each hair follicle point in the hair microscope image to be tested, calculate the distance between the hair follicle point and the pixel point of the skeleton in the global skeleton image, and select the pixel point of the skeleton in the global skeleton image that is closest to the hair follicle point as the starting point for tracking the hair growth direction of the corresponding hair follicle point.

[0075] S412. Determine the 8-neighbor connectivity of the starting point of each hair follicle point, that is, check whether there are pixels of the skeleton in the global skeleton graph in the 8-neighborhood of the starting point of each hair follicle point. If they exist and have not been added to the hair shaft skeleton path of the corresponding hair follicle point, add the pixels of the skeleton to the hair shaft skeleton path of the corresponding hair follicle point and use it as a new starting point to repeat step S412 until there are no new pixels of the skeleton. Extract the hair shaft skeleton path of the corresponding hair follicle point.

[0076] Specifically, the image of the hair microscope to be tested is converted to a grayscale image through grayscale preprocessing (grayscale conversion), and then binarized to obtain a binary image. The Zhang-Suen thinning algorithm is used to extract the skeleton from the binary image to obtain the global skeleton map of the image of the hair microscope to be tested, such as... Figure 4 As shown, the trichoscopic image of the hair to be tested is Figure 3 Figure (a) in the diagram can be selected according to actual needs. Figure 4 (a) is the grayscale image of the hair microscope image to be tested, (b) is the binary image of the hair microscope image to be tested, and (c) is the global skeleton diagram of the hair microscope image to be tested.

[0077] Then, based on the coordinates of hair follicle points in the trichoscopic image under test, for example, for the coordinates of a hair follicle point in the trichoscopic image under test, the distance between the hair follicle point and the pixels of the skeleton in the global skeleton map is calculated within a 5×5 (unit: pixels) rectangular neighborhood of the hair follicle point's coordinates, and the pixel of the skeleton in the global skeleton map that is closest to it is selected as the starting point for hair growth tracking of the corresponding hair follicle point. Subsequently, based on this starting point, pixel-level recursive tracking is performed along the skeleton in the global skeleton map using the 8-neighborhood connectivity rule. That is: detect whether there are any pixels with skeletons in the 8 neighboring positions around the current pixel (starting point) (for any pixel, its 8 neighborhoods include pixels in the 8 directions of up, down, left, right, upper left, upper right, lower left, and lower right, with coordinate offsets of [(-1,-1), (-1,0), (-1,1), (0,-1), (0,1), (1,-1), (1,0), (1,1)]). If there are any pixels with skeletons and they have not been visited (the skeleton is a pixel that has not been added to the hair shaft skeleton path of the current hair follicle point before), then add it to the hair shaft skeleton path of the current hair follicle point and use it as a new starting point to continue tracking until there are no new connected pixels (pixels with skeletons in the neighborhood represent connected pixels), thereby extracting the hair shaft skeleton path corresponding to the hair follicle point.

[0078] S42. Treat the hair shaft skeleton path of each hair follicle point as the local skeleton constraint of the corresponding hair follicle point and iteratively track it to obtain the final hair growth direction of the corresponding hair follicle point. The final hair growth direction of a hair follicle point is obtained as follows:

[0079] S421. Construct a rotating window with the hair follicle point as the midpoint of the bottom edge of the rectangle and calculate the grayscale response and coverage response of the rotating window at several rotation angles centered on the coordinates of the hair follicle point. Weight and fuse the grayscale response and coverage response of the corresponding rotation angle to obtain the comprehensive response of the corresponding rotation angle. Take the rotation angle corresponding to the maximum comprehensive response of the rotation angle as the prior direction angle.

[0080] In one embodiment, the grayscale responsivity and coverage responsivity of the rotating window at several rotation angles centered on the coordinates of the hair follicle point are calculated, then:

[0081] Rotation angle is grayscale response at time The formula is as follows:

[0082]

[0083] in, Indicates the rotation angle as The set of pixels covered by the rotating window at that time. for The number of pixels, For the first The grayscale value of each pixel To preset the grayscale threshold, For correction functions;

[0084] Rotation angle is Coverage response time The formula is as follows:

[0085]

[0086] in, For the first The mask value for each pixel.

[0087] Specifically, a rotating window is constructed with the hair follicle point as the midpoint of the base of a rectangle. For example, the window area of ​​the rectangle is 5×15. The rotating window is then rotated around the hair follicle point by several different rotation angles. Figure 5 As shown in Figure (a), it is preferable to cover a 360° rotation range centered on the hair follicle point. For each rotation angle, the grayscale responsivity and coverage responsivity are calculated for the rotation window region. The grayscale responsivity is used to represent the difference between the grayscale value of each pixel within the rotation window and a preset grayscale threshold. For the correction function, such as middle When it is positive, return , When negative, it returns 0. Coverage responsiveness is used to represent the degree of overlap between the rotated window and the hair shaft skeleton path of the corresponding hair follicle point. For the first The mask value of the nth pixel is used to represent the nth pixel. Whether a pixel belongs to hair is determined by a value of 1, which indicates that it belongs to hair, and a value of 0 indicates that it does not belong to hair.

[0088] Rotate by an angle of grayscale response and rotation angle is Coverage response time The rotation angle is obtained by weighted fusion. Overall Response Time The formula is as follows:

[0089] .

[0090] The rotation angle that maximizes the overall response of all rotation angles of the corresponding hair follicle point is taken as the prior direction angle and used for subsequent direction initialization and iterative tracking.

[0091] S422. Perform principal component analysis on the neighborhood of each hair follicle point, and use the prior direction angle to correct the principal component direction obtained by the principal component analysis to obtain the initial growth direction as the starting direction. The neighborhood of the hair follicle point is the rotation window corresponding to the prior direction angle.

[0092] In one embodiment, the initial growth direction is defined by the following formula:

[0093]

[0094] in, The initial growth direction, For symbolic functions, For transpose, The principal direction vector is formed after normalizing the principal component directions. The prior direction and angle.

[0095] Specifically, principal component analysis is performed on the rotation window corresponding to the rotation window with the maximum comprehensive response across all rotation angles of the corresponding hair follicle point to obtain the principal component direction of the hair at the corresponding hair follicle point (principal component analysis can calculate a vector for this rotation window region, which can represent the principal component direction of the hair at the current hair follicle point). The principal component direction is then corrected using the prior direction angle to obtain the initial growth direction. .in, This is a sign function that returns the sign of the corresponding input value: 1 if the input value is positive, -1 if the input value is negative, and 0 if the input value is zero.

[0096] S423. Iteratively track along the corresponding hair shaft skeleton path from the starting direction of the hair follicle point with a preset step size to generate a candidate direction set for the corresponding hair follicle point, and select the candidate direction with the highest consistency with the historical growth direction set as the growth direction for the current iteration step.

[0097] In one embodiment, the candidate direction set for hair follicle points is obtained as follows:

[0098] Centered on the growth direction of the previous iteration, several candidate angles are generated according to different offset angles to form a candidate angle set. For each candidate angle, a corresponding rotation window is generated and principal component analysis is performed to obtain the principal component direction of the hair at the corresponding hair follicle point as the candidate direction. All candidate directions of a hair follicle point constitute the candidate direction set of the corresponding hair follicle point.

[0099] In one embodiment, the candidate direction with the highest consistency with the historical growth direction set is selected from the candidate direction set as the growth direction for the current iteration step. Next candidate angle is Consistency between the candidate direction and the set of historical growth directions The calculation is as follows:

[0100]

[0101] in, The current iteration step Next candidate angle The principal direction vector is obtained by principal component analysis using a rotating window. , The current iteration step The following is the candidate angle set. Number of iterations The growth direction corresponding to the hair follicle point. Number of iterations The weights below, It is an absolute value.

[0102] Specifically, starting from the initial growth direction, iterative tracking is performed along the hair shaft skeleton path corresponding to the hair follicle point at a preset step size. In each iteration, the growth direction obtained in the previous iteration is used as the center, and several candidate angles are generated according to different offset angles to form a candidate angle set, such as... Figure 5As shown in Figure (b), a corresponding rotation window is generated for each candidate angle, and principal component analysis is performed on the rotation window of each candidate angle to obtain the principal component direction of the hair at the corresponding hair follicle point. The growth directions obtained from previous iterations are used as the historical growth direction set, and the candidate direction with the highest consistency with the historical growth direction set is selected from the candidate angle set as the growth direction of the corresponding hair follicle point at the current iteration step. The current iteration step Next candidate angle The principal direction vector is obtained by performing principal component analysis using a rotating window. The principal direction vector is formed by normalizing the principal component directions.

[0103] S424. Update the growth direction of the current iteration step of the corresponding hair follicle point to the starting direction of the corresponding hair follicle point, and return to execute step S423 until the iteration tracking terminates when the iteration step reaches the preset threshold or the end point of the hair shaft skeleton path is reached. Take the point at which the iteration tracking terminates as the termination point, and calculate the direction vector from the hair follicle point to the termination point as the final hair growth direction of the corresponding hair follicle point.

[0104] Specifically, the iteration tracking terminates when the number of iterations reaches a preset threshold or the end point of the hair shaft skeleton path is reached. The coordinates of the hair follicle point at the time of termination are obtained as the termination point. A direction vector (a unit vector from the hair follicle point to the termination point) is calculated using the hair follicle point and the termination point (visualized as a fixed length of 50px). This direction vector is then used as the final hair growth direction. Figure 6 As shown.

[0105] In summary, for each hair follicle point, the coordinates of the hair follicle point are used as the starting point, and the hair microscope image under local skeleton constraints (hair shaft skeleton path) is iteratively tracked to determine the unambiguous hair growth direction starting from the hair follicle point, thereby improving the detection accuracy.

[0106] S5. The coordinates of each hair follicle point and the final hair growth direction in the hair microscope image to be tested are used as the joint detection result of the hair microscope image to be tested. Experiments show that the joint detection result of the hair microscope image (including the coordinates of the hair follicle points and the final hair growth direction) of the proposed method achieves a significant improvement compared with the existing rotating box detection model. In terms of hair follicle point detection accuracy, the mean absolute error (MAE) between the predicted and actual hair follicle point coordinates is used as the metric. The proposed method achieves a prediction error of 4.69 pixels, which is 1.71 pixels higher than the YOLOV11 OBB model and 1.83 pixels higher than the MessiDet model. In terms of direction estimation, the mean cosine similarity between the predicted direction vector from the hair follicle point to the termination point and the actual direction vector from the hair follicle point to the termination point is used as the metric. The proposed method achieves an mean cosine similarity of 0.98 in direction detection, which is 0.11 higher than the YOLOV11 OBB model and 0.14 higher than the MessiDet model. In terms of detection efficiency, the frame rate (FPS) reaches 18.62, resulting in higher detection efficiency.

[0107] This method uses single-point annotation for hair follicle localization, reducing reliance on high-cost annotation methods such as rotated bounding boxes and lowering the annotation cost of training data from the source. Furthermore, it can perform accurate modeling based on the neighborhood information of the hair follicle and estimate the hair growth direction by combining local skeleton constraints, avoiding the uncertainty of forward and reverse directions in traditional rotated bounding box annotation methods. This eliminates directional semantic ambiguity and ensures the reliability and consistency of the hair growth direction.

[0108] Example 2:

[0109] A hair follicle location and growth direction joint detection system includes a processor and a memory. The memory is used to store a computer program. When the computer program is executed by the processor, it implements the hair follicle location and growth direction joint detection method as described in Example 1.

[0110] This system uses single-point annotation for hair follicle localization, reducing reliance on high-cost annotation methods such as rotated bounding boxes and lowering the annotation cost of training data from the source. It can also perform accurate modeling based on the neighborhood information of the hair follicle and estimate the hair growth direction by combining local skeleton constraints, avoiding the uncertainty of forward and reverse directions in traditional rotated bounding box annotation methods. This eliminates directional semantic ambiguity and ensures the reliability and consistency of the hair growth direction.

[0111] It should be understood that the memory and processor are electrically connected directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory stores a computer program that can run on the processor. By running the computer program stored in the memory, the processor implements the method for jointly detecting the location and growth direction of hair follicles in Embodiment 1 of the present invention.

[0112] The memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), and Electrically Erasable Programmable Read-Only Memory (EEPROM). The memory stores programs, and the processor executes these programs after receiving execution instructions.

[0113] The processor may be an integrated circuit chip with data processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. It can implement or execute the methods, steps, and logic block diagrams disclosed in Embodiment 1 of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0114] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0115] The embodiments described above are merely specific and detailed examples of the embodiments described in this application, and should not be construed as limiting the scope of the application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A method for jointly detecting the location and growth direction of hair follicles, characterized in that: Includes the following steps: S1. Obtain several hair follicle images and perform single-point annotation on all hair follicle points in each hair follicle image to construct a dataset; S2. Divide the dataset into a training set and a test set, and perform random data augmentation on the images in the training set; S3. The hair follicle point prediction model is trained using a training set with random data augmentation and tested using a test set to obtain a trained hair follicle point prediction model. The trained hair follicle point prediction model is used to predict the coordinates of all hair follicle points in the hair microscope image to be tested. The hair follicle point prediction model includes an encoder, a decoder, a detection head and a post-processing module connected in sequence. S4. Based on the coordinates of each hair follicle point in the hair microscope image to be tested, perform iterative tracking based on local skeleton constraints to obtain the final hair growth direction of the corresponding hair follicle point. S5. The coordinates of each hair follicle point in the hair microscopic image to be tested and the final hair growth direction are used as the joint detection result of the hair microscopic image to be tested.

2. The method for jointly detecting the location and growth direction of hair follicles as described in claim 1, characterized in that: The encoder includes four sequentially connected encoding layers, and the decoder includes four sequentially connected decoding layers. The first to fourth encoding layers of the encoder are sequentially connected to the fourth to first decoding layers of the decoder, respectively. The encoding layer comprises a first convolutional module, an attention module, and a max pooling layer connected in sequence. The first convolutional module comprises a convolutional layer, a normalization layer, a ReLU activation function, another convolutional layer, a normalization layer, and a ReLU activation function connected in sequence. The attention module comprises a channel attention module and a spatial attention module connected in sequence. The channel attention module comprises a first pooling unit, a convolutional layer, a ReLU activation function, a second pooling unit, a first addition operation, a sigmoid activation function, and a first multiplication operation connected in sequence. Both the first and second pooling units comprise parallel global average pooling layers and global max pooling layers. The first addition operation is used to multiply the second pooling layer... The output features of the global average pooling layer and the output features of the global max pooling layer of the spatial attention module are added together. The first multiplication operation is used to multiply the output features of the sigmoid activation function of the channel attention module and the input features of the first pooling unit to obtain channel enhancement features. The spatial attention module includes a third pooling unit, a convolutional layer, a sigmoid activation function, and a second multiplication operation connected in sequence. The third pooling unit includes a global average pooling layer and a global max pooling layer connected in sequence. The second multiplication operation is used to multiply the output features of the sigmoid activation function of the spatial attention module and the input features of the third pooling unit to obtain spatial enhancement features. The decoding layer includes an upsampling module, a second addition operation, and a second convolution module connected in sequence. The upsampling module includes an upsampling layer, a convolutional layer, a normalization layer, and a ReLU activation function connected in sequence. The second addition operation is used to skip-connect the output features of the upsampling module and the output features of the attention module of the corresponding encoding layer. The second convolution module is the same as the first convolution module. The detection head includes a convolutional layer, a normalization layer, a ReLU activation function, a convolutional layer, a convolutional layer, a normalization layer, a ReLU activation function, a convolutional layer, and a Sigmoid activation function connected in sequence. The post-processing module is used to extract features from the output features of the detection head using a non-maximum suppression algorithm to obtain the coordinates of the hair follicle points in the corresponding hair mirror image.

3. The method for jointly detecting the location and growth direction of hair follicles as described in claim 1, characterized in that: The method involves iteratively tracking each hair follicle point in the microscopic image of the hair under test, based on local skeleton constraints, to obtain the final hair growth direction of the corresponding hair follicle point, as detailed below: S41. After performing grayscale and binarization on the hair mirror image to be tested, the global skeleton map is obtained by Zhang-Suen thinning algorithm. Based on the coordinates of each hair follicle point in the hair mirror image to be tested, the hair shaft skeleton path of the corresponding hair follicle point in the global skeleton map is extracted respectively. S42. Treat the hair shaft skeleton path of each hair follicle point as the local skeleton constraint of the corresponding hair follicle point and iteratively track it to obtain the final hair growth direction of the corresponding hair follicle point. The final hair growth direction of a hair follicle point is obtained as follows: S421. Construct a rotating window with the hair follicle point as the midpoint of the bottom edge of the rectangle and calculate the grayscale response and coverage response of the rotating window at several rotation angles centered on the coordinates of the hair follicle point. Weight the grayscale response and coverage response of the corresponding rotation angle to obtain the comprehensive response of the corresponding rotation angle. Take the rotation angle corresponding to the maximum comprehensive response of the rotation angle as the prior direction angle. S422. Perform principal component analysis on the neighborhood of each hair follicle point, and correct the principal component direction obtained by the principal component analysis using the prior direction angle to obtain the initial growth direction as the starting direction. The neighborhood of the hair follicle point is the rotation window corresponding to the prior direction angle. S423. Iteratively track along the corresponding hair shaft skeleton path from the starting direction of the hair follicle point with a preset step size to generate a candidate direction set for the corresponding hair follicle point, and select the candidate direction with the highest consistency with the historical growth direction set as the growth direction of the current iteration step. S424. Update the growth direction of the current iteration step of the corresponding hair follicle point to the starting direction of the corresponding hair follicle point, and return to execute step S423 until the iteration tracking terminates when the iteration step reaches the preset threshold or the end point of the hair shaft skeleton path is reached. Take the point at which the iteration tracking terminates as the termination point, and calculate the direction vector from the hair follicle point to the termination point as the final hair growth direction of the corresponding hair follicle point.

4. The method for jointly detecting the location and growth direction of hair follicles as described in claim 3, characterized in that: The hair shaft skeleton path of each hair follicle point in the global skeleton map is extracted based on the coordinates of each hair follicle point in the hair microscope image to be tested, as follows: S411. Within a rectangular neighborhood of a preset size for the coordinates of each hair follicle point in the hair microscope image to be tested, calculate the distance between the hair follicle point and the pixel point of the skeleton in the global skeleton image, and select the pixel point of the skeleton in the global skeleton image that is closest to the hair follicle point as the starting point for tracking the hair growth direction of the corresponding hair follicle point. S412. Determine the 8-neighbor connectivity of the starting point of each hair follicle point, that is, check whether there are pixels of the skeleton in the global skeleton graph in the 8-neighborhood of the starting point of each hair follicle point. If they exist and have not been added to the hair shaft skeleton path of the corresponding hair follicle point, add the pixels of the skeleton to the hair shaft skeleton path of the corresponding hair follicle point and use it as a new starting point to repeat step S412 until there are no new pixels of the skeleton. Extract the hair shaft skeleton path of the corresponding hair follicle point.

5. The method for jointly detecting the location and growth direction of hair follicles as described in claim 3, characterized in that: The calculation of the grayscale response and coverage response of the rotating window at several rotation angles centered on the coordinates of the hair follicle point is as follows: Rotation angle is grayscale response at time The formula is as follows: in, Indicates the rotation angle as The set of pixels covered by the rotating window at that time. for The number of pixels, For the first The grayscale value of each pixel To preset the grayscale threshold, For correction functions; Rotation angle is Coverage response time The formula is as follows: in, For the first The mask value for each pixel.

6. The method for jointly detecting the location and growth direction of hair follicles as described in claim 3, characterized in that: The initial growth direction is given by the following formula: in, The initial growth direction, For symbolic functions, For transpose, The principal direction vector is formed after normalizing the principal component directions. The prior direction and angle.

7. The method for jointly detecting the location and growth direction of hair follicles as described in claim 3, characterized in that: The candidate direction set for the hair follicle points is obtained as follows: Centered on the growth direction of the previous iteration, several candidate angles are generated according to different offset angles to form a candidate angle set. For each candidate angle, a corresponding rotation window is generated and principal component analysis is performed to obtain the principal component direction of the hair at the corresponding hair follicle point as the candidate direction. All candidate directions of a hair follicle point constitute the candidate direction set of the corresponding hair follicle point.

8. The method for jointly detecting the location and growth direction of hair follicles as described in claim 7, characterized in that: The process involves selecting the candidate direction from the candidate direction set that has the highest consistency with the historical growth direction set as the growth direction for the current iteration step. Next candidate angle is Consistency between the candidate direction and the set of historical growth directions The calculation is as follows: in, The current iteration step Next candidate angle The principal direction vector is obtained by principal component analysis using a rotating window. , The current iteration step The following is the candidate angle set. Number of iterations The growth direction corresponding to the hair follicle point. Number of iterations The weights below, It is an absolute value.

9. The method for jointly detecting the location and growth direction of hair follicles as described in claim 1, characterized in that: The specific method of testing using a test set involves calculating the average Euclidean distance between the coordinates of the hair follicle points predicted by the trained hair follicle point prediction model and the coordinates of the actual hair follicle points, and taking the trained hair follicle point prediction model that minimizes the average Euclidean distance as the trained hair follicle point prediction model.

10. A system for jointly detecting the location and growth direction of hair follicles, characterized in that: It includes a processor and a memory, the memory being used to store a computer program, which, when executed by the processor, implements the method for joint detection of hair follicle location and growth direction as described in any one of claims 1 to 9.

Images