A hair mirror image unified quantitative analysis method based on semantic triangle representation

Abstract

The application discloses a hair mirror image unified quantitative analysis method based on semantic triangle representation, which comprises the following steps: obtaining a hair mirror image to be quantitatively analyzed, and inputting a trained semantic triangle hair network; the hair mirror image unified quantitative analysis method based on the semantic triangle representation constructs three parameters of each follicle point in the hair mirror image to be quantitatively analyzed, and then realizes quantitative analysis of the hair mirror image, and in the training process, a triangle labeling protocol is designed for the hair mirror image, the follicle opening position, the hair shaft width and the growth direction are uniformly coded into a triangle geometric structure, and the inherent 0° / 180° direction ambiguity of the rotating rectangular frame is fundamentally eliminated from the representation level; in the semantic triangle hair network, the hair triangle fusion module is used to perform multi-level feature aggregation and enhancement on the slender and curved hair structure by using a deformable convolution and a CBAM module.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and medical image analysis technology, specifically relating to a unified quantitative analysis method for trichoscopic images based on semantic triangle representation. Background Technology

[0002] Trichoscopy, a non-invasive and repeatable dermoscopy technique, is widely used for the quantitative assessment of alopecia. In routine diagnosis and longitudinal follow-up, trichoscopy provides high-resolution scalp images, supporting the assessment of hair density, hair shaft diameter distribution and miniaturization, follicular unit characteristics, and treatment-related changes. However, these quantitative assessments still primarily rely on manual hair counting and measurement, combined with subjective clinical judgment. This workflow is time-consuming and labor-intensive, exhibiting significant between- and within-observer variability, thus hindering its scaling to large-scale follow-up cohorts or standardized efficacy comparisons. With the rapid increase in the number of patients and imaging data, the clinical need for efficient, accurate, and auditable automated quantitative analysis methods is increasingly urgent.

[0003] In existing intelligent analysis methods for trichoscopy images, deep learning techniques are often used to detect or segment hair-related structures and further estimate clinically relevant indicators (such as hair shaft direction, thickness, and density). Existing representation and inference paradigms can be broadly categorized into three types: target detection methods based on axis-aligned or rotated bounding boxes, geometric regression methods based on line segments or keypoints, and pixel-level modeling methods based on semantic segmentation or instance segmentation. Despite increasing automation, precise quantitative analysis in trichoscopy still faces several key challenges. First, hair shafts are thin and often curved, with occlusion and crossing phenomena being common, resulting in geometric mismatch between simple bounding boxes and the actual structure. Axially aligned bounding boxes cannot stably correspond to hair follicle openings and cannot adequately represent the clinically relevant geometric information of the proximal hair shaft. Second, factors such as specular highlights, scales, hair coverage, imaging artifacts, and extremely fine vellus hair can blur boundaries or shorten visible hair segments, leading to significant fluctuations when estimating direction and width solely from visible fragments. Third, many methods decompose follicle opening localization and hair shaft attribute estimation into independent subtasks or multi-stage pipelines, leading to the accumulation of errors at each stage. Furthermore, they lack unified, interpretable, and visualization-friendly geometric constraints to ensure semantic and geometric consistency. Fourth, from a visualization perspective, rotated rectangles easily obscure the follicle location and create visual clutter, failing to intuitively convey the true direction of hair growth and creating a gap between the method and clinicians' intuitive perception of slender, direction-oriented structures. Fifth, existing rotating target detection methods typically mitigate the discontinuity in angle regression at the 0° and 180° boundaries by modifying angle encoding or loss functions, but the underlying rotated rectangle representation inherently possesses 180° rotational symmetry. For hair analysis, the follicle opening naturally defines the starting point and growth direction, requiring a geometric primitive with inherent directionality, rather than a centrally symmetric rectangle. Summary of the Invention

[0004] The purpose of this invention is to address the problems raised in the background art by proposing a unified quantitative analysis method for hair microscopy images based on semantic triangle representation.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] This invention proposes a unified quantitative analysis method for trigonometric microscopy images based on semantic triangle representation, comprising:

[0007] Acquire hair microscopic images to be quantified and input them into the trained semantic triangle hair network;

[0008] The semantic triangle hair network includes an encoder and a hair triangle fusion module connected in sequence, as well as two parallel branches: a first branch, a second branch, and a third branch. The first branch is a heatmap head, the second branch is a parameter head, and the third branch includes a context module, a direction-aware triangular encoding module, and a parameter adapter connected in sequence.

[0009] The output of the heatmap head is used to locate the output of the parameter head. The output of the parameter head is added to the output of the parameter adapter to obtain three refined parameters for each hair follicle point in the hair microscopic image to be quantified and analyzed. These parameters are used as the output of the semantic triangle hair network, and the three parameters are the orientation angle, radius and width, respectively.

[0010] In this process, a circle is drawn with the radius of each hair follicle as the center point. The circle intersects with the two sides of the hair shaft of the corresponding hair, and the two intersection points and the center point form a triangle. The line connecting the two intersection points forms the base of the triangle, and the length of the base is the width. The direction from the center point to the midpoint of the base is the direction angle.

[0011] The three refined parameters corresponding to each hair follicle point in the hair microscopic image to be quantified are used as the results of the quantification analysis.

[0012] Preferably, the encoder uses a ConvNeXt-Small network, and the first, second, third and fourth stages of the ConvNeXt-Small network output a first feature map, a second feature map, a third feature map and a fourth feature map in sequence.

[0013] Preferably, the hair triangle fusion module includes three branches, each branch including a deformable convolution, a CBAM module and a deformable convolution connected in sequence, and the three branches are the third branch, the fourth branch and the fifth branch respectively. The fourth feature map is upsampled and added to the third feature map, and then used as the input of the fifth branch. The third feature map is upsampled and added to the second feature map, and then used as the input of the fourth branch. The second feature map is upsampled and added to the first feature map, and then used as the input of the third branch. The outputs of the third branch, the fourth branch and the fifth branch are weighted and summed to obtain the fused features.

[0014] Preferably, the heatmap header includes three cascaded convolutional layers, a sigmoid map, and a bilinear upsampling operation connected in sequence;

[0015] The fusion features are used as input to the heatmap header to obtain a heatmap and an offset map. Each hair follicle point in the heatmap is represented as the peak value of a two-dimensional Gaussian distribution, and each point in the offset map is the offset of each hair follicle point in the heatmap.

[0016] The coordinates of each hair follicle point in the heatmap are corrected using the offset map to obtain the corrected heatmap.

[0017] Preferably, the parameter head includes three convolutional layers connected in sequence, and takes the fused features as the input of the parameter head, and takes the corrected heatmap as the output positioning of the parameter head. The parameter head outputs three coarse parameters for each hair follicle point in the corrected heatmap, namely the coarse orientation angle, radius and width.

[0018] Preferably, in the context module, the fused feature map is passed through three dilated convolutional modules with different dilation rates to obtain near-ring features, middle-ring features, and far-ring features. The corrected heatmap is passed through a convolutional layer, a first activation layer, a convolutional layer, and a softmax layer to obtain a first weight, a second weight, and a third weight. The near-ring features, middle-ring features, and far-ring features are multiplied by the first weight, the second weight, and the third weight in turn to obtain three multiplication results. After being added together, the results are passed through a convolutional layer and a batch normalization layer to obtain a fifth feature map. The fifth feature map is concatenated with the fused feature map and then passed through a convolutional layer and a second activation layer in turn. The fifth feature map is multiplied by the output of the second activation layer and then added to the fused feature map to obtain the output of the context module.

[0019] Each dilated convolutional module includes a dilated convolutional layer, a batch normalization layer, and a third activation layer connected in sequence.

[0020] Preferably, in the orientation-aware triangular encoding module, the output of the context module passes through a horizontal convolution branch, a vertical convolution branch, and an orientation field module to obtain a horizontal response, a vertical response, and an orientation field. The orientation field is then passed through a convolution module to obtain orientation fields in the horizontal and vertical directions. The horizontal and vertical orientation fields are then multiplied one-to-one with the horizontal and vertical responses to obtain two multiplication results. After being added together, the results pass through a convolution module, a convolutional layer, and a batch normalization layer to obtain a sixth feature map. The sixth feature map is then added to the output of the context module to obtain the output of the orientation-aware triangular encoding module.

[0021] Each of the convolutional branches includes a convolutional layer, a batch normalization layer, and a fourth activation layer connected in sequence, and the orientation field module includes a convolutional layer, a batch normalization layer, a fifth activation layer, and a convolutional layer connected in sequence.

[0022] The convolutional module includes a convolutional layer, a sixth activation layer, another convolutional layer, and a softmax layer connected in sequence.

[0023] Preferably, in the parameter adapter, local feature blocks of each hair follicle point are extracted from the output of the orientation-aware triangular encoding module according to a preset size, with each hair follicle point as the center. According to the positional relationship of each hair follicle in the feature map output by the orientation-aware triangular encoding module, the local feature blocks of all hair follicle points are stitched together into a ROI feature map. The ROI feature map is then passed through a convolutional layer, a seventh activation layer, a global average pooling layer, and a multilayer perceptron in sequence to obtain the residuals of the three parameters of each hair follicle point, namely the residuals of orientation angle, radius, and width. The residuals of orientation angle, radius, and width of each hair follicle point are added one by one to the coarse orientation angle, radius, and width to obtain the refined orientation angle, radius, and width of each hair follicle point.

[0024] Preferably, a Cartesian coordinate system is established using one of the points in the input image input to the semantic triangle hair network as a reference;

[0025] During training, after obtaining the output of the semantic triangle hair network, the predicted coordinates of the two points on the base of each triangle and the predicted coordinates of the midpoint of the base plate are calculated using the position of each hair follicle point and the refined direction angle, radius and width.

[0026] The total loss function of the semantic triangle hair network The calculation formula is as follows:

[0027] ;

[0028] in, For vertex localization loss of the heatmap header, The regression loss is for the parameter header. The geometric consistency loss is the sum of the two points on the bottom edge. The supervised loss is for the direction-aware triangulation module and the parameter adapter.

[0029] Preferably, the unified quantitative analysis method for hair microscopy images based on semantic triangle representation further includes refining the width based on the output of the semantic triangle hair network, calculating the true width of the hair by combining the magnification of the hair microscope and the imaging calibration relationship, distinguishing the hair as terminal hair or vellus hair based on the true width, and statistically analyzing the ratio of terminal hair to vellus hair in a preset area. The evaluation results of the degree of hair follicle miniaturization and hair loss grade are obtained through the ratio of terminal hair to vellus hair, and the evaluation results are also used as the results of quantitative analysis.

[0030] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0031] This unified quantitative analysis method for trichoscopic images based on semantic triangle representation constructs a semantic triangle hair network to refine three parameters for each hair follicle point in the trichoscopic image to be analyzed, thereby achieving quantitative analysis of the trichoscopic image. Furthermore, during the training process, a triangle annotation protocol is designed for the trichoscopic image to uniformly encode the hair follicle opening position, hair shaft width, and growth direction into a triangular geometric structure, fundamentally eliminating the inherent 0° / 180° directional ambiguity of the rotating rectangle at the representation level.

[0032] In this semantic triangular hair network, the hair triangle fusion module utilizes deformable convolution and CBAM modules to perform multi-level feature aggregation enhancement on the fine and curved hair structure. It also introduces a context module, a direction-aware triangular encoding module, and a parameter adapter to guide and enhance the hair geometric parameters and impose consistency constraints under interference conditions such as occlusion and specular highlights, which significantly improves the robustness of radius, direction, and width estimation.

[0033] This semantic triangle hair network is an end-to-end framework that simultaneously outputs three refined parameters for each hair follicle point in the hair microscopic image, avoiding multi-stage error accumulation and reducing post-processing dependence. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the semantic triangle hair network structure in the unified quantitative analysis method for hair microscopy images based on semantic triangle representation in this invention.

[0035] Figure 2 This is a schematic diagram of the hair triangle fusion module in this invention;

[0036] Figure 3 This is a schematic diagram of the context module in this invention;

[0037] Figure 4 This is a schematic diagram of the direction-sensing triangulation encoding module in this invention;

[0038] Figure 5 This is a schematic diagram of the parameter adapter in this invention. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0041] In one embodiment, such as Figures 1-5 As shown, a unified quantitative analysis method for trichoscopic images based on semantic triangle representation is provided, including:

[0042] Step 1: Obtain the hair microscopic image to be quantified and input it into the trained semantic triangle hair network;

[0043] Among them, the hair microscopic images to be quantified contain multiple hair follicle points (hair follicle points are the opening locations of hair follicles. Under hair microscopic imaging, the hair follicle opening is the most directly observable structure, which is the location where hair exits from the skin surface).

[0044] Step 2: The semantic triangle hair network includes an encoder and a hair triangle fusion module connected in sequence, as well as two parallel branches: a first branch, a second branch, and a third branch. The first branch is a heatmap head, the second branch is a parameter head, and the third branch includes a context module, a direction-aware triangulation module, and a parameter adapter connected in sequence. The output of the heatmap head is used to locate the output of the parameter head. The output of the parameter head is added to the output of the parameter adapter to obtain three refined parameters for each hair follicle point in the hair microscopic image to be quantified and analyzed. These three parameters are used as the output of the semantic triangle hair network, and they are the orientation angle, radius, and width, respectively.

[0045] In this process, a circle is drawn with the radius of each hair follicle as the center point. The circle intersects with the two sides of the hair shaft of the corresponding hair, and the two intersection points and the center point form a triangle (since the vertex and the base cannot be interchanged under a 180° rotation, this geometric primitive eliminates directional ambiguity at the representation level). The line connecting the two intersection points forms the base of the triangle, the length of which is the width, and the direction from the center point to the midpoint of the base is the direction angle.

[0046] Step 2.1: The encoder uses a ConvNeXt-Small network, and the first stage (Stage 1), the second stage (Stage 2), the third stage (Stage 3), and the fourth stage (Stage 4) of the ConvNeXt-Small network output the first feature map, the second feature map, the third feature map, and the fourth feature map in sequence.

[0047] It should be noted that the trichoscopic images to be quantified were scaled to 1024×1024 and then extracted by the ConvNeXt-Small network to obtain the first feature map (256×256, 96 channels), the second feature map (128×128, 192 channels), the third feature map (64×64, 384 channels), and the fourth feature map (32×32, 768 channels). The first feature map has the highest resolution, and the fourth feature map has the lowest resolution. (The process of obtaining the four feature maps is the extraction process of the existing ConvNeXt-Small network and is considered existing technology, so it will not be described in detail.)

[0048] Step 2.2, as follows Figure 2 As shown, the hair triangle fusion module includes three branches. Each branch includes a deformable convolution (3×3 kernel, stride 1, padding 1, dilation 1, no bias), a CBAM module (an existing module, not described in detail), and a deformable convolution (3×3 kernel, stride 1, padding 1, dilation 1, no bias) connected in sequence. The three branches are the third, fourth, and fifth branches, respectively. The fourth feature map is upsampled (i.e., bilinear interpolation) and added to the third feature map, which is then used as the input to the fifth branch. The third feature map is upsampled (i.e., bilinear interpolation) and added to the second feature map, which is then used as the input to the fourth branch. The second feature map is upsampled (i.e., bilinear interpolation) and added to the first feature map, which is then used as the input to the third branch. The outputs of the third, fourth, and fifth branches are weighted and summed (using learnable weighting coefficients) to obtain the fused feature (256×256, 256 channels).

[0049] It should be noted that each branch of the hair triangle fusion module first learns the adaptive sampling offset of instances through deformable convolutional blocks, enabling the convolutional kernels to sample along the hair trajectory to obtain more stable features; secondly, it performs channel attention and spatial attention through the CBAM module to highlight slender targets; finally, it further reduces the deviation of rigid grid sampling on irregular hair contours through deformable convolutional networks; finally, it performs weighted fusion, with high-resolution branches receiving larger weights to ensure the localization accuracy of hair follicle vertices, and low-resolution branches providing global context.

[0050] Step 2.3: The heatmap header includes three cascaded convolutional layers (the first two convolutional layers use 3×3 convolution with padding of 1, and the last convolutional layer uses 1×1 convolution), a Sigmoid mapping (mapped to [0,1]), and a bilinear upsampling operation.

[0051] Using the fusion features as input to the heatmap header, we obtain a heatmap (512×512, 1 channel) and an offset map (512×512, 2 channels). Each hair follicle point in the heatmap is represented as the peak value of a two-dimensional Gaussian distribution, and each point in the offset map is the offset (sub-pixel offset) of each hair follicle point in the heatmap.

[0052] The coordinates of each hair follicle point in the heatmap are corrected using the offset map (the coordinates are corrected by using the offset in the offset map and the corresponding hair follicle point), resulting in a corrected heatmap (512×512, with 1 channel, and the peak point in the corrected heatmap is the location of the hair follicle point).

[0053] Step 2.4: The parameter head consists of three convolutional layers connected in sequence (all three convolutional layers use 1×1 convolution), and the fused features are used as the input of the parameter head. The corrected heatmap is used as the output of the parameter head for localization. The parameter head outputs three coarse parameters for each hair follicle point in the corrected heatmap, namely the coarse orientation angle, radius and width.

[0054] Step 2.5, as follows Figure 3 As shown, in the context module, the fused feature map is passed through three dilated convolutional modules with different dilation rates to obtain near-loop features (the dilation rate of the dilated convolutional layer in the corresponding dilated convolutional module is 1), middle-loop features (the dilation rate of the dilated convolutional layer in the corresponding dilated convolutional module is 2), and far-loop features (the dilation rate of the dilated convolutional layer in the corresponding dilated convolutional module is 3). The corrected heatmap is then passed through a convolutional layer, a first activation layer (ReLU activation function), a convolutional layer, and a softmax layer in sequence, outputting weights with a size of 256×256 and 3 channels. These weights are then split to obtain a first weight W1 and a second weight W1, each with a size of 256×256 and 1 channel. The weights W2 and W3 are used to multiply the near-loop, middle-loop, and far-loop features one by one with the first, second, and third weights, respectively, to obtain three multiplication results. These results are then added together and passed through a convolutional layer (1×1 convolution with a stride of 1) and a batch normalization layer (BN) to obtain the fifth feature map. The fifth feature map is then concatenated with the fused feature map and passed through a convolutional layer (1×1 convolution with a stride of 1) and a second activation layer (Sigmoid activation function). The fifth feature map is multiplied by the output of the second activation layer and then added to the fused feature map to obtain the output of the context module (size 256×256, number of channels 256).

[0055] Each dilated convolutional module includes a dilated convolutional layer, a batch normalization (BN) layer, and a third activation layer (ReLU activation function) connected in sequence.

[0056] It should be noted that the context module enhances the structured contextual clues for geometric estimation while preserving semantic information.

[0057] Step 2.6, as follows Figure 4 As shown, in the orientation-aware triangular encoding module, the output of the context module passes through the horizontal convolution branch (where the convolutional layer uses 1×3 convolution with a stride of 1 and padding of (0,1)), the vertical convolution branch (where the convolutional layer uses 3×1 convolution with a stride of 1 and padding of (1,0)), and the orientation field module to obtain the horizontal response, vertical response, and orientation field. The orientation field is then passed through the convolution module to obtain the orientation fields in the horizontal and vertical directions, respectively. The orientation fields in the horizontal and vertical directions are then multiplied one-to-one with the horizontal and vertical responses to obtain two multiplication results. After being added together, the results are passed through the convolution module, convolutional layer, and batch normalization layer to obtain the sixth feature map (size 256×256, number of channels 256). The sixth feature map is then added to the output of the context module to obtain the output of the orientation-aware triangular encoding module.

[0058] Each convolutional branch includes a convolutional layer, a batch normalization layer, and a fourth activation layer (ReLU activation function) connected in sequence. The orientation field module includes a convolutional layer (3×3 convolution, stride 1, padding 1), a batch normalization layer, a fifth activation layer (ReLU activation function), and a convolutional layer (1×1 convolution, stride 1) connected in sequence.

[0059] The convolution module consists of a convolutional layer (1×1 convolution), a sixth activation layer (ReLU activation function), another convolutional layer (1×1 convolution), and a softmax layer connected in sequence.

[0060] It should be noted that the direction-aware triangulation module maintains semantic structure while introducing direction sensitivity.

[0061] Step 2.7, as follows Figure 5 As shown, in the parameter adapter, local feature blocks of each hair follicle point are extracted from the output of the orientation-aware triangular encoding module according to a preset size, with each hair follicle point as the center. According to the positional relationship of each hair follicle in the feature map output by the orientation-aware triangular encoding module, the local feature blocks of all hair follicle points are stitched together into a ROI feature map (size 7×7, number of channels 256). The ROI feature map is then passed through a convolutional layer (1×1 convolution), a seventh activation layer (ReLU activation function), a global average pooling layer, and a multilayer perceptron (where the multilayer perceptron includes a linear layer, a ReLU activation function + Dropout layer, a linear layer, and a ReLU activation function connected in sequence) to obtain the residuals of the three parameters of each hair follicle point, namely the residuals of the orientation angle, radius, and width. The residuals of the orientation angle, radius, and width of each hair follicle point are added one by one to the coarse orientation angle, radius, and width to obtain the refined orientation angle, radius, and width of each hair follicle point.

[0062] It should be noted that the sequential connections mentioned above all refer to sequential connections from data input to output.

[0063] Step 3: The refined three parameters corresponding to each hair follicle point in the trichoscopic image to be quantified are used as the results of the quantification analysis. Simultaneously, the unified quantification analysis method for trichoscopic images based on semantic triangle representation also includes the refined width based on the output of the semantic triangle hair network, and the calculation of the true width of the hair by combining the magnification of the trichoscopic image and the imaging calibration relationship. Based on the true width, the hair is distinguished as terminal hair or vellus hair, and the ratio of terminal hair to vellus hair within a preset area is statistically analyzed. The evaluation results of the degree of hair follicle miniaturization and hair loss grade are obtained through the ratio of terminal hair to vellus hair, and these evaluation results are also used as the results of the quantification analysis.

[0064] In another embodiment, the training process of the semantic triangle hair network is as follows:

[0065] The dataset contains 387 labeled long-hair trichoscopic images (images of the top of the scalp, the occipital region, and 1 cm behind the anterior hairline were acquired using a trichoscopic microscope at 50x magnification, with an image resolution of 2560×1920, covering various hair loss-related diseases such as seborrheic alopecia and alopecia areata. This embodiment includes 387 long-hair trichoscopic images from 129 subjects), and is divided into training, validation, and test sets in a 7:1.5:1.5 ratio. Each long-hair trichoscopic image serves as a training sample. When labeling each trichoscopic image in the dataset, a rectangular coordinate system is first established using one point in each trichoscopic image as the origin (e.g., the upper left corner of the trichoscopic image is the origin, the rightward direction is the X-axis square image, and the upward direction is the positive Y-axis). The hair follicle opening is labeled as the hair follicle point. (Under this trigonometric imaging, the hair follicle opening is a directly observable structure; this location is taken as the vertex of the triangle); Draw a circle with a radius of ... A circle, in which The preset value (under a fixed magnification of the hair microscope), Determined based on typical ranges of resolution and hair width. When acquiring data at different magnifications or resolutions, (Alignment can be done in physical units first, then converted to pixel values ​​to maintain protocol consistency). The circle intersects with the two sides of the corresponding hair shaft at two points, which are labeled as follows: and ;Will , and Construct a triangle for the three points of the triangle, and The vertex of the triangle, and Let these be the other two points of the triangle, and label the length of the triangle's base. =| - | represents the width of the hair (i.e., the width of the hair shaft), and marks the vertex to the midpoint of the bottom edge. direction (That is, the direction of hair growth);

[0066] During training, after obtaining the output of the semantic triangle hair network, the predicted coordinates of the two vertices of the base of each triangle and the predicted coordinates of the midpoint of the base are calculated using the position of each hair follicle point and the refined orientation angle, radius, and width. The corresponding calculation formulas are as follows:

[0067] ;

[0068] ;

[0069] ;

[0070] in, The predicted coordinates of the midpoint of the bottom edge. and These are the predicted coordinates of the two points on the bottom edge. This is a transpose.

[0071] The parameters of the semantic triangle hair network are updated using backpropagation with the total loss function until the semantic triangle hair network converges, resulting in a trained semantic triangle hair network. The parameter settings involved in the training process are as follows: The PyTorch framework is used for implementation, and training is performed on an NVIDIA RTX A6000 GPU. End-to-end training for 300 epochs is conducted using the AdamW optimizer, with an initial learning rate of 1e-4, weight decay of 1e-4, and a cosine annealing learning rate scheduling strategy. The first 20 epochs are used as a warm-up phase, with a batch size of 16.

[0072] Among them, the total loss function The calculation formula is as follows:

[0073] ;

[0074] in,

[0075] ;

[0076] ;

[0077] ;

[0078] ;

[0079] ;

[0080] ;

[0081] ;

[0082] ;

[0083] in, For vertex localization loss of the heatmap header, The regression loss is the parameter header. The geometric consistency loss is the sum of the two points on the bottom edge. The supervised loss is for the direction-aware triangulation encoding module and the parameter adapter. , , , , , , and These are all weight parameters, and are set sequentially to 2.0, 1.0, 1.0, 1.0, 1.0, 2.0, 1.0, and 1.0. For the focus of loss, For offset loss, This represents the number of positive samples in the training samples, where the location of each hair follicle point in the training samples is a positive sample, and the remaining areas are negative samples. For heat map at location The predicted probability at that location. The negative sample weight adjustment parameter is set to 5 in this embodiment. The focusing parameter is set to 5 in this embodiment. and These are the horizontal and vertical coordinates, respectively. For heat map at location The true label value at a given location is generated from a two-dimensional Gaussian distribution based on the actual hair follicle location. Indicates the center of the hair follicle point. Indicates the background or Gaussian decay region. This is the set of all positive sample points (the coordinates of the actual hair follicle points). For the offset map at position The predicted offset at that location. For the offset map at position The actual offset at that location. For the loss of direction angle, For the radius loss, For the loss of width, For point Predicted coordinates For point Predicted coordinates For point The actual coordinates, For point The actual coordinates, For the loss of the direction-aware triangulation module, For the loss of the parameter adapter, For the direction-aware triangulation module in position The predicted direction unit vector, For in position The unit vector in the true direction. For the refined trigonometric adapter, the first The predicted direction unit vector for each hair follicle point For the first The true direction unit vector of each hair follicle point For the refined trigonometric adapter, the first Predicted radius of each hair follicle point For the first The true radius of each hair follicle point For the refined trigonometric adapter, the first Predicted width of each hair follicle point For the first The actual width of each hair follicle point.

[0084] This unified quantitative analysis method for hair microscopy images based on semantic triangle representation constructs a semantic triangle hair network to refine three parameters for each hair follicle point in the hair microscopy image to be analyzed, thereby achieving quantitative analysis of the hair microscopy image. During training, a triangle annotation protocol is designed for the hair microscopy image, uniformly encoding the hair follicle opening position, hair shaft width, and growth direction into a triangular geometric structure, fundamentally eliminating the inherent 0° / 180° directional ambiguity of the rotating rectangle at the representation level. In this semantic triangle hair network, a hair triangle fusion module utilizes deformable convolution and CBAM modules to perform multi-level feature aggregation on the fine and curved hair structure. This method enhances the robustness of radius, direction, and width estimations by introducing a context module, a direction-aware triangular encoding module, and a parameter adapter. It guides and constrains the geometric parameters of hair under interference conditions such as occlusion and specular highlights, significantly improving these parameters. The semantic triangular hair network is an end-to-end framework that simultaneously outputs three refined parameters for each hair follicle point in the trigonometric image, avoiding multi-stage error accumulation and reducing post-processing dependence. This method provides a unified output that directly supports the extraction of various clinically relevant indicators (such as hair density, terminal hair / vellus hair ratio, and growth direction consistency) without requiring additional complex post-processing. Furthermore, triangular visualization is clearer and more clinically interpretable than rectangular bounding boxes.

[0085] In another embodiment, the present invention also provides a unified quantitative analysis device for trichoscopic images based on semantic triangle representation, including a processor and a memory storing a plurality of computer instructions, wherein the computer instructions, when executed by the processor, implement the steps of the unified quantitative analysis method for trichoscopic images based on semantic triangle representation.

[0086] For specific limitations on the unified quantitative analysis device for trichoscopic images based on semantic triangle representation, please refer to the limitations on the unified quantitative analysis method for trichoscopic images based on semantic triangle representation mentioned above, which will not be repeated here.

[0087] The memory and processor are electrically connected directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory stores a computer program that can run on the processor, and the processor implements the method of the present invention by running the computer program stored in the memory.

[0088] The memory may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory is used to store programs, which the processor executes upon receiving execution instructions.

[0089] The processor may be an integrated circuit chip with data processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0090] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0091] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A unified quantitative analysis method for trigonometric microscopy images based on semantic triangle representation, characterized in that: The unified quantitative analysis method for hair microscopy images based on semantic triangle representation includes: Acquire hair microscopic images to be quantified and input them into the trained semantic triangle hair network; The semantic triangle hair network includes an encoder and a hair triangle fusion module connected in sequence, as well as two parallel branches: a first branch, a second branch, and a third branch. The first branch is a heatmap head, the second branch is a parameter head, and the third branch includes a context module, a direction-aware triangular encoding module, and a parameter adapter connected in sequence. The output of the heatmap head is used to locate the output of the parameter head. The output of the parameter head is added to the output of the parameter adapter to obtain three refined parameters for each hair follicle point in the hair microscopic image to be quantified and analyzed. These parameters are used as the output of the semantic triangle hair network, and the three parameters are the orientation angle, radius and width, respectively. In this process, a circle is drawn with the radius of each hair follicle as the center point. The circle intersects with the two sides of the hair shaft of the corresponding hair, and the two intersection points and the center point form a triangle. The line connecting the two intersection points forms the base of the triangle, and the length of the base is the width. The direction from the center point to the midpoint of the base is the direction angle. The three refined parameters corresponding to each hair follicle point in the hair microscopic image to be quantified are used as the results of the quantification analysis.

2. The unified quantitative analysis method for trigonometric microscopy images based on semantic triangle representation as described in claim 1, characterized in that: The encoder uses a ConvNeXt-Small network, and the first, second, third and fourth stages of the ConvNeXt-Small network output the first feature map, the second feature map, the third feature map and the fourth feature map respectively.

3. The unified quantitative analysis method for trigonometric microscopy images based on semantic triangle representation as described in claim 2, characterized in that: The hair triangle fusion module includes three branches, each branch including a deformable convolution, a CBAM module and a deformable convolution connected in sequence, and the three branches are the third branch, the fourth branch and the fifth branch respectively. The fourth feature map is upsampled and added to the third feature map, and then used as the input of the fifth branch. The third feature map is upsampled and added to the second feature map, and then used as the input of the fourth branch. The second feature map is upsampled and added to the first feature map, and then used as the input of the third branch. The outputs of the third branch, the fourth branch and the fifth branch are weighted and summed to obtain the fused features.

4. The unified quantitative analysis method for trigonometric microscopy images based on semantic triangle representation as described in claim 3, characterized in that: The heatmap header includes three cascaded convolutional layers, a sigmoid map, and a bilinear upsampling operation connected in sequence. The fusion features are used as input to the heatmap header to obtain a heatmap and an offset map. Each hair follicle point in the heatmap is represented as the peak value of a two-dimensional Gaussian distribution, and each point in the offset map is the offset of each hair follicle point in the heatmap. The coordinates of each hair follicle point in the heatmap are corrected using the offset map to obtain the corrected heatmap.

5. The unified quantitative analysis method for trigonometric microscopy images based on semantic triangle representation as described in claim 4, characterized in that: The parameter head includes three convolutional layers connected in sequence, and takes the fused features as the input of the parameter head and the corrected heatmap as the output of the parameter head. The parameter head outputs three coarse parameters for each hair follicle point in the corrected heatmap, namely the coarse orientation angle, radius and width.

6. The unified quantitative analysis method for trichoscopic images based on semantic triangle representation as described in claim 4, characterized in that: In the context module, the fused feature map is passed through three dilated convolutional modules with different dilation rates to obtain near-ring features, middle-ring features, and far-ring features. The corrected heatmap is then passed through a convolutional layer, a first activation layer, another convolutional layer, and a softmax layer to obtain a first weight, a second weight, and a third weight. The near-ring features, middle-ring features, and far-ring features are then multiplied one by one with the first weight, the second weight, and the third weight, respectively, to obtain three multiplication results. After being added together, the results are passed through a convolutional layer and a batch normalization layer to obtain a fifth feature map. The fifth feature map is then concatenated with the fused feature map and passed through a convolutional layer and a second activation layer. The fifth feature map is then multiplied with the output of the second activation layer and added to the fused feature map to obtain the output of the context module. Each dilated convolutional module includes a dilated convolutional layer, a batch normalization layer, and a third activation layer connected in sequence.

7. The unified quantitative analysis method for trigonometric microscopy images based on semantic triangle representation as described in claim 1, characterized in that: In the orientation-aware triangular encoding module, the output of the context module passes through the horizontal convolution branch, the vertical convolution branch, and the orientation field module to obtain the horizontal response, the vertical response, and the orientation field. The orientation field is then passed through the convolution module to obtain the orientation fields in the horizontal and vertical directions. The orientation fields in the horizontal and vertical directions are then multiplied one-to-one with the horizontal and vertical responses to obtain two multiplication results. After being added together, the results pass through the convolution module, the convolutional layer, and the batch normalization layer to obtain the sixth feature map. The sixth feature map is then added to the output of the context module to obtain the output of the orientation-aware triangular encoding module. Each of the convolutional branches includes a convolutional layer, a batch normalization layer, and a fourth activation layer connected in sequence, and the orientation field module includes a convolutional layer, a batch normalization layer, a fifth activation layer, and a convolutional layer connected in sequence. The convolutional module includes a convolutional layer, a sixth activation layer, another convolutional layer, and a softmax layer connected in sequence.

8. The unified quantitative analysis method for trigonometric microscopy images based on semantic triangle representation as described in claim 5, characterized in that: In the parameter adapter, local feature blocks of each hair follicle point are extracted from the output of the orientation-aware triangular encoding module according to a preset size, with each hair follicle point as the center. According to the positional relationship of each hair follicle in the feature map output by the orientation-aware triangular encoding module, the local feature blocks of all hair follicle points are stitched together into a ROI feature map. The ROI feature map is then passed through a convolutional layer, a seventh activation layer, a global average pooling layer, and a multilayer perceptron in sequence to obtain the residuals of the three parameters of each hair follicle point, namely the residuals of orientation angle, radius, and width. The residuals of orientation angle, radius, and width of each hair follicle point are added one by one to the coarse orientation angle, radius, and width to obtain the refined orientation angle, radius, and width of each hair follicle point.

9. The unified quantitative analysis method for trichoscopic images based on semantic triangle representation as described in claim 1, characterized in that: A Cartesian coordinate system is established based on one of the points in the input image into the semantic triangle hair network; During training, after obtaining the output of the semantic triangle hair network, the predicted coordinates of the two points on the base of each triangle and the predicted coordinates of the midpoint of the base plate are calculated using the position of each hair follicle point and the refined direction angle, radius and width. The total loss function of the semantic triangle hair network The calculation formula is as follows: ； in, For vertex localization loss of the heatmap header, The regression loss is for the parameter header. The geometric consistency loss is the sum of the two points on the bottom edge. The supervised loss is for the direction-aware triangulation module and the parameter adapter.

10. The unified quantitative analysis method for trigonometric microscopy images based on semantic triangle representation as described in claim 1, characterized in that: The unified quantitative analysis method for hair microscopy images based on semantic triangle representation also includes refining the width based on the output of the semantic triangle hair network, and calculating the true width of the hair by combining the magnification of the hair microscope and the imaging calibration relationship. Based on the true width, the hair is distinguished as terminal hair or vellus hair, and the ratio of terminal hair to vellus hair in a preset area is statistically analyzed. The evaluation results of the degree of hair follicle miniaturization and hair loss grade are obtained through the ratio of terminal hair to vellus hair, and the evaluation results are also used as the results of quantitative analysis.

Images