Facial acne lesion detection analysis method and system
By integrating RT-DETR v2, SAM 2, and YOLOv12 algorithms, an end-to-end facial acne lesion detection system was constructed, solving the problems of detection accuracy, speed, and privacy protection in existing technologies, and achieving efficient and accurate facial acne detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHUHAI YIMIAN INTELLIGENT TECHNOLOGY CO LTD
- Filing Date
- 2026-03-11
- Publication Date
- 2026-06-23
AI Technical Summary
Existing facial acne detection algorithms struggle to balance accuracy and speed, and lack privacy protection mechanisms. Non-maximum suppression (NMS) post-processing becomes a performance bottleneck, resulting in poor real-time performance.
By integrating three state-of-the-art algorithms—RT-DETR v2, SAM 2, and YOLOv12—an end-to-end facial acne lesion detection system is constructed. The system extracts multi-scale features through the hybrid encoder of RT-DETR v2, enhances the detection of minute lesions through the region attention module of YOLOv12, and achieves privacy protection through the zero-shot segmentation capability of SAM 2.
It improves the efficiency and accuracy of facial acne lesion detection, increases inference speed by more than 30%, improves the accuracy of detecting minor lesions by 15%, and improves segmentation accuracy by 20%, achieving standardization of the entire process from image acquisition to hierarchical diagnosis.
Smart Images
Figure CN122265174A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of medical artificial intelligence technology, and in particular to a method and system for detecting and analyzing facial acne lesions. Background Technology
[0002] Facial acne is a common chronic inflammatory disease of the pilosebaceous unit, with a global prevalence of approximately 9.4%, reaching as high as 85% in adolescents. Accurate identification and statistical analysis of lesion types are crucial for developing treatment plans and evaluating efficacy. However, existing technologies have significant shortcomings in the following aspects: First, detection algorithms struggle to balance accuracy and speed. Existing methods often employ single-target detection models. For example, two-stage detection networks (Faster R-CNN, Cascade R-CNN) offer high accuracy but slow inference speed; single-stage detection networks (YOLO series, SSD) are fast but have low detection rates for small targets (<2mm), and are prone to false negatives and missed detections in densely populated acne areas (>50 lesions / cm²). Second, there is a lack of standardized privacy protection mechanisms. Existing methods often upload images containing complete facial biometrics to the cloud for processing, posing a serious risk of data privacy breaches. Third, non-maximum suppression (NMS) post-processing becomes a performance bottleneck. Traditional detectors rely on Non-Maximum Sampling (NMS) to eliminate duplicate detection boxes, but the computational complexity of NMS increases quadratically with the number of detection boxes, severely impacting real-time performance in scenarios with dense skin lesions. Therefore, constructing an efficient detection system suitable for detecting minute skin lesions is a pressing technical challenge. Summary of the Invention
[0003] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a method and system for detecting and analyzing facial acne lesions. By integrating three state-of-the-art algorithms—RT-DETR v2, SAM 2, and YOLOv12—it constructs an end-to-end, NMS-free, high-precision facial acne lesion detection and analysis system, which can improve the efficiency and accuracy of facial acne lesion detection and analysis.
[0004] In a first aspect, embodiments of this application provide a method for detecting and analyzing facial acne lesions, including: Acquire multi-view facial images of the detection object; wherein, the multi-view facial images are acquired from multiple perspectives, including: left side view, front view, and right side view; The multi-view facial images are processed using zero-shot privacy segmentation based on the SAM 2 model, removing the privacy regions of the eyes and mouth, and separating the left cheek region image, the right cheek region image, and the forehead region image. A combined region image is obtained by combining the left cheek region image, the right cheek region image, and the forehead region image; A locally deployed facial acne image recognition model is loaded, and the model is used to perform skin lesion detection processing on the input combined region image, outputting the target detection result; wherein, the fusion architecture of the facial acne image recognition model includes: RT-DETR v2 model and YOLOv12 model; The data is graded based on the target detection results, and a structured report is generated.
[0005] Secondly, embodiments of this application provide a facial acne lesion detection and analysis system, characterized in that it includes an image acquisition device, a controller, and a display device, wherein the image acquisition device is communicatively connected to the controller, and the controller is communicatively connected to the display device; the image acquisition device is used to acquire multi-view facial images, the controller is configured to execute the facial acne lesion detection and analysis method as described in any one of the embodiments of the first aspect, and the display device is used to display the target detection results.
[0006] This application embodiment includes: in the process of facial acne lesion detection and analysis, firstly, acquiring multi-view facial images of the detection object; wherein, the multi-view facial images are acquired from multiple perspectives, including: left side perspective, frontal perspective, and right side perspective; performing zero-sample privacy segmentation processing on the multi-view facial images based on the SAM2 model, removing the eye privacy region and mouth privacy region, and separating the left cheek region image, right cheek region image, and forehead region image; combining the left cheek region image, the right cheek region image, and the forehead region image to obtain a region combination image; loading a locally deployed facial acne image recognition model, and performing lesion detection processing on the input region combination image through the facial acne image recognition model, outputting the target detection result; wherein, the fusion architecture of the facial acne image recognition model includes: RT-DETR v2 model and YOLOv12 model; performing data classification processing based on the target detection result, and generating a structured report. First, this application constructs a detection architecture that integrates the RT-DETR v2 and YOLOv12 models. The RT-DETR v2 model serves as the backbone detector, utilizing its hybrid encoder (AIFI+CCFF) to extract multi-scale features and generating initial detection results through IoU-aware query selection. The YOLOv12 model acts as the auxiliary detector, enhancing its ability to perceive minute skin lesions through a region attention module (A²). The results from the backbone and auxiliary detectors are fused using adaptive confidence weighting, improving detection accuracy while eliminating the NMS bottleneck. Second, this application proposes a privacy-preserving and region segmentation method based on SAM2: leveraging SAM2's zero-shot segmentation capability, it accurately segments privacy-sensitive regions such as the eyes and mouth using point prompts, constructing precise polygon masks to achieve privacy desensitization at the source of data processing. Third, this application achieves standardization of the entire process from image acquisition to hierarchical diagnosis. By employing multi-view standardized imaging, zero-sample privacy segmentation, fusion architecture intelligent detection, and case-driven hierarchical diagnosis, a repeatable and quantifiable acne diagnosis process has been established, providing reliable data support for clinical diagnosis and treatment. In other words, the embodiments of this application can construct an end-to-end, NMS-free, high-precision facial acne lesion detection and analysis system by integrating three state-of-the-art algorithms: RT-DETR v2, SAM 2, and YOLOv12, thereby improving the efficiency and accuracy of facial acne lesion detection and analysis.
[0007] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the description and the accompanying drawings. Attached Figure Description
[0008] Figure 1 This is a schematic diagram of the structure of a facial acne lesion detection and analysis system provided in one embodiment of this application; Figure 2 This is a schematic flowchart of a facial acne lesion detection and analysis method provided in one embodiment of this application; Figure 3 This is a schematic diagram of the zero-sample privacy segmentation process provided in one embodiment of this application; Figure 4 This is a schematic diagram of the fusion architecture of a facial acne image recognition model provided in one embodiment of this application; Figure 5 This is a schematic diagram of the structure of a hybrid encoder provided in one embodiment of this application. Detailed Implementation
[0009] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments.
[0010] It should be noted that although a logical order is shown in the flowcharts in this application, in some cases, the steps shown or described may be performed in a different order than that shown in the flowcharts. In the description of this application, "several" means one or more, and "more" means two or more. The terms "first" and "second" are used only to distinguish technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of technical features indicated, or implicitly indicating the order in which the technical features are indicated.
[0011] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.
[0012] Facial acne is a common chronic inflammatory disease of the pilosebaceous unit, with a global prevalence of approximately 9.4%, reaching as high as 85% in adolescents. Accurate identification and statistical analysis of lesion types are crucial for developing treatment plans and evaluating efficacy. However, existing technologies have significant shortcomings in the following aspects: First, detection algorithms struggle to balance accuracy and speed. Existing methods often employ single-target detection models. For example, two-stage detection networks (Faster R-CNN, Cascade R-CNN) offer high accuracy but slow inference speed; single-stage detection networks (YOLO series, SSD) are fast but have low detection rates for small targets (<2mm), and are prone to false negatives and missed detections in densely populated acne areas (>50 lesions / cm²). Second, there is a lack of standardized privacy protection mechanisms. Existing methods often upload images containing complete facial biometrics to the cloud for processing, posing a serious risk of data privacy breaches. Third, non-maximum suppression (NMS) post-processing becomes a performance bottleneck. Traditional detectors rely on Non-Matching Detection (NMS) to eliminate duplicate detection boxes, but the computational complexity of NMS increases quadratically with the number of detection boxes, which seriously affects real-time performance in scenarios with dense skin lesions.
[0013] In recent years, international research institutions have made breakthrough progress in the field of object detection: (1) RT-DETR v2 (released in July 2024) is a newly launched real-time detection Transformer architecture. It adopts a hybrid encoder (AIFI+CCFF) to decouple intra-scale interaction and cross-scale fusion. It directly outputs detection results through the IoU-aware Query Selection mechanism without NMS post-processing. It achieves real-time performance of 53.1% AP and 108 FPS on the COCO dataset. (2) SAM2 (released in 2024) is a zero-shot segmentation basic model that supports a unified architecture for image and video segmentation and has strong zero-shot generalization ability. (3) YOLOv12 (released in February 2025) introduces an attention-centered architecture, adopts a region attention module (A²) and a residual efficient layer aggregation network (R-ELAN), which significantly improves detection accuracy while maintaining real-time speed. However, the aforementioned latest algorithms have not been applied in the field of facial acne detection. How to effectively integrate the end-to-end detection capability of RT-DETR v2, the zero-shot segmentation capability of SAM2, and the attention mechanism of YOLOv12 to build an efficient system suitable for detecting small skin lesions is a technical problem that urgently needs to be solved.
[0014] Based on this, this application provides a method and system for detecting and analyzing facial acne lesions. By integrating three state-of-the-art algorithms—RT-DETR v2, SAM 2, and YOLOv12—it can construct an end-to-end, NMS-free, high-precision facial acne lesion detection and analysis system, thereby improving the efficiency and accuracy of facial acne lesion detection and analysis.
[0015] The embodiments of this application will be further described below with reference to the accompanying drawings.
[0016] like Figure 1 As shown, the facial acne lesion detection and analysis system includes an image acquisition device, a controller, and a display device. The image acquisition device is communicatively connected to the controller, and the controller is communicatively connected to the display device. The image acquisition device is used to acquire multi-view facial images. The controller is configured to execute the facial acne lesion detection and analysis method provided in the embodiments of this application. The display device is used to display the target detection results. Specifically, the controller is configured with a SAM2 model, a RT-DETR v2 model, a YOLOv12 model, and a fusion module. It can be understood that in the facial acne lesion detection and analysis system, the image acquisition device, the controller, and the display device cooperate to realize the facial acne lesion detection and analysis method provided in the embodiments of this application. By fusing the three latest algorithms, RT-DETR v2, SAM2, and YOLOv12, an end-to-end, NMS-free, high-precision facial acne lesion detection and analysis system is constructed, improving the efficiency and accuracy of facial acne lesion detection and analysis.
[0017] Those skilled in the art will understand that the system structure shown in the figures does not constitute a limitation on the embodiments of this application, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0018] It will be understood by those skilled in the art that the system architecture and application scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. It is known by those skilled in the art that with the evolution of system architecture and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.
[0019] Based on the above system structure, the following are various embodiments of the facial acne lesion detection and analysis method of this application.
[0020] Firstly, such as Figure 2 As shown, this facial acne lesion detection and analysis method can be applied to, for example... Figure 1 In the system framework shown, the facial acne lesion detection and analysis method may include, but is not limited to, steps S100 to S500.
[0021] Step S100: Acquire multi-view facial images of the object being detected; wherein, the multi-view facial images are acquired from multiple perspectives, including: left side view, front view and right side view.
[0022] Step S200: Perform zero-shot privacy segmentation on multi-view facial images based on the SAM 2 model, remove the privacy regions of the eyes and mouth, and separate the left cheek region image, right cheek region image and forehead region image.
[0023] Step S300: Combine the images of the left cheek region, the right cheek region, and the forehead region to obtain a combined region image.
[0024] Step S400: Load the locally deployed facial acne image recognition model, perform skin lesion detection processing on the input region combination image through the facial acne image recognition model, and output the target detection result; wherein, the fusion architecture of the facial acne image recognition model includes: RT-DETR v2 model and YOLOv12 model.
[0025] Step S500: Perform data classification processing based on the target detection results and generate a structured report.
[0026] In the process of facial acne lesion detection and analysis via steps S100 to S500, firstly, multi-view facial images of the target are acquired. These multi-view images are collected from multiple perspectives, including left side view, front view, and right side view. Zero-shot privacy segmentation is performed on the multi-view facial images based on the SAM2 model, removing the privacy regions of the eyes and mouth, resulting in images of the left cheek, right cheek, and forehead. A combined region image is obtained by combining these three images. A locally deployed facial acne image recognition model is loaded, and the model performs lesion detection processing on the input combined region image, outputting the target detection result. The fusion architecture of the facial acne image recognition model includes the RT-DETR v2 model and the YOLOv12 model. Data is then graded based on the target detection result, and a structured report is generated. First, this application constructs a detection architecture that integrates the RT-DETR v2 and YOLOv12 models. The RT-DETRv2 model serves as the backbone detector, utilizing its hybrid encoder (AIFI+CCFF) to extract multi-scale features and generating initial detection results through IoU-aware query selection. The YOLOv12 model acts as the auxiliary detector, enhancing its ability to perceive minute skin lesions through a region attention module (A²). The results from the backbone and auxiliary detectors are fused using adaptive confidence weighting, improving detection accuracy while eliminating the NMS bottleneck. Second, this application proposes a privacy-preserving and region segmentation method based on SAM2: leveraging SAM2's zero-shot segmentation capability, it accurately segments privacy-sensitive regions such as the eyes and mouth using point prompts, constructing precise polygon masks to achieve privacy desensitization at the source of data processing. Third, this application achieves standardization of the entire process from image acquisition to hierarchical diagnosis. By employing multi-view standardized imaging, zero-sample privacy segmentation, fusion architecture intelligent detection, and case-driven hierarchical diagnosis, a repeatable and quantifiable acne diagnosis process has been established, providing reliable data support for clinical diagnosis and treatment. In other words, the embodiments of this application can construct an end-to-end, NMS-free, high-precision facial acne lesion detection and analysis system by integrating three state-of-the-art algorithms: RT-DETR v2, SAM 2, and YOLOv12, thereby improving the efficiency and accuracy of facial acne lesion detection and analysis.
[0027] Further explaining step S100, in this step, high-quality input data is acquired through a standardized image acquisition process. In some embodiments, authentication and triggering are performed first: patient QR code information is collected, and the image acquisition process is initiated after successful verification. Other authentication methods can also be used, such as facial recognition, medical insurance card reading, etc. After authentication and triggering are completed, multi-view image acquisition is performed: facial images of the detection object from three perspectives—left side, front, and right side—are acquired through a preset image acquisition device. The horizontal angles of the three perspectives are -46° (left side), 0° (front), and +46° (right side), respectively.
[0028] According to some embodiments of this application, step S200 is further described, and step S200 includes, but is not limited to, steps S210 to S250.
[0029] Step S210: Input the multi-view facial images into the SAM 2 image encoder to generate image embeddings.
[0030] Step S220: Select positive sample points as point prompts in the eye privacy area and mouth privacy area.
[0031] Step S230: Convert the dot cue into a cue embedding using a cue encoder.
[0032] Step S240: Perform masking processing on the input cue embedding and image embedding using a mask decoder to output an accurate segmentation mask.
[0033] Step S250: Set the pixel values in the mask area indicated by the segmentation mask to zero or remove them to remove the privacy areas of the eyes and mouth, and separate the left cheek area image, the right cheek area image and the forehead area image.
[0034] Specifically, this application implements zero-shot privacy segmentation processing based on SAM 2 through steps S210 to S250. This step is one of the core components of this solution, utilizing the zero-shot segmentation capability of SAM 2 to achieve privacy protection and data standardization. See details... Figure 3 In the zero-shot privacy segmentation process, the image to be processed is fed into the SAM 2 image encoder (using the ViT-H / 16 architecture, containing 32 Transformer blocks) to generate a 256×256 pixel image embedding. Positive sample points are selected as cues in the privacy regions of the eyes and mouth: 5 points for the left eye and 5 points for the right eye; 8 points for the mouth. The cue encoder converts the point cues into cue embeddings, which are then input into the mask decoder (4 decoder layers) along with the image embeddings, outputting a precise segmentation mask. The pixel values within the masked region are set to zero, achieving effective isolation of the privacy region.
[0035] Through steps S210 to S250, effective isolation of the patient's facial privacy area was achieved.
[0036] After obtaining the left cheek region image, right cheek region image and forehead region image in step S200, step S300 combines the left cheek region image, right cheek region image and forehead region image to obtain a region combination image, so as to facilitate subsequent recognition of the region combination image.
[0037] Step S400 is further explained. Specifically, according to some embodiments of this application, in conjunction with... Figure 4 and Figure 5 As shown, the fusion architecture of the facial acne image recognition model includes: the RT-DETR v2 model and the YOLOv12 model; the RT-DETR v2 model includes: a first backbone network, a hybrid encoder, an IoU-aware query selection module, and a Transformer decoder; the hybrid encoder includes: an attention-based intra-scale feature interaction module and a CNN-based cross-scale feature fusion module; the YOLOv12 model includes: a second backbone network, a region attention module, and a feature pyramid network.
[0038] Specifically, RT-DETR v2 employs a hybrid encoder, which extracts multi-scale features through an attention-based intra-scale feature interaction module (AIFI) and a CNN-based cross-scale feature fusion module (CCFF); selects high-quality features from the hybrid encoder output as initial object queries through an IoU-aware query selection module; and directly predicts object categories and bounding boxes through a Transformer decoder without NMS post-processing.
[0039] Specifically, in the YOLOv12 model, the second backbone network includes: Residual Efficient Layer Aggregation Network (R-ELAN); the YOLOv12 model uses a region attention module ( By dividing the feature map into specific regions for efficient attention calculation, YOLOv12 enhances the detection capability of minor skin lesions through the region attention module; in addition, the YOLOv12 model uses the Residual Efficient Layer Aggregation Network (R-ELAN) to enhance training stability.
[0040] According to some embodiments of this application, step S400 is further described, wherein the input region combination image is processed by a facial acne image recognition model to detect skin lesions and output target detection results, including but not limited to steps S410 to S430.
[0041] Step S410: Process the region combination image using the RT-DETR v2 model to obtain the first output result.
[0042] According to some embodiments of this application, step S410 includes, but is not limited to, the following steps: extracting first multi-scale features from the region combination image through a first backbone network and a hybrid encoder; selecting high-quality features from the output first multi-scale features as initial object queries through an IoU-aware query selection module; and directly predicting object categories and bounding boxes through a Transformer decoder without NMS post-processing.
[0043] Step S420: Process the region combination image in parallel using the YOLOv12 model to obtain the second output result.
[0044] According to some embodiments of this application, the region combination image is processed by the YOLOv12 model to obtain a second output result, including but not limited to the following steps: extracting feature maps from the region combination image through a second backbone network; dividing the feature maps into specific regions for efficient attention calculation through a region attention module, and enhancing the detection of skin lesions smaller than a preset size; wherein the preset size is less than 16×16 pixels; extracting second multi-scale features using a feature pyramid network; performing parallel inference on the second multi-scale features, and outputting the skin lesion coordinates, classification labels, and confidence scores respectively.
[0045] Step S430: Perform adaptive confidence-weighted fusion processing on the first output of the RT-DETR v2 model and the second output of the YOLOv12 model to obtain the target detection result.
[0046] According to some embodiments of this application, in this step, the adaptive confidence weighted fusion processing includes: weighting and fusing the confidence scores of the overlapping regions using a preset weighted fusion formula to obtain the target confidence score; the weighted fusion formula is: S_final=β·S_detr+(1 β)·S_yolo.
[0047] Where S_final is the total confidence level; β is the first fusion weight coefficient, and the value of the first fusion weight coefficient β ranges from 0.6 to 0.8; (1 β) is the second fusion weight coefficient; S_detr is the confidence score of the RT-DETR v2 model; S_yolo is the confidence score of the YOLOv12 model.
[0048] This application achieves high-precision end-to-end detection without the need for NMS by integrating two state-of-the-art detectors, RT-DETR v2 and YOLOv12. Figure 3As shown, the RT-DETR v2 branch employs a hybrid encoder to extract multi-scale features: AIFI performs intra-scale interaction on S3, S4, and S5 features; CCFF aggregates multi-scale features into a unified feature sequence. The IoU-aware query selection module selects the features with the highest IoU scores in Top-K=300 as the initial object query. The Transformer decoder (6 layers) directly predicts the object category and bounding box without NMS post-processing. The YOLOv12 branch runs in parallel, using a region attention module (A²) to specifically detect tiny skin lesions smaller than 16×16 pixels. The outputs of the two detectors are fused using adaptive confidence weighting: S_final=β·S_detr+(1-β)·S_yolo, where β ranges from 0.6 to 0.8.
[0049] Through steps S410 to S430, by fusing two state-of-the-art detectors, RT-DETR v2 and YOLOv12, end-to-end, high-precision detection without NMS is achieved, improving the detection accuracy and efficiency of facial acne lesion detection and analysis.
[0050] According to some embodiments of this application, step S500 is further described, which involves performing data classification processing based on the target detection results and generating a structured report, including but not limited to steps S510 to S540.
[0051] Step S510: Obtain the skin lesion type classification and the corresponding quantity statistics based on the target detection results.
[0052] Step S520: Based on the classification of skin lesion types and the statistical results of the number of cases, perform retrieval processing in the local historical diagnostic database to determine a preset number of target similar cases from multiple historical cases.
[0053] According to some embodiments of this application, the retrieval process is used to employ a tree-based fast nearest neighbor retrieval algorithm to calculate similarity based on Euclidean distance, and obtain a preset number of target similar cases ranked first by the similarity; wherein, the formula for calculating similarity is S=1 / (1+D); where D is the Euclidean distance and S is the similarity.
[0054] In one example, step S520 specifically includes: First, a feature vector v1 = [n_1,n_2,n_3,n_4,n_5,n_6, d_L, d_R, d_F] is generated based on the classification of skin lesion types and the statistical results of the number of lesions. Wherein, n_1 refers to the number of comedones, i.e., the count of open comedones (blackheads); n_2 refers to the number of papules, i.e., the count of inflammatory red papules; n_3 refers to the number of pustules, i.e., the count of inflammatory lesions containing pus; n_4 refers to the number of nodules, i.e., their diameter; n_5 refers to the number of cysts, i.e., the count of cystic lesions containing pus or semi-solid material; n_6 refers to the number of scars, i.e., the count of atrophic or hypertrophic scars; d_L refers to the lesion density on the left side of the face, i.e., the total number of lesions per unit area (cm²) in the left cheek region; d_R refers to the lesion density on the right side of the face, i.e., the total number of lesions per unit area (cm²) in the right cheek region; and d_F refers to the lesion density on the forehead region, i.e., the total number of lesions per unit area (cm²) in the forehead region. It can be seen that the feature vector v is used to calculate the similarity with historical cases, where n_1~n_6 are the lesion type distribution features, and d_L, d_R, and d_F are the lesion spatial distribution features.
[0055] Secondly, a fast nearest neighbor retrieval algorithm based on tree structure is adopted in the retrieval process, and the Euclidean distance D=||v1-v2|| between the historical feature vector v2 and the feature vector v1 corresponding to the historical cases in the local historical diagnostic database is calculated.
[0056] Then, after obtaining the Euclidean distance D, the similarity is calculated based on the Euclidean distance. The formula for calculating the Euclidean distance similarity is S=1 / (1+D); the value of the similarity S ranges from (0,1]; where the larger the value of the similarity S, the higher the similarity, and the smaller the value of the similarity S, the lower the similarity.
[0057] Finally, the calculated similarity scores are used to rank the cases, and a preset number of historical cases with high similarity scores are selected as target similar cases. Specifically, when the preset number K is configured to 5, 5 target similar cases can be found.
[0058] Step S530: Based on a preset number of target similar cases, perform hierarchical processing to determine the hierarchical diagnosis results.
[0059] According to some embodiments of this application, the hierarchical processing employs a weighted voting mechanism, wherein the weight corresponding to each of the target similarity cases in the weighted voting mechanism is . Where K is the preset number of similar cases to the target. It is the similarity corresponding to the j-th target similar case; the weight of the target similar case is proportional to the corresponding similarity.
[0060] Specifically, the grading standards are: Grade 0, Grade 1, Grade 2, Grade 3, and Grade 4. Specifically, Grade 0 (none): no skin lesions, no visible acne lesions on the face; Grade 1 (very mild): a small number of comedones, occasional papules, total number of comedones <5, papules <3, no pustules, nodules, or cysts; Grade 2: (mild) comedones predominate, a small number of papules and pustules, 5-20 comedones, 3-10 papules, <5 pustules, no nodules or cysts; Grade 3 (moderate): multiple papules and pustules, a small number of nodules, >20 comedones, 10-30 papules, 5-15 pustules, <5 nodules, no cysts; Grade 4 (severe): numerous inflammatory skin lesions, multiple nodules and cysts; specifically, >30 papules, >15 pustules, 5 nodules or one cyst.
[0061] The weighted voting mechanism is implemented as follows: When the preset quantity K=5, the five most similar target cases are retrieved. Each target case corresponds to a historical classification result (level 0, 1, 2, 3, or 4). Voting is then conducted based on similarity. Specifically, the weight corresponding to the target case is calculated based on the similarity, and the voting score of the historical classification result is calculated based on the weight. Finally, the historical classification result with the highest voting score is determined as the target classification diagnosis result. If a tie occurs, the historical classification result corresponding to the target case with higher similarity is selected as the target classification diagnosis result. The classification standard is a quantitative refinement of the IGA classification standard, transforming the original qualitative description into a quantitative judgment standard based on the statistical number of skin lesions, thereby improving the objectivity and repeatability of the classification results.
[0062] Specifically, each target-similar case has a corresponding historical classification (level 0, level 1, level 2, level 3, or level 4) and weight. Based on the weight, the voting score corresponding to the historical classification is calculated, and the historical classification with the highest voting score is the classification diagnosis result for this time.
[0063] Step S540: Generate a structured report based on the classification and grading diagnosis results of skin lesion types.
[0064] Specifically, this step generates a structured report that includes skin lesion classification and count, grading diagnosis results, and treatment recommendations.
[0065] The data processing is completed through steps S510 to S540, and a structured report is output so that patients and doctors can intuitively understand the test results.
[0066] In summary, this application presents a method and system for detecting and analyzing facial acne lesions based on a fusion architecture of Real-Time Detection Transformer (RT-DETRv2), Zero-Shot Segmentation Model (SAM2), and Attention Mechanism Detector (YOLOv12). By integrating three of the latest algorithms released in 2024-2025—RT-DETRv2, SAM2, and YOLOv12—an end-to-end, high-precision detection system without NMS is constructed. The technical effects include the following three aspects: (1) No NMS post-processing is required, and the inference speed is improved by more than 30%; (2) The detection accuracy of small lesions is improved by more than 15%; (3) SAM2 zero-shot segmentation improves the segmentation accuracy by more than 20%; (4) A complete closed loop from imaging to hierarchical diagnosis is established, providing repeatable and quantifiable diagnostic evidence for clinical practice.
[0067] The above provides a detailed description of the preferred embodiments of this application. However, this application is not limited to the above-described embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application. All such equivalent modifications or substitutions are included within the scope defined by this application.
Claims
1. A method for detecting and analyzing facial acne lesions, characterized in that, include: Acquire multi-view facial images of the detection object; wherein, the multi-view facial images are acquired from multiple perspectives, including: left side view, front view, and right side view; The multi-view facial images are processed using zero-shot privacy segmentation based on the SAM 2 model, removing the privacy regions of the eyes and mouth, and separating the left cheek region image, the right cheek region image, and the forehead region image. A combined region image is obtained by combining the left cheek region image, the right cheek region image, and the forehead region image; A locally deployed facial acne image recognition model is loaded, and the model is used to perform skin lesion detection processing on the input combined region image, outputting the target detection result; wherein, the fusion architecture of the facial acne image recognition model includes: RT-DETR v2 model and YOLOv12 model; The data is graded based on the target detection results, and a structured report is generated.
2. The method for detecting and analyzing facial acne lesions according to claim 1, characterized in that, The SAM2 model-based zero-shot privacy segmentation process is applied to the multi-view facial images, removing privacy regions around the eyes and mouth, and separating the left cheek region image, right cheek region image, and forehead region image, including: The multi-view facial images are input into the SAM 2 image encoder to generate image embedding; Positive sample points are selected as point prompts in the eye privacy area and the mouth privacy area; The point cues are converted into cues embeddings using a cue encoder; The prompt embedding and the image embedding, which are input together, are masked by a mask decoder to output an accurate segmentation mask; The pixel values within the mask area indicated by the segmentation mask are set to zero or removed to remove the eye privacy area and the mouth privacy area, thereby separating the left cheek area image, the right cheek area image and the forehead area image.
3. The method for detecting and analyzing facial acne lesions according to claim 1, characterized in that, The RT-DETR v2 model includes: a first backbone network, a hybrid encoder, an IoU-aware query selection module, and a Transformer decoder; the hybrid encoder includes: an attention-based intra-scale feature interaction module and a CNN-based cross-scale feature fusion module; the YOLOv12 model includes: a second backbone network, a region attention module, and a feature pyramid network.
4. The method for detecting and analyzing facial acne lesions according to claim 3, characterized in that, The step of performing skin lesion detection processing on the input combined region image using the facial acne image recognition model and outputting target detection results includes: The first output result is obtained by processing the combined region image using the RT-DETR v2 model. The combined region image is processed in parallel using the YOLOv12 model to obtain a second output result; The first output of the RT-DETR v2 model and the second output of the YOLOv12 model are subjected to adaptive confidence-weighted fusion processing to obtain the target detection result.
5. The method for detecting and analyzing facial acne lesions according to claim 4, characterized in that, The process of processing the combined region image using the RT-DETR v2 model to obtain the first output result includes: First multi-scale features are extracted from the combined region image using the first backbone network and the hybrid encoder; The IoU-aware query selection module selects high-quality features from the output first multi-scale features as the initial object query. The Transformer decoder directly predicts object categories and bounding boxes without requiring NMS post-processing.
6. The method for detecting and analyzing facial acne lesions according to claim 4, characterized in that, The process of processing the combined region image using the YOLOv12 model to obtain a second output result includes: Feature maps are extracted from the combined region image using the second backbone network; The region attention module divides the feature map into specific regions for efficient attention calculation, and enhances the detection of skin lesions smaller than a preset size; wherein, the preset size is less than 16×16 pixels; The second multi-scale features are extracted using the aforementioned feature pyramid network; Parallel inference is performed on the second multi-scale feature to output the lesion coordinates, classification labels, and confidence scores.
7. The method for detecting and analyzing facial acne lesions according to claim 4, characterized in that, The adaptive confidence-weighted fusion process includes: The target confidence score is obtained by weighting and fusing the confidence scores of the overlapping regions using a preset weighted fusion formula; the weighted fusion formula is: S_final=β·S_detr+(1 β)·S_yolo; Where S_final is the total confidence level; β is the first fusion weight coefficient, and the value of the first fusion weight coefficient β ranges from 0.6 to 0.8; (1 β) is the second fusion weight coefficient; S_detr is the confidence score of the RT-DETR v2 model; S_yolo is the confidence score of the YOLOv12 model.
8. The method for detecting and analyzing facial acne lesions according to claim 4, characterized in that, The step of performing data classification processing based on the target detection results and generating a structured report includes: Based on the target detection results, obtain the skin lesion type classification and the corresponding quantity statistics of the skin lesion type classification; Based on the skin lesion type classification and the statistical results of the quantity, a search is performed in the local historical diagnostic database to determine a preset number of target similar cases from multiple historical cases; The graded diagnosis results are determined by grading the target similar cases based on a preset number of such cases. The structured report is generated based on the skin lesion type classification and the graded diagnosis results.
9. The method for detecting and analyzing facial acne lesions according to claim 8, characterized in that, The retrieval process employs a tree-based fast nearest neighbor retrieval algorithm, calculates similarity based on Euclidean distance, and obtains a preset number of target similar cases ranked first by the similarity score; wherein, the formula for calculating similarity is S=1 / (1+D); where D is the Euclidean distance and S is the similarity score; The hierarchical processing employs a weighted voting mechanism, in which the weight corresponding to each target similarity case is: Where K is the preset number of similar cases to the target. It is the similarity corresponding to the j-th target similar case; the weight corresponding to the target similar case is proportional to the corresponding similarity.
10. A facial acne lesion detection and analysis system, characterized in that, The device includes an image acquisition unit, a controller, and a display unit. The image acquisition unit is communicatively connected to the controller, and the controller is communicatively connected to the display unit. The image acquisition unit is used to acquire multi-view facial images. The controller is configured to execute the facial acne lesion detection and analysis method as described in any one of claims 1 to 9. The display unit is used to display the target detection results.