An intelligent analysis method and system based on dermoscope images

Abstract

The application discloses an intelligent analysis method and system based on dermoscope images, and belongs to the technical field of medical artificial intelligence. The method generates a lesion mask and contour through lesion segmentation, establishes a unified spatial coordinate system, and makes lesion measurement, ABCDE quantitative analysis, auxiliary classification and report annotation all based on the same lesion object. The scale calibration is performed by extracting the candidate scale structure, analyzing the arrangement period and establishing a period grid model, and the scale reliability is calculated. According to the image quality state and the scale reliability, a double-gated judgment is performed, and only when both conditions are met, the measurement result based on the real physical size is output. All analysis results and spatial evidence correlation are written into a unified structured case data to drive multi-form report output. The application realizes integrated collaborative analysis of lesion segmentation, scale measurement, quality evaluation, clinical quantification, auxiliary classification and report generation, and improves the consistency, reliability and traceability of the analysis results.

Description

Technical Field

[0001] This invention belongs to the field of medical artificial intelligence technology, specifically an intelligent analysis method and system based on dermoscopy images. Background Technology

[0002] A dermoscopy is a medical imaging device that uses polarized light or contact optical imaging to observe the skin surface and superficial structures. It can reduce surface scattering and reflection interference, allowing doctors to observe skin structural features that are difficult to see directly with the naked eye, such as pigment networks, striped structures, spherical structures, and blue-white structures.

[0003] In clinical diagnosis, doctors usually make a comprehensive assessment by combining the morphology, boundary, color, size and trend of change of the lesion. Among them, the ABCDE assessment rule (Asymmetry, Border, Color, Diameter, Evolution) is an internationally recognized auxiliary judgment method.

[0004] With the development of digital dermatoscopy and artificial intelligence technology, computer vision-based automatic dermatoscopy image analysis systems have become an important tool for auxiliary diagnosis in dermatology. Their core functions include: lesion segmentation, real physical scale measurement, image quality assessment, ABCDE clinical quantitative analysis, auxiliary classification, and structured report generation.

[0005] In existing intelligent dermoscopy image analysis systems, the aforementioned tasks are typically performed by independent algorithm modules: a lesion segmentation module extracts the lesion region, a scale measurement module calculates the physical dimensions, a quality assessment module determines image quality, a classification module predicts the disease category, and a reporting module generates the final report. While each module can complete its respective task, the lack of a unified spatial correlation mechanism for lesions throughout the analysis process means that existing systems cannot guarantee that all stages—segmentation, measurement, classification, and reporting—revolve around the same lesion object, the same spatial coordinate system, and the same physical constraints. This results in a lack of consistency and traceability throughout the entire analysis process. Summary of the Invention

[0006] The purpose of this invention is to provide an intelligent analysis method and system based on dermoscopy images to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: an intelligent analysis method based on dermoscopy images, comprising the following steps:

[0008] Acquire dermoscopic images to be analyzed, establish case information and record basic image parameters; perform lesion region segmentation on the dermoscopic images to generate lesion masks and corresponding lesion contours, and establish spatial coordinate information of the lesion masks and lesion contours; perform scale reference information recognition on the dermoscopic images, establish the conversion relationship between image pixel size and actual physical size based on the recognition results, and calculate the scale confidence level characterizing the reliability of the conversion relationship; perform image quality assessment on the dermoscopic images to determine the image quality status; perform dual gating judgment based on the scale confidence level and the image quality status, when the scale... When the credibility meets the preset conditions and the image quality meets the preset requirements, the output of lesion measurement results based on the actual physical size is allowed; based on the same lesion mask and the same lesion contour, the geometric parameters of the lesion are calculated, and dermoscopic ABCDE clinical quantitative analysis is performed; based on the lesion area defined by at least one of the lesion mask and the lesion contour, artificial intelligence-assisted classification is performed, and at least one of the lesion category probability and benign / malignant risk probability is output; all analysis results and corresponding spatial evidence information are associated and written into unified structured case data; an analysis report and visualization results are generated based on the unified structured case data.

[0009] As a further preferred embodiment of this technical solution, the step of segmenting the lesion region of the dermoscopy image to generate a lesion mask and the corresponding lesion contour specifically includes: initially locating the lesion region in the input image to obtain the range of suspected lesions; segmenting the lesion within the range of suspected lesions or the entire image to obtain a lesion probability map P(x,y), where x and y represent image coordinates, and P represents the probability that the position belongs to the lesion region; binarizing the lesion probability map according to a preset threshold T, and determining that the pixel belongs to the lesion region when P(x,y)≥T, thereby generating an initial lesion mask; optimizing the initial lesion mask, the optimization process including at least one of connected component analysis, abnormal region removal, hole filling, and boundary smoothing to obtain a final lesion mask M; and extracting the boundary of the final lesion mask M to generate the lesion contour.

[0010] As a further preferred embodiment of this technical solution: the step of identifying scale reference information in the dermoscopy image, establishing a conversion relationship between image pixel size and actual physical size based on the identification result, and calculating the scale reliability characterizing the reliability of the conversion relationship specifically includes: locating the scale reference region in the image; detecting candidate scale structures in the scale reference region, obtaining the direction information, positional relationship, and interval distance of each candidate scale; establishing a scale periodic model based on the repetition pattern among multiple candidate scales, wherein the scale periodic model determines the true scale spacing based on at least one of the following factors: consistency of candidate scale spacing, consistency of scale direction, continuity of scale arrangement, and abnormal edge detection results; calculating the actual physical length q=d / s corresponding to a unit pixel based on the actual physical distance d corresponding to adjacent standard scales and the image distance s, and establishing a conversion relationship between pixel distance and actual physical distance; calculating the scale reliability based on at least one of the following factors: effective scale quantity, scale arrangement consistency, periodic error, detection result stability, and consistency of multiple image enhancement results.

[0011] As a further preferred embodiment of this technical solution: the step of performing dual gating judgment based on the scale credibility and the image quality status specifically includes: when the image quality status meets the preset requirements and the scale credibility meets the preset conditions, entering the allowed physical quantity output state, outputting the lesion area, maximum caliper diameter, and minimum caliper diameter based on the real physical dimensions; when the image quality status does not meet the preset requirements, entering the quality restricted state, prohibiting the output of measurement results based on the real physical dimensions, and generating a quality anomaly prompt; when the scale credibility does not meet the preset conditions, entering the scale restricted state, prohibiting the output of measurement results based on the real physical dimensions, and generating a scale unavailable prompt; when the image quality status does not meet the preset requirements and the scale credibility does not meet the preset conditions, entering the joint restricted state, prohibiting the output of measurement results based on the real physical dimensions, and simultaneously generating a quality anomaly prompt and a scale unavailable prompt; in each restricted state, retaining and outputting pixel-level analysis results.

[0012] As a further preferred embodiment of this technical solution: the step of calculating the geometric parameters of the lesion based on the same lesion mask and the same lesion contour, and performing dermoscopic ABCDE clinical quantitative analysis specifically includes: calculating the pixel area of ​​the lesion based on the lesion mask M, and calculating the actual physical area of ​​the lesion in combination with the transformation relationship; calculating the maximum and minimum caliper diameters of the lesion based on the lesion contour, wherein the maximum caliper diameter is the physical distance between the two boundary points with the largest distance in the lesion contour, and the minimum caliper diameter is the physical distance in the orthogonal support direction corresponding to the direction of the maximum caliper diameter; based on the... The lesion mask M calculates at least one morphological parameter of the lesion, including perimeter, roundness, eccentricity, and convexity. The ABCDE clinical quantitative analysis includes: establishing a mirror relationship based on the lesion's principal axis to calculate asymmetry; dividing the lesion boundary into multiple sectors to analyze the degree of boundary irregularity; statistically analyzing the number and distribution of preset standard color categories within the lesion region; and detecting differential structures within the lesion region, wherein the differential structures include at least one of pigment networks, stripe structures, spherical structures, and blue-white structures. Wherein, A represents asymmetry, B represents boundary features, C represents color features, D represents differential structures, and E represents evolutionary features.

[0013] As a further preferred embodiment of this technical solution, the method further includes a lesion follow-up analysis step: acquiring at least two dermoscopic images of the same lesion at different time points; performing lesion segmentation, scale calibration, and image quality assessment on the images at each time point; when the image quality status at each time point meets the preset requirements and the scale consistency meets the preset conditions, outputting at least one evolution information among lesion area change, diameter change, ABCD quantitative index change, and classification probability change; when the image quality status at any time point does not meet the preset requirements or the scale consistency does not meet the preset conditions, outputting an incomparable status prompt.

[0014] As a further preferred embodiment of this technical solution: the image quality assessment includes the detection of at least one of the following influencing factors: image blur level, abnormal exposure state, shadow occlusion, hair occlusion, highlight reflection, and incomplete cropping of lesion area; wherein, the incomplete cropping of lesion area is determined based on the relative positional relationship between the lesion mask and the image boundary.

[0015] As a further preferred embodiment of this technical solution: the scale reference information includes at least one of the following: a physical scale in the image, a calibration pattern of known size, pixel spacing parameters provided by the imaging device, and calibration parameters related to the imaging distance; the calibration pattern is a checkerboard, a circular mark, or a coded mark.

[0016] As a further preferred embodiment of this technical solution: the lesion category includes at least one of melanoma, pigmented nevus, basal cell carcinoma, and vitiligo; the unified structured case data adopts JSON, XML, database records, or medical information exchange standard formats; the analysis report includes at least one output format of display interface, machine-readable data, HTML report, or PDF report.

[0017] This invention also provides an intelligent analysis system based on dermoscopy images, comprising:

[0018] The image input module is used to acquire dermoscopic images to be analyzed and record case information and basic image parameters;

[0019] The lesion segmentation module is used to obtain the target lesion area from the dermoscopy image and extract the lesion outline to generate lesion mask data;

[0020] The scale calibration module is used to identify scale reference information in the image, establish the conversion relationship between image pixel size and actual physical size, and calculate scale reliability.

[0021] The image quality assessment module is used to evaluate whether the input image meets the analysis requirements, which include at least one of the following: sharpness, exposure status, occlusion, reflection, and lesion integrity.

[0022] The quality and scale gating module is used to determine whether to allow the output of measurement results based on the actual physical size based on the image quality status and the scale confidence level.

[0023] The lesion analysis module is used to calculate the geometric parameters of lesions based on the same lesion mask and the same lesion contour, and to perform ABCDE clinical quantitative analysis.

[0024] An auxiliary classification module is used to output at least one of the lesion category probability and benign / malignant risk probability based on the lesion region defined by at least one of the lesion mask and the lesion outline.

[0025] The structured data output module is used to save lesion area information, measurement results, classification results, quality status, and interpretive evidence as unified structured case data.

[0026] The report generation module is used to generate at least one output format, namely, a display interface, machine-readable data, an HTML report, or a PDF report, based on the unified structured case data.

[0027] As a further preferred embodiment of this technical solution: the image input module performs integrity detection, format parsing, and color space unification processing on the input image, and records the original image width W. o and height Ho The system performs proportional scaling on the image to obtain the analysis image size. × And save the horizontal scaling ratio. = / and vertical scaling ratio = / The lesion contour, lesion mask, scale reference area, measurement endpoint, and classification interpretation area all record their corresponding coordinate spaces and can be converted between the original image coordinates and the analysis image coordinates according to the horizontal scaling ratio and the vertical scaling ratio.

[0028] As a further preferred embodiment of this technical solution: the system supports adjusting the inference strategy according to the device's computing power, adopting different number of models, different image enhancement methods, or different inference accuracies under different computing resource conditions, while maintaining consistent input data structure, consistent output result format, and consistent quality control rules; the system can be deployed on local devices, edge computing units, in-house servers, or controlled cloud environments, and each module can be located on the same device or deployed in a distributed manner through controlled interfaces.

[0029] Compared with the prior art, the beneficial effects of the present invention are:

[0030] 1. In this invention, by generating lesion masks and lesion outlines with spatial mapping relationships, the calculation of lesion area, measurement of caliper diameter, analysis of ABCDE indicators, input of classification models, and annotation of report images are all completed based on the same lesion area and the same coordinate system, which solves the problem of inconsistent analysis results and lack of traceability caused by independent processing of each module in the prior art.

[0031] 2. In this invention, by extracting candidate scale structures, analyzing scale arrangement periods, establishing a periodic grid model, and completing missing scales, half-period or double-period errors caused by double edges, missing scales, rotation, and reflection are effectively suppressed. At the same time, scale credibility is generated for downstream modules to use, solving the problem that existing scale measurement methods lack reliability assessment and still output incorrect physical dimensions when there are anomalies in the scale reference object.

[0032] 3. In this invention, measurement results based on real physical dimensions are only allowed to be output when the image quality meets the standard and the scale reliability meets the conditions. Otherwise, pixel-level results are retained and the reason for the limitation is output, which avoids the system outputting seemingly accurate but actually erroneous data under unreliable conditions, thereby improving the interpretability and risk controllability of the medical device output.

[0033] 4. In this invention, the lesion outline, measurement line, ABCD analysis area and classification heat map are all directly derived from the actual calculation object rather than being redrawn. All output forms come from the same data source, which fundamentally eliminates the problem of data inconsistency between different outputs, allowing doctors to directly verify the original basis of each conclusion. Attached Figure Description

[0034] Figure 1 This is a flowchart illustrating the overall process of an intelligent analysis method based on dermoscopy images.

[0035] Figure 2 A flowchart illustrating the system modules and data relationships;

[0036] Figure 3 Flowchart for geometric calibration of scale reference information;

[0037] Figure 4 A dual-gated state diagram for quality and scale;

[0038] Figure 5 A collaborative analysis diagram of lesion geometry, ABCD classification, and auxiliary classification;

[0039] Figure 6 Generate graphs for structured case data and various report formats. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] Example 1: Complete Implementation Process of Intelligent Analysis Method Based on Dermoscopy Images

[0042] This embodiment describes the complete implementation process of the method of the present invention.

[0043] (I) Image Acquisition and Basic Parameter Recording

[0044] The image input module acquires the dermoscopic image to be analyzed. Dermoscopic images can be acquired through a dermoscopic acquisition device, a mobile skin imaging device, a medical image management system, or a local / remote image database.

[0045] The system first performs an integrity check on the input image to verify whether the image file is complete and readable; then it performs format parsing to identify the image file format (such as JPEG, PNG, DICOM, etc.); and finally it performs color space unification processing to convert the image to a preset standard color space (such as RGB or Lab color space).

[0046] The system records the original image width. and height To accommodate the processing capabilities of different computing resources, the system can scale the image to obtain the analysis image size. × And save the horizontal scaling ratio. = / and vertical scaling ratio = / All subsequent spatial information, including lesion contours, lesion masks, scale reference regions, measurement endpoints, and classification interpretation regions, is recorded in their corresponding coordinate space. When it is necessary to restore the original image coordinates, coordinate transformation is performed according to the above proportional relationships.

[0047] (ii) Image quality assessment

[0048] The image quality assessment module performs quality analysis on the input image to determine if there are any factors that could affect the reliability of the results. The quality assessment includes the detection of the following factors:

[0049] (1) Image blur detection: The variance of the image gradient magnitude is calculated using the Laplacian operator. The image is converted to grayscale, and a 3×3 Laplacian kernel is used to convolve the image to calculate the variance of the response map. The smaller the variance, the more blurred the image. When the variance is lower than a preset threshold, the image is determined to have a blur problem.

[0050] (2) Exposure anomaly detection: Statistically analyze the histogram distribution of image pixel values. When the percentage of pixels with concentrated values ​​in dark or bright areas exceeds a preset threshold, the image is determined to have an exposure anomaly.

[0051] (3) Shadow detection: The image is converted to the HSV color space, and the dark area is segmented in the V channel using the Otsu method with adaptive thresholding. Noise is removed by combining morphological operations to obtain the shadow area mask. The proportion of the shadow area in the lesion area is calculated. When the proportion exceeds the preset threshold, the image is determined to have shadow occlusion.

[0052] (4) Hair occlusion detection: The hair region in the image is identified by using a hair segmentation model based on deep learning or traditional image processing methods such as Gabor filtering. The proportion of the hair region in the lesion region is calculated. When the proportion exceeds the preset threshold, the image is determined to have hair occlusion.

[0053] (5) High-brightness reflection detection: The image is converted to the HSV color space, and the high-brightness area is segmented in the V channel by thresholding. Combined with the S channel to remove colored highlights, a reflective area mask is obtained. The proportion of the reflective area in the lesion area is calculated. When the proportion exceeds the preset threshold, it is determined that the image has high-brightness reflection.

[0054] (6) Lesion cropping detection: After lesion segmentation, the bounding box of the lesion mask is calculated, and the distance between the bounding box and the four boundaries of the image is determined. When the distance between the bounding box of the lesion mask and the image boundary is less than a preset threshold, the lesion is determined to be incompletely cropped.

[0055] Image quality assessment is divided into two stages: the first stage is performed immediately after image input, completing an initial quality check independent of the lesion mask (including blur detection, exposure anomaly detection, shadow detection, hair occlusion detection, and highlight reflection detection); the second stage is performed after lesion segmentation, supplementing cropping and detection based on the lesion mask to form the final image quality status. When the image quality seriously fails to meet the analysis requirements (such as severe blurring or severe exposure anomalies), the system can directly terminate the subsequent analysis process and output a quality anomaly warning.

[0056] (III) Lesion segmentation

[0057] The lesion segmentation module performs lesion region localization and contour segmentation on the input image.

[0058] First, the lesion region is initially located in the input image to obtain the suspected lesion range. Localization methods can include image saliency analysis or traditional image processing methods. The scale region can be located using object detection methods (such as YOLO, Faster R-CNN, etc.).

[0059] Subsequently, lesion segmentation is performed within the localized region or the entire image to obtain a lesion probability map P(x,y), where x and y represent image coordinates, and P represents the probability that the location belongs to the lesion region. The segmentation model can employ convolutional neural networks (such as U-Net, DeepLab, etc.) or deep learning models such as the visual Transformer.

[0060] In one specific embodiment, a U-Net network structure is used for lesion segmentation. U-Net includes an encoder path and a decoder path. The encoder progressively extracts high-level semantic features of the image through convolutional and pooling layers. The decoder fuses high-level features with low-level detail features through upsampling and skip connections, ultimately outputting a lesion probability map of the same size as the input image. During the training phase, the network uses a large number of dermoscopic images labeled with lesion masks for supervised learning. The loss function combines binary cross-entropy loss and Dice loss.

[0061] The lesion probability map is binarized according to a preset threshold T: when P(x,y)≥T, the pixel is determined to belong to the lesion region, and an initial lesion mask is generated. The typical value range of the threshold T is 0.3 to 0.7, and in a preferred embodiment, it is 0.5.

[0062] The initial lesion mask is optimized, including:

[0063] (1) Connectivity analysis - retain the largest connected region as the main lesion region and remove isolated small regions;

[0064] (2) Hole filling - using morphological closing operation or flooding filling algorithm to fill the holes inside the lesion mask;

[0065] (3) Boundary smoothing – Morphological opening operation or Gaussian filtering is used to smooth the lesion boundary. After the above optimization process, the final lesion mask is obtained. The boundary of the final lesion mask is extracted to generate the lesion contour.

[0066] When multiple candidate lesions exist, the system can determine the target lesion based on parameters such as lesion confidence, region size, and location relationship, or create multiple lesion objects for independent analysis.

[0067] (iv) Scale reference information identification and scale calibration

[0068] The scale calibration module identifies scale reference information in the image. This scale reference information includes physical scales in the image, calibration patterns of known sizes (such as checkerboard patterns, circular markers, or coded markers), pixel spacing parameters provided by the imaging device, and calibration parameters related to the imaging distance. Scale region localization can be achieved using object detection methods (such as YOLO, Faster R-CNN, etc.), template matching methods, or edge detection methods.

[0069] The following explanation uses a solid-scale reference object with repeating scale structures as an example.

[0070] First, the system locates a scale reference region in the image using template matching, edge detection, or deep learning object detection methods. The scale reference region is typically located at the corner or edge of the image.

[0071] Then, image enhancement processing is performed on the scale region, including grayscale conversion, contrast enhancement, edge enhancement, and local magnification. Candidate scale structures in the scale reference region are then detected. Detection methods include edge detection (such as the Canny edge detector), line detection (such as the Hough transform), and corner detection. The detection results include the orientation information, position coordinates, and geometric features of each candidate scale.

[0072] The system establishes a scale-period model based on the repetitive patterns among multiple candidate scales. The specific steps are as follows:

[0073] (1) Analyze the directional distribution of all candidate scales to determine the main direction of the scale. By statistically analyzing the direction histogram, select the direction with the highest frequency as the main direction, and remove candidate scales that deviate from the main direction by more than a preset angle threshold as noise.

[0074] (2) Project the candidate tick positions onto a one-dimensional space along the main direction to obtain a one-dimensional position sequence. Analyze the spacing distribution between adjacent candidate ticks in the sequence, construct a spacing histogram, and identify the main peak in the histogram as the candidate period.

[0075] (3) Verify the rationality of the candidate period: Check whether the scale positions with the period as the interval have good consistency. When there is an abnormal double edge situation (one scale line produces two edges), the system detects half of the true spacing. Correction is made by analyzing the multiple relationship of the spacing - if a large number of spacings are detected as integer multiples of a certain base value, then the base value is taken as the true scale spacing.

[0076] (4) Based on the determined true scale intervals, a periodic grid model is constructed. The candidate scale positions are fitted onto the periodic grid using the least squares method or random sampling consensus method to further verify the regularity of the scale arrangement. At the same time, a common baseline is fitted to determine the starting position of the scale lines.

[0077] (5) Identify the location of missing scales based on the periodic grid model and common baseline, and complete them.

[0078] If the actual distance between adjacent standard scales is d millimeters and the corresponding image distance is s pixels, then the actual physical length q corresponding to a unit pixel is d / s millimeters / pixel, thus establishing the conversion relationship between pixel distance and actual physical distance.

[0079] Simultaneously, the system calculates scale reliability based on factors such as the number of effective scales, scale arrangement consistency, periodic error, detection result stability, and consistency of various image enhancement results. Scale reliability can be expressed as a value between 0 and 1, or divided into several levels such as "high reliability," "medium reliability," "low reliability," and "unreliable."

[0080] When the scale confidence level is insufficient (e.g., below a preset threshold), the system will not output millimeter-level size results, but will retain pixel-level analysis results and generate a scale unavailable message.

[0081] (v) Dual gating judgment of quality and dimensional standards

[0082] The quality and scale gating module performs dual gating judgments based on image quality status and scale confidence level:

[0083] When the image quality meets the preset requirements and the scale reliability meets the preset conditions, it enters the state of allowing physical quantity output, and outputs the lesion area, maximum caliper diameter and minimum caliper diameter based on the real physical size.

[0084] When the image quality does not meet the preset requirements, it enters a quality-limited state, prohibits the output of measurement results based on the actual physical size, and generates a quality anomaly warning sign.

[0085] When the scale reliability does not meet the preset conditions, the scale is restricted, the output of measurement results based on the actual physical size is prohibited, and a scale unavailable prompt is generated.

[0086] When the image quality does not meet the preset requirements and the scale reliability does not meet the preset conditions, it enters a joint restricted state, prohibiting the output of measurement results based on the actual physical size, and simultaneously generating a quality anomaly warning sign and a scale unavailable warning sign.

[0087] Under each restricted state, the system prohibits the output of measurement results based on the actual physical dimensions and outputs the corresponding restriction reason identifier, while retaining and outputting pixel-level analysis results.

[0088] (vi) Calculation of lesion geometric parameters and ABCDE clinical quantitative analysis

[0089] The lesion analysis module calculates the geometric parameters of the lesion and performs ABCDE clinical quantitative analysis based on the same lesion mask and the same lesion contour.

[0090] Geometric parameter calculations include:

[0091] (1) Lesion area: Based on the successful identification of the scale and the establishment of the pixel-physical size conversion relationship, the number of pixels in the lesion area is counted based on the lesion mask. Multiply by the area per unit pixel (q²) to obtain the actual physical area of ​​the lesion S = ×q² (mm²); If a valid scale mark cannot be identified or the scale is not reliable enough, the millimeter-level true area will not be output, and only the pixel-level area result will be retained;

[0092] (2) Maximum caliper diameter: Traverse all boundary point pairs on the lesion contour, calculate their physical distances, and take the maximum value as the maximum caliper diameter. In a preferred embodiment, the farthest point pair is efficiently calculated on the convex hull vertex using a rotating caliper algorithm;

[0093] (3) Minimum caliper diameter: After determining the direction of the maximum caliper diameter, calculate the minimum width of the lesion outline in the support direction orthogonal to that direction;

[0094] (4) Other morphological parameters: circumference, roundness (4π×area / circumference²), eccentricity, convexity, etc. are calculated based on the lesion mask.

[0095] ABCDE clinical quantitative analysis includes:

[0096] A (Asymmetry Analysis): Calculate the principal axis direction of the lesion (by calculating the covariance matrix of the pixel coordinates of the lesion region, taking the eigenvector corresponding to its largest eigenvalue as the principal axis direction), divide the lesion into two halves along the principal axis, compare the degree of difference between the two halves in area and color distribution, and generate an asymmetry score.

[0097] B (Boundary Analysis): Divide the lesion outline into multiple sectors (e.g., 8 or 16 sectors). Calculate the distance from the lesion boundary to the lesion center for each sector, and analyze the variance of the distances in each sector. The larger the variance, the more irregular the boundary. Simultaneously, Fourier descriptors can be used to perform frequency domain analysis on the lesion outline, and a comprehensive boundary irregularity score can be generated.

[0098] C (Color Analysis): Within the lesion area, in the RGB, HSV, or Lab color space, standard color categories (including black, dark brown, light brown, red, white, blue-gray, etc.) are identified through threshold segmentation, K-means clustering, or pixel classification models. The number of color categories and the pixel proportion of each category are counted, and a color score is generated.

[0099] D (Differential Structure Analysis): Detects differential structures within the lesion area using dermoscopy, including pigment networks, striations, spherical structures, and blue-white structures. Differential structure detection can employ multi-label segmentation models, multiple independent binary segmenters, object detection models, or traditional image processing methods (such as morphological analysis and texture analysis). For each differential structure, the output shows its presence and its location.

[0100] E (Evolution Analysis): For cases requiring follow-up analysis, multiple dermoscopic images of the same lesion at different time points are acquired. Lesion segmentation, scale calibration, and image quality assessment are performed on the images at each time point. When the image quality at each time point meets preset requirements and scale consistency meets preset conditions, evolution information such as area change, diameter change, ABCD quantification index change, and classification probability change is output. If the image quality at any time point does not meet preset requirements or scale consistency does not meet preset conditions, an incomparable status prompt is output.

[0101] (vii) Artificial intelligence-assisted classification

[0102] The auxiliary classification module performs artificial intelligence-assisted classification based on the lesion region defined by at least one of the lesion mask and the lesion outline.

[0103] In one specific embodiment, the lesion region image (cropped from the original image based on a lesion mask) is input into a trained deep learning classification model. The classification model may employ a convolutional neural network (such as ResNet-50, EfficientNet, etc.) or a visual Transformer architecture.

[0104] Training Phase: A large dataset of dermoscopic images labeled with lesion categories and benign / malignant labels was collected. This dataset covers various skin lesion types, including but not limited to melanoma, pigmented nevi, basal cell carcinoma, and vitiligo. Lesion segmentation and labeling were performed on each image to generate lesion masks. The lesion region images were used as model input, and the lesion category and benign / malignant labels were used as supervision signals. End-to-end training was performed using the cross-entropy loss function. Data augmentation techniques (including image rotation, translation, scaling, flipping, and color dithering) were employed during training to improve the model's generalization ability.

[0105] Inference phase: The lesion area image from the dermoscopic image to be analyzed is input into the trained classification model, and the model outputs the probability distribution of each lesion category (e.g., melanoma probability). %, probability of pigmented nevi %, probability of basal cell carcinoma %, probability of vitiligo (%) and benign / malignant risk probability (benign probability and malignant probability).

[0106] Simultaneously, class activation mapping or gradient-weighted class activation mapping methods are used to generate classification interpretation heatmaps, indicating the image regions that the model focuses on when making classification decisions. The classification interpretation heatmaps are spatially aligned with lesion masks and lesion contours to ensure that the interpreted regions are consistent with the spatial location of the lesions.

[0107] (viii) Generation of unified structured case data

[0108] The structured data output module associates all analysis results and corresponding spatial evidence information into a unified structured case database. This unified structured case database can be in JSON, XML, database records, or standard medical information exchange formats.

[0109] Unified structured case data should include at least the following information: basic case information (case number, patient information, collection time, etc.), basic image information (original image size, analyzed image size, scaling ratio, etc.), lesion spatial information (lesion mask coordinates, lesion contour coordinates), geometric parameters (pixel area, physical area, maximum caliper diameter, minimum caliper diameter, perimeter, circularity, etc.), ABCDE quantitative indicators (asymmetry score, boundary score, color score, differential structure detection results, etc.), auxiliary classification results (probability of each lesion category, probability of benign or malignant risk, classification interpretation area), image quality status (overall status and details of each quality indicator), scale information (scale reliability, scale transformation relationship), and gating status (allowed / restricted status and restriction reason identifier).

[0110] (ix) Report generation

[0111] The report generation module generates analysis reports and visualization results based on unified structured case data.

[0112] The evidence map generator overlays visual annotations onto the original image based on lesion masks, lesion outlines, measurement endpoints, asymmetry axes, boundary sectors, standard color regions, differential structure regions, and classification interpretation regions from the unified structured case data, generating evidence maps.

[0113] All outputs—software interface, JSON data, HTML reports, and PDF reports—come from the same data source (unified structured case data), ensuring consistency in case information, measurement results, classification probabilities, and evidence. Figure 1 To.

[0114] Example 2: Specific Implementation of a Scale Calibration Method Based on Repeated Scale Periodicity

[0115] This embodiment describes in detail the specific implementation of the scale calibration method based on the periodicity of repeated scales in the scale calibration module.

[0116] When a dermoscopy image contains a solid-scale reference (such as a ruler with repeating graduations), the system performs the following steps (see...). Figure 3 ):

[0117] Step 301: Candidate Tick Extraction

[0118] The system locates a scale reference region R in an image using template matching, edge detection, or deep learning object detection methods. The scale reference region is typically located at a corner or edge of the image. Grayscale conversion, contrast enhancement, edge enhancement, and local magnification are performed on the scale reference region. A Canny edge detector is used to extract all edges within the scale reference region, and Hough transform is used to detect straight line structures, selecting line segments with a length greater than a preset threshold and consistent direction as candidate tick marks. The detection results include the direction information, position coordinates, and geometric features of each candidate tick mark.

[0119] Step 302: Main Direction Filtering

[0120] Analyze the directional distribution of all candidate scales to determine the dominant direction. Map the directional angles of the candidate scales to the range of 0° to 180°, construct a direction histogram, and select the direction with the highest frequency as the dominant direction. Candidate scales that deviate from the dominant direction by more than a preset angle threshold (e.g., 15°) are discarded as noise.

[0121] Step 303: One-dimensional projection

[0122] Project the candidate tick marks onto a one-dimensional space along the principal direction. For the center point coordinates of each candidate tick mark ( , ), calculate the projection value of its unit vector u along the principal direction. Obtain a one-dimensional position sequence { }. Analyze the spacing distribution between adjacent candidate tick marks in this sequence. .

[0123] Step 304: Evaluation of Candidate Cycles

[0124] Calculate the spacing between all adjacent candidate tick marks. Construct a spacing histogram and identify the dominant peak value in the histogram. , as candidate periods. Verify the rationality of the candidate periods: for each candidate tick position t_i, check if there exists a position located at... Other candidate tick marks near (k is an integer). Count the number of candidate tick marks that conform to this periodic pattern. ,when Total number of candidate scales When the ratio is greater than a preset threshold (e.g., 0.7), the cycle is confirmed as a valid cycle.

[0125] When abnormal double-edge conditions exist, the dominant peak detected by the system may be half of the actual spacing. In this case, the system further checks whether there is a peak value of approximately 2 in the spacing histogram. The secondary peak, and whether there are a large number of intervals. For integer multiples of the same value, if a significant multiple relationship is detected, the base value (i.e., the minimum effective spacing) is taken as the true scale spacing. .

[0126] Step 305: Periodic grid fitting

[0127] Based on the determined actual scale spacing A periodic grid model is constructed. The candidate tick positions are fitted onto the periodic grid using the least squares method: an initial offset t_0 is found such that all candidate tick positions... and +k· The sum of squared residuals (where k is an integer) is minimized. The fitted residuals are used to evaluate the regularity of the scale arrangement.

[0128] Step 306: Common Baseline Fitting

[0129] Fit a common baseline to determine the start and end positions of the scale lines. Determine the minimum projection value of the scale lines along the principal direction. and maximum projection value To form a scale range [ , ].

[0130] Step 307: Complete missing scale marks

[0131] Based on the periodic grid model and common baseline, identify the locations of missing scale points. Within the scale range [ , Inside, with Generate a complete sequence of tick marks for the interval { +k· The positions where no corresponding candidate tick is found are marked as missing ticks. For missing ticks, interpolation is performed to complete them based on the average position of adjacent ticks.

[0132] Step 308: Scale and Confidence Output

[0133] Based on the actual physical distance d (determined by the known parameters of the scale reference) corresponding to adjacent standard scales and the image distance s (calculated from the completed scale sequence), the actual physical length q = d / s (mm / pixel) corresponding to a unit pixel is calculated, and the conversion relationship between pixel distance and actual physical distance is established.

[0134] The reliability of the scale is determined by a combination of the following factors: the number of effective scales (the more effective scales, the higher the reliability), the consistency of scale arrangement (the smaller the fitting residual, the higher the reliability), the period error (the smaller the deviation between the candidate period and the actual scale distance, the higher the reliability), and the stability of the detection results (the higher the consistency of the detection results under different image enhancement conditions, the higher the reliability).

[0135] In one specific embodiment, the scale confidence level C_scale can be calculated using the following formula:

[0136]

[0137] in, , , , For each weight, satisfy + + + =1; R_fit is the fitting residual, and R_max is the preset maximum allowable residual; A stability score (between 0 and 1) is assigned. In a preferred embodiment, =0.3, =0.25, =0.25, =0.2.

[0138] Example 3: Specific Implementation of the Auxiliary Classification Model

[0139] This embodiment describes in detail the specific implementation of the auxiliary classification module.

[0140] Training phase:

[0141] Collect a large dataset of dermoscopic images labeled with lesion categories and benign / malignant labels. The dataset should cover a variety of skin lesion types, including but not limited to melanoma, pigmented nevi, basal cell carcinoma, and vitiligo. Perform lesion segmentation and annotation on each image to generate a lesion mask. The dataset should contain at least 5000 images to ensure the model has sufficient generalization ability.

[0142] EfficientNet-B4 is used as the backbone network of the classification model. EfficientNet-B4 is an efficient convolutional neural network architecture that balances the network's depth, width, and resolution through a compound scaling method, maintaining high classification accuracy while having a moderate number of parameters and computational cost.

[0143] During training, lesion region images (cropped from the original images and scaled to a uniform size, such as 380×380 pixels, based on a lesion mask) are used as model input, and lesion category labels and benign / malignant labels are used as supervision signals. A weighted cross-entropy loss function is employed to mitigate the class imbalance problem in the dataset.

[0144]

[0145] in, The true label (0 or 1) for the i-th class. The probability of the i-th class predicted by the model. Let be the weight of the i-th class (the weight is proportional to the reciprocal of the number of samples in the class).

[0146] The optimizer used is AdamW, with an initial learning rate of 1×10⁻⁶. -4 The batch size was set to 32, the number of training rounds was set to 100, and the learning rate cosine annealing decay strategy was adopted.

[0147] To improve the model's generalization ability, data augmentation techniques are used during training, including: random rotation (-30° to 30°), random translation (±10%), random scaling (0.8 to 1.2 times), random horizontal and vertical flips (50% probability each), and color jitter (±20% each for brightness, contrast, and saturation).

[0148] Reasoning stage:

[0149] The lesion region image of the dermoscopic image to be analyzed (cropped and scaled to 380×380 pixels according to the lesion mask) is input into the trained EfficientNet-B4 model, and the model outputs the probability distribution of each lesion category.

[0150] In one specific embodiment, for a melanoma sample, the model outputs: melanoma probability is 92.3%, nevus probability is 3.1%, basal cell carcinoma probability is 2.8%, vitiligo probability is 0.5%, and the combined probability of the remaining categories is 1.3%. The system outputs melanoma as the lesion category for this case, and also outputs the corresponding benign / malignant risk probability (malignant probability is 94.6%, benign probability is 5.4%).

[0151] Meanwhile, the Gradient Weighted Class Activation Mapping (Grad-CAM) method is used to generate a classification interpretation heatmap. The calculation steps of Grad-CAM are as follows: (1) The input image is forward propagated through the model to obtain the predicted score of the target category. (2) Calculate the target category score Feature map of the last convolutional layer gradient (3) Perform global average pooling on the gradient of each channel to obtain the importance weight of that channel. (4) The feature maps of each channel are linearly combined according to their weights to obtain a coarse-grained heatmap H=ReLU( (5) Upsample the heatmap using bilinear interpolation to the input image size to obtain the final explanatory heatmap. Then, scale the explanatory heatmap according to the scaling ratio. , The image is converted to its original coordinate space and then overlaid with the lesion mask for display.

[0152] Alternative implementation plan

[0153] The present invention also provides the following alternative embodiments:

[0154] (1) The lesion region can be obtained by salient localization, traditional image segmentation, or manual interactive bounding box selection; the lesion mask can be obtained by deep learning segmentation (such as U-Net, DeepLab, SegFormer, etc.), active contour models, graph cut algorithms, or their fusion. The scale reference region (scale) can be located using object detection methods (such as YOLO, Faster R-CNN, etc.) or template matching methods. Lesion segmentation and scale detection are two independent tasks and can be implemented separately using different algorithms.

[0155] (2) Scale reference information may be physical scale reference objects, checkerboard grid, circular markers, coded markers, known size patches, device pixel pitch metadata, fixed imaging distance calibration parameters, or multi-source combinations.

[0156] (3) The repeating period can be determined by grid fitting, spectrum analysis, autocorrelation, robust regression, random sampling consistency or Bayesian estimation.

[0157] (4) Scale confidence can be generated by rule weighting, statistical models, probability models or learning quality evaluators.

[0158] (5) Image quality indicators can be implemented using spatial domain methods, frequency domain methods, no-reference image quality models, or labeled quality classification models; quality gating can be implemented using hard thresholds, hierarchical gating, or risk-weighted gating.

[0159] (6) The maximum caliper point pair can be determined by exhaustively enumerating convex hull vertices, rotating calipers, distance transformation, or equivalent geometric algorithms; the minimum caliper can be determined by orthogonal support lines, minimum width, or fitting the minor axis of an ellipse.

[0160] (7) Standard colors can be identified in RGB, HSV, Lab or other perceptual color spaces through threshold segmentation, clustering, nearest color center or pixel classification models.

[0161] (8) The differential structure can be implemented using a multi-label segmentation model, multiple independent binary segmenters, target detection model, key point detection model or traditional image processing methods.

[0162] (9) The classification model can be a single model, multi-model ensemble, test-time enhancement, probability calibration, weighted fusion or stacked fusion; the classification interpretation heatmap can be generated by class activation mapping, gradient weighted class activation mapping, occlusion sensitivity method or LIME.

[0163] (10) Structured results can be in JSON, XML, database records or medical information exchange standards; reports can be in PDF, HTML, device interface or print output.

[0164] (11) The system can be deployed on local devices, edge computing units, in-house servers or controlled cloud; each module can be located on the same device or can be deployed in a distributed manner through controlled interfaces.

[0165] (12) Follow-up registration can be done using feature point matching, contour registration, flexible registration or scale normalization methods; if reliable registration is not possible, only an incomparable status prompt will be output.

[0166] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An intelligent analysis method based on dermoscopy images, characterized in that, Includes the following steps: Acquire dermoscopic images to be analyzed, and establish case information and record basic image parameters; The dermoscopic image is segmented to generate a lesion mask and a corresponding lesion outline, and the spatial coordinate information of the lesion mask and the lesion outline is established. The dermoscopy image is subjected to scale reference information recognition. Based on the recognition results, a conversion relationship between image pixel size and actual physical size is established, and the scale confidence level, which characterizes the reliability of the conversion relationship, is calculated. The dermoscopy images are subjected to image quality assessment to determine their image quality status; A dual-gating judgment is performed based on the scale confidence level and the image quality status. When the scale confidence level meets the preset conditions and the image quality status meets the preset requirements, the lesion measurement results based on the actual physical size are allowed to be output. Based on the same lesion mask and the same lesion contour, the geometric parameters of the lesion are calculated, and a dermoscopic ABCDE clinical quantitative analysis is performed. Based on the lesion region defined by at least one of the lesion mask and the lesion outline, artificial intelligence-assisted classification is performed, and at least one of the lesion category probability and benign / malignant risk probability is output. All analysis results and corresponding spatial evidence information are correlated and written into a unified structured case data set; Analysis reports and visualization results are generated based on the unified structured case data.

2. The intelligent analysis method based on dermoscopy images according to claim 1, characterized in that, The step of segmenting the dermoscopy image to generate a lesion mask and corresponding lesion contour specifically includes: Initial localization of lesion areas is performed on the input image to obtain the range of suspected lesions; Segment the lesion within the suspected lesion area or the entire image area to obtain a lesion probability map P(x,y), where x and y represent image coordinates and P represents the probability that the location belongs to the lesion area. The probability map of the lesion is binarized according to a preset threshold T. When P(x,y)≥T, the pixel is determined to belong to the lesion region, and an initial lesion mask is generated. The initial lesion mask is optimized, and the optimization process includes at least one of connected region analysis, abnormal region removal, hole filling and boundary smoothing to obtain the final lesion mask M. Extract the boundary of the final lesion mask M to generate the lesion contour.

3. The intelligent analysis method based on dermoscopy images according to claim 1, characterized in that, The steps of performing scale reference information recognition on the dermoscopy image, establishing a conversion relationship between image pixel size and actual physical size based on the recognition result, and calculating the scale confidence level characterizing the reliability of the conversion relationship specifically include: Locate the scale reference region in the image; Detect candidate scale structures in the scale reference region to obtain the orientation information, positional relationship, and interval distance of each candidate scale. A scale periodic model is established based on the repetition pattern among multiple candidate scales. The scale periodic model determines the true scale spacing based on at least one of the following factors: consistency of candidate scale spacing, consistency of scale direction, continuity of scale arrangement, and abnormal edge detection results. Based on the actual physical distance d corresponding to adjacent standard scales and the image distance s, calculate the actual physical length q=d / s corresponding to a unit pixel, and establish the conversion relationship between pixel distance and actual physical distance; The scale reliability is calculated based on at least one of the following factors: effective scale count, scale arrangement consistency, periodic error, detection result stability, and consistency of multiple image enhancement results.

4. The intelligent analysis method based on dermoscopy images according to claim 1, characterized in that, The step of performing dual gating judgment based on the scale confidence level and the image quality status specifically includes: When the image quality status meets the preset requirements and the scale credibility meets the preset conditions, the system enters the physical quantity output state and outputs the lesion area, maximum caliper diameter, and minimum caliper diameter based on the actual physical dimensions. When the image quality does not meet the preset requirements, it enters a quality-limited state, prohibits the output of measurement results based on the actual physical size, and generates a quality anomaly warning sign. When the reliability of the scale does not meet the preset conditions, the scale is restricted, the output of measurement results based on the actual physical size is prohibited, and a scale unavailable prompt is generated. When the image quality status does not meet the preset requirements and the scale reliability does not meet the preset conditions, a joint restricted state is entered, prohibiting the output of measurement results based on the real physical size, and simultaneously generating a quality anomaly warning sign and a scale unavailable warning sign. Under each constrained state, pixel-level analysis results are retained and output.

5. The intelligent analysis method based on dermoscopy images according to claim 1, characterized in that, The steps of calculating the geometric parameters of the lesion based on the same lesion mask and the same lesion contour, and performing dermoscopic ABCDE clinical quantitative analysis, specifically include: The pixel area of ​​the lesion is calculated based on the lesion mask M, and the actual physical area of ​​the lesion is calculated in combination with the transformation relationship. The maximum and minimum caliper diameters of the lesion are calculated based on the lesion contour. The maximum caliper diameter is the physical distance between the two boundary points with the largest distance in the lesion contour, and the minimum caliper diameter is the physical distance in the orthogonal support direction corresponding to the direction of the maximum caliper diameter. Based on the lesion mask M, at least one morphological parameter among the lesion's perimeter, roundness, eccentricity, and convexity is calculated. The ABCDE clinical quantitative analysis includes: establishing a mirror relationship based on the lesion's main axis to calculate asymmetry; dividing the lesion boundary into multiple sectors to analyze the degree of boundary irregularity; statistically analyzing the number and distribution of preset standard color categories within the lesion region; and detecting differential structures within the lesion region, wherein the differential structures include at least one of pigment networks, stripe structures, spherical structures, and blue-white structures. Where A represents asymmetry, B represents boundary features, C represents color features, D represents differential structure, and E represents evolutionary features.

6. The intelligent analysis method based on dermoscopy images according to claim 1, characterized in that, It also includes lesion follow-up analysis steps: Obtain at least two dermoscopic images of the same lesion at different time points; Lesion segmentation, scale calibration, and image quality assessment were performed on images at each time point. When the image quality at each time point meets the preset requirements and the scale consistency meets the preset conditions, output at least one evolution information among the changes in lesion area, diameter, ABCD quantification index, and classification probability. If the image quality at any point in time does not meet the preset requirements or the scale consistency does not meet the preset conditions, an incomparable status prompt will be output.

7. An intelligent analysis system based on dermoscopy images, characterized in that, include: The image input module is used to acquire dermoscopic images to be analyzed and record case information and basic image parameters; The lesion segmentation module is used to obtain the target lesion area from the dermoscopy image and extract the lesion outline to generate lesion mask data; The scale calibration module is used to identify scale reference information in the image, establish the conversion relationship between image pixel size and actual physical size, and calculate scale reliability. The image quality assessment module is used to evaluate whether the input image meets the analysis requirements, which include at least one of the following: sharpness, exposure status, occlusion, reflection, and lesion integrity. The quality and scale gating module is used to determine whether to allow the output of measurement results based on the actual physical size based on the image quality status and the scale confidence level. The lesion analysis module is used to calculate the geometric parameters of lesions based on the same lesion mask and the same lesion contour, and to perform ABCDE clinical quantitative analysis. An auxiliary classification module is used to output at least one of the lesion category probability and benign / malignant risk probability based on the lesion region defined by at least one of the lesion mask and the lesion outline. The structured data output module is used to save lesion area information, measurement results, classification results, quality status, and interpretive evidence as unified structured case data. The report generation module is used to generate at least one output format, namely, a display interface, machine-readable data, an HTML report, or a PDF report, based on the unified structured case data.

8. The intelligent analysis system based on dermoscopy images according to claim 7, characterized in that, The image input module performs integrity checks, format parsing, and color space unification processing on the input image, and records the original image width. and height The system performs proportional scaling on the image to obtain the analysis image size. × And save the horizontal scaling ratio. = / and vertical scaling ratio = / The lesion contour, lesion mask, scale reference area, measurement endpoint, and classification interpretation area all record their corresponding coordinate spaces and can be converted between the original image coordinates and the analysis image coordinates according to the horizontal scaling ratio and the vertical scaling ratio.

9. The intelligent analysis system based on dermoscopy images according to claim 7, characterized in that, The scale calibration module identifies candidate scale structures in the scale reference area, establishes a scale period model based on the repetition pattern among multiple candidate scales, and determines the true scale spacing by analyzing the consistency of candidate scale spacing, scale direction consistency, scale arrangement continuity, and abnormal edge conditions. The scale reliability is determined based on at least one of the following factors: the number of effective scales, the consistency of scale arrangement, periodic error, the stability of detection results, and the consistency of multiple image enhancement results; the scale reference information includes at least one of the following: a physical scale in the image, a calibration pattern of known size, pixel spacing parameters provided by the imaging device, and calibration parameters related to the imaging distance.

10. The intelligent analysis system based on dermoscopy images according to claim 7, characterized in that, The system supports adjusting the inference strategy according to the device's computing power, using different number of models, different image enhancement methods, or different inference accuracies under different computing resource conditions, while maintaining consistent input data structure, consistent output result format, and consistent quality control rules. The system can be deployed on local devices, edge computing units, in-house servers, or controlled cloud environments. Each module can be located on the same device or deployed in a distributed manner through controlled interfaces.

Images