Multimodal data fusion-based pigmented disease diagnosis method, device, equipment and medium

Abstract

The invention relates to a multimodal data fusion-based pigmented disease diagnosis method, apparatus and device, and a medium. The method comprises the following steps: firstly, acquiring multi-modal data of pigmented diseases, and unifying resolution and wavelength range to obtain a standardized multi-modal data set; on the basis of the data set, each modal feature is extracted by using a convolutional neural network, a data quality score is calculated, a replacement weight is calculated for a missing modal, and a multi-modal feature data set is obtained through complete modal correlation feature supplementation. Setting a weight according to a lesion feature diagnosis contribution degree, fusing features such as asymmetry and diameter change by using weighted average to generate a fusion feature vector, and performing principal component analysis dimensionality reduction to obtain a dimensionality reduction feature vector; finally, diagnosis classification is judged according to a preset malignant correlation threshold value, similar risk cases are grouped in combination with a clustering algorithm, and a personalized diagnosis and treatment scheme is generated according to risk characteristics and diagnosis and treatment requirements. According to the method, the accuracy and reliability of pigmented disease diagnosis can be improved, risk grouping and personalized diagnosis and treatment are realized, and the treatment pertinence is improved.

Description

Technical Field

[0001] This invention belongs to the field of clinical diagnosis, and in particular relates to methods, devices, equipment and media for the diagnosis of pigmentary diseases using multimodal data fusion. Background Technology

[0002] With the deep integration of medical imaging technology and artificial intelligence in clinical diagnosis, the diagnosis of pigmented diseases (such as melanoma and nevi) is gradually evolving from traditional single-modality reliance to multimodal data collaborative analysis. However, traditional diagnostic methods for pigmented diseases and early intelligent solutions still have significant limitations: On the one hand, traditional diagnosis relies heavily on doctors' experience in interpreting single-modality data. Dermoscopy images are easily affected by lighting and hair occlusion, and pathological sections are invasive and cannot be applied to early screening in real time. The fragmentation of multimodal information can easily lead to biased diagnostic conclusions. On the other hand, early intelligent diagnostic models are mostly developed for single-modality applications and have not established a systematic multimodal data fusion mechanism. They also ignore data quality differences (such as low-resolution images and noise interference signals) and the problem of missing modalities. They lack effective noise reduction and missing data completion methods, and the output results are mostly limited to a binary classification of "benign / malignant," making it difficult to combine the rate of disease progression and individual patient differences to provide risk stratification and personalized treatment guidance, thus failing to fully meet the actual needs of precision medicine in clinical practice. Summary of the Invention

[0003] Therefore, it is necessary to provide a multimodal data fusion diagnostic method, device, equipment, and medium for pigmentary diseases that can improve the accuracy and efficiency of diagnosis and provide more targeted clinical guidance, addressing the aforementioned technical problems.

[0004] In a first aspect, this application provides a diagnostic method for pigmentary disorders based on multimodal data fusion, including:

[0005] Multimodal data of pigmentary diseases were acquired, and the resolution and wavelength range of the multimodal data were standardized to obtain a standardized multimodal dataset.

[0006] Based on a standardized multimodal dataset, a convolutional neural network is used to extract modal features and calculate data quality scores. Low-quality data is denoised and replacement weights for missing modalities are determined to obtain a multimodal feature dataset.

[0007] After fusing lesion features using a weighted average method on the multimodal feature dataset, the dimensionality-reduced feature vectors are obtained by processing them using the principal component analysis algorithm.

[0008] The diagnostic classification is obtained by judging the dimensionality reduction feature vector through a preset malignant association threshold. Based on the diagnostic classification, cases with similar risk characteristics are grouped, and personalized treatment plans are generated by combining the risk characteristics of the groups and the treatment needs.

[0009] In one embodiment, a convolutional neural network is used to extract modal features and calculate data quality scores based on a standardized multimodal dataset. Low-quality data is denoised, and alternative weights for missing modalities are determined to obtain a multimodal feature dataset, including:

[0010] Based on a standardized multimodal dataset, convolutional neural networks are used to extract feature information for each modality, resulting in a modal feature vector set.

[0011] For each set of feature data in the modal feature vector set, calculate the corresponding data quality score to obtain the quality assessment result.

[0012] Based on the quality assessment results, feature data with quality scores below a preset threshold are selected, and the low-quality feature data is denoised to obtain the denoised modal feature dataset.

[0013] The modal feature dataset is evaluated. If each modal feature in the dataset does not cover the preset key feature dimensions and the proportion of missing feature data is higher than the preset threshold, the substitution weights corresponding to the missing modalities are calculated to obtain the feature dataset with the substitution weights for the missing modalities.

[0014] Key feature dimensions include the lesion boundary dimension of dermoscopy images, the cell structure dimension of histopathological sections, and the characteristic wavelength dimension of spectral analysis data.

[0015] Based on the feature dataset, alternative weights are used to generate supplementary features by fusing the feature data associated with the missing modalities and the complete modalities. The supplementary features are then filled into the missing positions to obtain a complete multimodal feature dataset.

[0016] In one embodiment, the data quality score is calculated using the following formula:

[0017]

[0018] in, This represents the data quality score, with a range of values. , This represents the weighting coefficients, which are preset based on modal characteristics. This represents the number of features in the modal feature vector set that meet the preset feature validity threshold. This represents the total number of features in the modal feature vector set. The energy value of the noise signal in the feature data is represented by a filter and calculated. This represents the energy value of the effective signal in the feature data.

[0019] In one embodiment, after fusing lesion features using a weighted average method on the multimodal feature dataset, principal component analysis is used to process the data to obtain a dimensionality-reduced feature vector, including:

[0020] Lesion features related to pigmentary diseases are extracted from the multimodal feature dataset to obtain a lesion feature set; lesion features include asymmetry, diameter variation, and differences in pigment distribution.

[0021] A pre-defined weighted average method is used to fuse all lesion features in the lesion feature set to obtain a fused feature vector; the weights of the weighted average method are set according to the diagnostic contribution of each lesion feature.

[0022] Extract disease attributes that have key distinguishing properties for disease diagnosis from the fused feature vectors to generate a disease attribute set; disease attributes include malignancy risk association attributes and disease progression characteristic attributes.

[0023] Based on the lesion attribute set, a standardization algorithm is used to unify the numerical range of each attribute to obtain a standardized feature vector.

[0024] Principal component analysis is applied to the standardized feature vectors to reduce their dimensionality, retaining the main feature information and removing redundant dimensions, thus obtaining the dimensionality-reduced feature vectors.

[0025] In one embodiment, a diagnostic classification is obtained by judging the dimensionality-reduced feature vector through a preset malignant association threshold. Based on the diagnostic classification, cases with similar risk characteristics are grouped. Personalized treatment plans are then generated by combining the risk characteristics of the groups with treatment needs, including:

[0026] The feature dimension data and feature value distribution are extracted from the dimensionality-reduced feature vector, and a preset malignant association threshold is used to perform diagnostic classification judgment to obtain the diagnostic classification result.

[0027] The deviation correction coefficients for matching are calculated based on the diagnostic classification results, and the initial calibration parameters are obtained through parameter mapping transformation.

[0028] Based on the initial calibration parameters, an iterative optimization algorithm is used to calibrate the dimensionality-reduced feature vector to obtain the calibrated feature vector.

[0029] The malignancy assessment results of pigmentary diseases are determined by analyzing the quantitative indicators related to the degree of malignancy in the calibrated feature vectors; the quantitative indicators include the probability of malignancy and the rate of disease progression.

[0030] Risk probability distribution data for different disease stages are extracted from the malignancy assessment results, and a risk assessment matrix is ​​constructed according to preset dimensions.

[0031] Based on the risk assessment matrix, a clustering algorithm is applied to group cases with similar risk characteristics, resulting in disease risk clustering groups.

[0032] Personalized treatment plans are generated by combining the risk characteristics and treatment needs of each group in the disease risk clustering; the treatment needs include treatment sensitivity parameters and follow-up period.

[0033] In one embodiment, the deviation correction factor is calculated using the following formula:

[0034]

[0035] in, The deviation correction factor is represented, and its value range is [not specified]. , This represents the average deviation coefficient of historical cases corresponding to the current diagnostic category. This indicates the confidence level of the current diagnostic classification result. This represents the mean deviation rate between the actual values ​​of the core lesion features in the current dimensionality-reduced feature vector and the feature values ​​of the diagnostic classification criteria. This represents the bias sensitivity coefficient.

[0036] Secondly, this application also provides a diagnostic device for pigmentary diseases based on multimodal data fusion, the device comprising:

[0037] The multimodal data processing module is used to acquire multimodal data of pigmentary diseases, and to perform resolution and wavelength range unification processing on the multimodal data to obtain a standardized multimodal dataset.

[0038] The feature extraction optimization module is used to extract modal features and calculate data quality scores based on a standardized multimodal dataset using a convolutional neural network, denoise low-quality data and determine alternative weights for missing modalities to obtain a multimodal feature dataset.

[0039] The lesion feature fusion module is used to fuse lesion features from a multimodal feature dataset using a weighted average method, and then process them using principal component analysis to obtain a dimensionality-reduced feature vector.

[0040] The diagnostic plan generation module is used to judge the dimensionality reduction feature vector by a preset malignant association threshold to obtain a diagnostic classification, group cases with similar risk characteristics according to the diagnostic classification, and generate personalized diagnosis and treatment plans by combining the risk characteristics of the groups and the diagnosis and treatment needs.

[0041] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0042] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0043] The aforementioned multimodal data fusion diagnostic method, device, computer equipment, and storage medium for pigmentary diseases first acquire multimodal data of pigmentary diseases, encompassing multi-source information related to lesions, such as dermoscopic images, histopathological sections, and spectral analysis data. Subsequently, the acquired multimodal data undergoes preprocessing to unify the resolution and wavelength range of different modalities and eliminate interference from data format differences, resulting in a standardized multimodal dataset. Based on the standardized multimodal dataset, a convolutional neural network is used to extract feature information for each modality, while simultaneously calculating the data quality score corresponding to each set of feature data. For low-quality feature data with a quality score below a preset threshold, denoising is performed using techniques such as filtering. If some modalities are missing, the replacement weights corresponding to the missing modalities are further calculated, and the missing modalities are supplemented using the correlation features of the complete modalities, ultimately obtaining a multimodal feature dataset. For a multimodal feature dataset, weights are assigned based on the diagnostic contribution of each lesion feature. A weighted average method is used to fuse lesion features such as asymmetry, diameter variation, and pigment distribution differences, generating a fused feature vector. Principal component analysis is then used to reduce the dimensionality of the fused feature vector, retaining key feature information and removing redundant dimensions, resulting in a dimensionality-reduced feature vector. Finally, a preset malignancy association threshold is used to evaluate the dimensionality-reduced feature vector, determining the diagnostic classification of pigmentary diseases. Based on the diagnostic classification results, a clustering algorithm is applied to group cases with similar risk characteristics, forming disease risk clusters. Combining the risk characteristics of each group (such as malignancy probability value and lesion progression rate) and treatment needs (such as treatment sensitivity parameters and follow-up period), personalized treatment plans are generated for different patients. This method significantly improves the accuracy and reliability of pigmentary disease diagnosis. Through multimodal information complementarity and quality control, it reduces diagnostic bias caused by limitations or interference from single data sources, providing more accurate disease judgment criteria for clinical practice. Meanwhile, its risk grouping and personalized treatment plan generation capabilities can overcome the limitations of traditional diagnoses that are "uniform in conclusion and lack stratification," matching appropriate treatment strategies for patients with different risk levels and individual needs, improving the targeting of treatment and the rationality of follow-up, and facilitating the implementation of precision medicine in clinical practice. Furthermore, through standardized processes and intelligent processing, this method reduces the over-reliance on individual physician experience in diagnosis, improving the diagnostic level in primary healthcare institutions or non-specialist settings, effectively alleviating the problem of uneven distribution of medical resources leading to differences in treatment capabilities, and expanding the application scope of high-quality diagnostic technologies. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 A flowchart of a multimodal data fusion diagnostic method for pigmentary diseases provided in an embodiment of the present invention;

[0046] Figure 2 This is a structural block diagram of a multimodal data fusion diagnostic device for pigmentary diseases provided in an embodiment of the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0048] In one embodiment, such as Figure 1 As shown, this application provides a method for diagnosing pigmentary disorders based on multimodal data fusion, which may include the following steps:

[0049] Step S101: Obtain multimodal data of pigmentary diseases, and perform resolution and wavelength range unification processing on the multimodal data to obtain a standardized multimodal dataset.

[0050] Specifically, multimodal data on pigmentary diseases is acquired, covering multi-source information related to the lesions. This data includes, but is not limited to, dermoscopic images (used to present macroscopic features such as lesion surface morphology and pigment distribution), histopathological section data (used to reflect microscopic features such as lesion cell structure and nuclear morphology), and spectral analysis data (used to provide chemical characteristics such as lesion material composition and molecular structure). After data acquisition, preprocessing is performed to address the format differences of different modalities: for image data (such as dermoscopic images and histopathological sections), interpolation algorithms are used to unify the resolution, ensuring consistent pixel size and image ratio across different sources; for spectral analysis data, wavelength calibration technology is used to unify the wavelength measurement range, enabling direct comparison of spectral data acquired by different devices in the wavelength dimension.

[0051] Step S102: Based on the standardized multimodal dataset, a convolutional neural network is used to extract modal features and calculate the data quality score. The low-quality data is denoised and the replacement weights of the missing modalities are determined to obtain the multimodal feature dataset.

[0052] Based on a standardized multimodal dataset, a convolutional neural network is used for modal feature extraction: for image-related modalities, layer-by-layer features such as lesion edges, textures, and pigment regions are extracted step by step through convolutional and pooling layers; for spectral modalities, key information such as signal intensity and peak distribution corresponding to feature wavelengths is extracted through an adapted convolutional kernel structure, ultimately outputting a set of modal feature vectors covering the core information of each modality. Simultaneously, a data quality score is calculated for each set of modal feature vectors. The calculation logic combines feature validity and signal purity. For example, the percentage of features in the feature vectors that meet a preset validity threshold (e.g., feature values ​​within a reasonable physiological range) is statistically analyzed, and the ratio of noise energy to effective signal energy in the signal is combined to comprehensively determine data quality. For low-quality feature data with a quality score below a preset threshold, Gaussian filtering and wavelet denoising techniques are used to remove signal interference and retain effective feature information. If some modalities are found to be missing in the modal feature vector set (e.g., some cases did not provide pathological slide data due to contraindications), replacement weights are calculated based on the correlation between the missing modalities and other complete modalities (e.g., the statistical correlation between dermoscopic features and pathological features). Using the associated feature data of the complete modalities, supplementary features of the missing modalities are generated according to the replacement weights. Finally, a multimodal feature dataset with complete features and meeting the quality standards is obtained.

[0053] Step S103: After fusing lesion features using a weighted average method on the multimodal feature dataset, the dimensionality-reduced feature vector is obtained by processing it using the principal component analysis algorithm.

[0054] Specifically, for the obtained multimodal feature dataset, lesion features directly related to the diagnosis of pigmentary diseases are first extracted, including asymmetry (symmetry of lesion morphology and pigment distribution), diameter variation (dynamic trend or current size of lesion diameter), and pigment distribution differences (differences in pigment concentration and type within the lesion and with surrounding normal skin). Then, a weighted average method is used to fuse the extracted lesion features. The weights are determined based on the diagnostic contribution of each lesion feature—for example, through statistical analysis of historical clinical case data, if the difference in pigment distribution has the strongest ability to distinguish malignant lesions, it is assigned the highest weight, ensuring that the influence of key features is reflected during the fusion process. Finally, a fused feature vector containing multimodal core lesion information is generated. Since the fused feature vectors may have problems such as excessive feature dimensionality and redundant features (e.g., the diameter features extracted from different modalities have duplicate information), the principal component analysis algorithm is further used to reduce its dimensionality: by calculating the covariance matrix of the feature vectors, the principal components with the highest variance contribution are extracted (usually retaining the principal components whose cumulative variance contribution exceeds a preset threshold), and redundant dimensions and noise interference are removed, finally obtaining a dimensionality-reduced feature vector with reduced dimensionality and high information concentration.

[0055] Step S104: The dimensionality-reduced feature vector is judged by a preset malignant association threshold to obtain a diagnostic classification. Cases with similar risk characteristics are grouped according to the diagnostic classification, and personalized treatment plans are generated by combining the risk characteristics of the groups and the treatment needs.

[0056] For example, a preset malignancy association threshold is first invoked. This threshold needs to be determined through training with a large number of clinically labeled cases. For instance, a threshold is set for the combination of features strongly correlated with malignant lesions in the dimensionality-reduced feature vector. If the calculated result of the dimensionality-reduced feature vector is higher than this threshold, it is judged as "malignant suspected class," and if it is lower than the threshold, it is judged as "benign tendency class," thus obtaining the diagnostic classification result for each case. Based on the diagnostic classification result, a clustering algorithm (such as the K-means algorithm) is used to group the cases: based on the risk features in the dimensionality-reduced feature vector (such as the probability value of malignancy, the rate of disease progression), cases with similar risk features are grouped into the same group, forming disease risk clustering groups of different risk levels (such as high-risk group, medium-risk group, and low-risk group). Finally, personalized treatment plans are generated based on the risk characteristics and treatment needs of each group: for the high-risk group, a plan of surgical intervention + high-frequency follow-up is formulated based on its high probability of malignancy and treatment sensitivity parameters (such as sensitivity to specific targeted drugs); for the low-risk group, a plan of regular observation + basic nursing guidance is formulated based on its low progression rate, to ensure that the plan is accurately matched with the risk level and individual needs of each group of cases.

[0057] The aforementioned multimodal data fusion-based diagnostic method for pigmentary diseases first acquires multimodal data of pigmentary diseases, including dermoscopic images, histopathological sections, spectral analysis data, and other multi-source lesion information. This data is preprocessed to unify resolution and wavelength range to eliminate format interference, resulting in a standardized multimodal dataset. Based on this dataset, a convolutional neural network is used to extract features from each modality and calculate data quality scores. Low-quality data is denoised, and replacement weights are calculated for missing modalities, supplemented by features associated with complete modalities, resulting in a multimodal feature dataset. Weights are then assigned according to the diagnostic contribution of lesion features, and a weighted average is used to fuse features such as asymmetry and diameter variation to generate a fused feature vector. Principal component analysis is then used to reduce the dimensionality of the fused feature vector. Finally, a pre-set malignancy association threshold is used to determine the diagnostic classification, and clustering algorithms are used to group cases with similar risk. Personalized treatment plans are generated based on risk characteristics (such as malignancy probability) and treatment needs (such as treatment sensitivity parameters). This method can improve the accuracy and reliability of diagnosis of pigmentary diseases by reducing the bias of single data through multimodal information complementarity and quality control; it breaks through the limitations of traditional diagnosis that is "unstratified" and realizes risk grouping and personalized diagnosis and treatment, thereby improving the targeted nature of treatment; it reduces the dependence on doctors' experience by relying on standardized and intelligent processes, improves the diagnostic level in primary care and non-specialist scenarios, and alleviates the problem of uneven distribution of medical resources.

[0058] In one embodiment, the process of extracting modal features and calculating data quality scores using a convolutional neural network based on a standardized multimodal dataset, denoising low-quality data, and determining alternative weights for missing modalities to obtain a multimodal feature dataset may include the following steps:

[0059] Step S201: Based on the standardized multimodal dataset, use a convolutional neural network to extract the feature information of each modality to obtain a modal feature vector set.

[0060] Step S202: Calculate the corresponding data quality score for each set of feature data in the modal feature vector set to obtain the quality assessment result.

[0061] Step S203: Based on the quality assessment results, filter feature data with a quality score lower than a preset threshold, denoise the low-quality feature data, and obtain the denoised modal feature dataset.

[0062] Step S204: Judge the modal feature dataset. If the modal features of each modal feature dataset do not cover the preset key feature dimensions and the proportion of missing feature data is higher than the preset threshold, calculate the replacement weights corresponding to the missing modalities to obtain the feature dataset with replacement weights for the missing modalities.

[0063] Preferably, the key feature dimensions include the lesion boundary dimension of the dermoscopy image, the cell structure dimension of the histopathological section, and the feature wavelength dimension of the spectral analysis data.

[0064] Step S205: Based on the feature dataset, supplementary features are generated by using alternative weights to fuse the feature data of the missing modality with the complete modality. The supplementary features are then filled into the missing positions to obtain a complete multimodal feature dataset.

[0065] Specifically, based on the generated standardized multimodal dataset encompassing dermoscopic images (uniform resolution to 512×512 pixels), histopathological sections (uniform resolution to 256×256 pixels), and spectral analysis data (uniform wavelength range to 400-1000nm), features are first extracted using adapted convolutional neural networks (CNNs) for different modalities: For dermoscopic images, a 2D-CNN architecture (containing 3 convolutional layers, kernel sizes 3×3 / 5×5 / 3×3, and 2 max-pooling layers) is used to extract features such as lesion edge gradients, pigment region contours, and texture distribution, outputting a 256-dimensional dermoscopic modality feature vector; for histopathological sections (multi-layer continuous slice data), a 3D-CNN architecture (5×5×5 convolutional kernels) is used to extract microscopic features such as cell cluster distribution, cell nucleus morphology (roundness, area), and intercellular spaces, outputting a 128-dimensional pathological modality feature vector; for spectral analysis data, a 1D-CNN architecture (length 10) is used. The convolution kernel extracts information such as signal intensity and peak distribution corresponding to feature wavelengths (e.g., 550nm, 680nm), outputs 64-dimensional spectral modal feature vectors, and finally integrates them to form a modal feature vector set containing all modal feature information;

[0066] Next, a data quality score is calculated for each set of feature data in the vector set. The calculation logic combines feature validity (the percentage of features whose values ​​fall within a preset physiological range (e.g., lesion diameter 2-20mm)) with signal purity (the ratio of noise signal energy to effective signal energy, i.e., signal-to-noise ratio, calculated using a filter). For example, when the percentage of effective features is ≥80% and the signal-to-noise ratio is ≥30dB, the quality score is considered acceptable. Subsequently, based on the quality assessment results, features with scores below a preset threshold (e.g., the acceptable threshold of 70) are selected. Low-quality feature data (in points) are processed using Gaussian filtering to remove salt-and-pepper noise from dermatoscope / pathology images and wavelet denoising to remove baseline drift noise from spectral data, preserving effective feature information to obtain a denoised modal feature dataset. This dataset is then assessed to determine if it covers preset key feature dimensions (lesion boundary dimension of dermatoscope images, including boundary regularity, boundary blurring, and boundary pigment penetration; cell structure dimension of histopathological sections, including cell atypia, number of mitotic figures, and cell density; characteristic wavelength dimension of spectral analysis data, including characteristic wavelength signal peak value, peak half-width, and signal stability). If any key dimension is not covered and the proportion of missing feature data exceeds a preset threshold (e.g., 20%), then... By analyzing the statistical correlation between missing and complete modalities in historical cases (e.g., the correlation coefficient r=0.72 between dermoscopic boundary features and pathological cell atypia), the substitution weights corresponding to the missing modalities are calculated (e.g., when the pathological modality is missing, the weight of the dermoscopic modality is 0.6 and the weight of the spectral modality is 0.4), resulting in a feature dataset with substitution weights for the missing modalities. Finally, based on this dataset, the associated feature data of the complete modality are fused using the substitution weights (e.g., dermoscopic boundary features and spectral peak features are weighted and summed according to a weight of 0.6 / 0.4), generating supplementary features for the missing modalities. These supplementary features are then filled into the missing data positions, ultimately resulting in a multimodal feature dataset with complete features (no missing key dimensions) and meeting quality standards.

[0067] This embodiment extracts modality-specific features through convolutional neural networks, ensuring that key information in the multimodal data is not lost and providing a comprehensive feature foundation for subsequent analysis. By calculating data quality scores and denoising low-quality data, it effectively removes interfering information from the feature data, improving its purity and reliability. Clearly defining key feature dimensions ensures that the extracted features are directly relevant to the diagnosis of pigmentary diseases, avoiding interference from irrelevant features in subsequent analysis. By calculating missing modality substitution weights and generating supplementary features, it solves the problem of missing modality data for some cases, ensuring the integrity of the multimodal feature dataset and avoiding analytical biases caused by incomplete data. The resulting complete multimodal feature dataset provides high-quality, highly complete input data for subsequent lesion feature fusion, dimensionality reduction, and diagnostic classification, improving the accuracy and stability of pigmentary disease diagnosis from the data source.

[0068] In one embodiment, the data quality score can be calculated using the following formula:

[0069]

[0070] in, This represents the data quality score, with a range of values. , This represents the weighting coefficients, which are preset based on modal characteristics. This represents the number of features in the modal feature vector set that meet the preset feature validity threshold. This represents the total number of features in the modal feature vector set. The energy value of the noise signal in the feature data is represented by a filter and calculated. This represents the energy value of the effective signal in the feature data.

[0071] This embodiment combines two core dimensions—feature validity and signal purity—and introduces weighting coefficients preset according to modal characteristics to achieve a comprehensive and accurate assessment of the quality of multimodal feature data. This avoids the one-sidedness of focusing solely on the number of features while ignoring signal noise interference. Furthermore, the flexible setting of α adapts to the characteristic differences of different modalities (such as dermoscopy images and spectral analysis data). For example, when spectral data is more sensitive to noise, the weight of the signal purity dimension can be increased, ensuring that the assessment results match the actual quality characteristics of each modality. The data quality score obtained based on this formula provides a clear and reliable basis for subsequent low-quality data denoising and missing modality supplementation, effectively screening and optimizing data, reducing multimodal feature dataset quality problems caused by invalid features or noise interference. This provides high-quality data input for subsequent lesion feature fusion, dimensionality reduction feature vector generation, and diagnostic classification, ensuring the accuracy and stability of each stage in the diagnosis of pigmentary diseases from the data source and reducing diagnostic errors caused by data quality deviations.

[0072] In one embodiment, after fusing lesion features using a weighted average method on the multimodal feature dataset, the dimensionality-reduced feature vector is obtained by processing it using a principal component analysis algorithm, which may include the following steps:

[0073] Step S301: Extract lesion features related to pigmentary diseases from the multimodal feature dataset to obtain a lesion feature set; lesion features include asymmetry, diameter variation, and pigment distribution differences.

[0074] Step S302: A preset weighted average method is used to fuse all lesion features in the lesion feature set to obtain a fused feature vector; the weights of the weighted average method are set according to the diagnostic contribution of each lesion feature.

[0075] Step S303: Extract lesion attributes that have key distinguishing properties for disease diagnosis from the fused feature vector to generate a lesion attribute set; lesion attributes include malignancy risk association attributes and lesion progression characteristic attributes.

[0076] Step S304: Based on the lesion attribute set, a standardization algorithm is used to uniformly process the numerical range of each attribute to obtain a standardized feature vector.

[0077] Step S305: Apply principal component analysis algorithm to the standardized feature vector to reduce dimensionality, retain the main feature information and remove redundant dimensions to obtain the dimensionality-reduced feature vector.

[0078] Specifically, lesion features directly related to the diagnosis of pigmented diseases are screened and extracted from a multimodal feature dataset to form a lesion feature set. This set includes asymmetry (symmetry of lesion morphology and pigment distribution), diameter variation (current diameter of the lesion and its variation over a certain period), and pigment distribution differences (differences in pigment concentration in different areas within the lesion and differences in pigment contrast with surrounding normal skin). Based on this lesion feature set, a pre-defined weighted average method is used for feature fusion calculation. During the fusion process, statistical analysis of historical clinical case data is first used to determine the contribution of each lesion feature to the diagnostic results (e.g., differences in pigment distribution have a stronger ability to distinguish malignant lesions, and their diagnostic contribution is higher). Each lesion feature is assigned a weight based on its contribution (higher than other features), and then all lesion features are weighted and summed to obtain a fused feature vector. From the fused feature vector, lesion attributes with key distinguishing significance for judging the benignity or malignancy of the disease are further screened to generate a lesion attribute set. This set specifically includes malignancy risk-related attributes (such as pigment distribution patterns and boundary morphology features highly correlated with malignant lesions) and lesion progression characteristic attributes (such as the rate of change of lesion diameter and the rate of pigment diffusion). For the differences in the numerical range of each attribute in the lesion attribute set (such as diameter change in millimeters and asymmetry in percentage), standardization algorithms (such as Z-score standardization and Min-Max standardization) are used to process the numerical values ​​of each attribute and uniformly map them to the same numerical range to obtain a standardized feature vector. Finally, principal component analysis algorithm is applied to the standardized feature vector. By calculating the covariance matrix, extracting eigenvalues ​​and eigenvectors, the principal components with larger eigenvalues ​​(usually with a cumulative variance contribution of ≥85%) are retained, and redundant and highly correlated dimensions are removed to finally obtain a dimensionality-reduced feature vector.

[0079] This embodiment extracts lesion features directly related to pigmentary diseases to avoid interference from irrelevant features in subsequent analysis, ensuring the relevance of the input data. It employs a weighted average fusion method based on diagnostic contribution, giving higher-value features greater weight in the fusion process, highlighting the impact of key features on diagnostic results, and improving the effectiveness of the fused feature vector. It extracts lesion attributes with key discriminative power to further focus on core diagnostic information and reduce information redundancy caused by unnecessary attributes. Standardization eliminates dimensionality interference caused by differences in the numerical ranges of various attributes, ensuring that different attributes have equal weight and reference value in subsequent analysis. Principal component analysis is used for dimensionality reduction, simplifying data dimensions while retaining key feature information, reducing the computational complexity of subsequent diagnostic classification, and avoiding the risk of model overfitting caused by the curse of dimensionality. This provides high-quality, low-redundancy input data for subsequent diagnostic classification based on dimensionality-reduced feature vectors, effectively improving the accuracy and efficiency of pigmentary disease diagnosis.

[0080] In one embodiment, a diagnostic classification is obtained by judging the dimensionality-reduced feature vector through a preset malignant association threshold. Based on the diagnostic classification, cases with similar risk characteristics are grouped. A personalized treatment plan is generated by combining the risk characteristics of the groups and the treatment needs. This may include the following steps:

[0081] Step S401: Extract feature dimension data and feature value distribution from the dimensionality-reduced feature vector, call the preset malignant association threshold to perform diagnostic classification judgment, and obtain the diagnostic classification result.

[0082] Step S402: Calculate the matching deviation correction coefficient based on the diagnostic classification results, and obtain the initial calibration parameters through parameter mapping transformation.

[0083] Step S403: Based on the initial calibration parameters, an iterative optimization algorithm is used to calibrate the dimensionality-reduced feature vector to obtain the calibrated feature vector.

[0084] Step S404: Analyze the quantitative indicators related to the degree of malignancy in the calibrated feature vector to determine the malignancy assessment result of pigmentary diseases; the quantitative indicators include the malignancy probability value and the disease progression rate.

[0085] Step S405: Extract risk probability distribution data for different disease stages from the malignancy assessment results, and construct a risk assessment matrix according to preset dimensions.

[0086] Step S406: Based on the risk assessment matrix, a clustering algorithm is applied to group cases with similar risk characteristics to obtain disease risk cluster groups.

[0087] Step S407: Combine the risk characteristics and treatment needs of each group in the disease risk clustering to generate a personalized treatment plan; the treatment needs include treatment sensitivity parameters and follow-up period.

[0088] Specifically, the process involves extracting specific data and distribution patterns of feature values ​​for each feature dimension from the dimensionality-reduced feature vector (e.g., the proportion of feature values ​​in different intervals and peak positions). A pre-defined malignancy correlation threshold (determined based on the correlation analysis of feature distribution and diagnostic results from a large number of labeled cases; for example, a threshold of 0.65 for a certain feature combination indicates a malignant tendency) is used for diagnostic classification, outputting results such as "benign" or "malignant suspected." Based on this classification, historical case statistics for the corresponding category are retrieved, and a matching deviation correction coefficient (integrating historical average deviation, current classification confidence, and feature value deviation rate) is calculated. This coefficient is then converted into initial calibration parameters using pre-defined parameter mapping rules (e.g., linear transformation, exponential mapping). Using these initial calibration parameters as a benchmark, an iterative optimization algorithm (e.g., gradient descent) is employed to adjust the parameters of the dimensionality-reduced feature vector. Through multiple iterations, the deviation between the feature values ​​and the diagnostic standard values ​​is reduced, resulting in a calibrated feature vector. Finally, the quantitative indicators directly related to the degree of malignancy in the calibrated feature vector are analyzed, including the malignancy probability value (e.g., 0). Values ​​in the -1 range (0.8 indicates a high probability of malignancy) and lesion progression rate (e.g., monthly diameter increase in millimeters) are used to determine the malignancy assessment result of pigmentary diseases. Risk probability distribution data for different disease stages (e.g., early stage, progressive stage, late stage) are extracted from the malignancy assessment results (e.g., early malignancy probability distribution, progressive stage spread risk distribution), and a risk assessment matrix is ​​constructed according to preset dimensions (e.g., risk level, progression rate). Based on the risk assessment matrix, clustering algorithms (e.g., K-means clustering) are applied to group cases, grouping cases with similar risk characteristics (e.g., malignancy probability value range, progression rate range) into the same group, resulting in disease risk clustering groups (e.g., high-risk rapid progression group, medium-risk stable group, low-risk slow progression group). Combining the risk characteristics of each group (e.g., high malignancy probability and rapid progression rate in the high-risk group) and treatment needs (e.g., treatment sensitivity parameters: responsiveness to specific laser treatments; follow-up period: once a month for the high-risk group, once every 3 months for the low-risk group), a personalized treatment plan is generated, covering treatment selection, intervention timing, and follow-up frequency.

[0089] This embodiment effectively reduces the impact of eigenvalue bias on diagnostic results by using dimensionality-reduced feature vector-based diagnostic classification, combined with bias correction and iterative calibration, thereby improving the accuracy and stability of diagnostic classification. By analyzing malignancy association quantification indicators to determine malignancy assessment results, the judgment of disease malignancy shifts from qualitative to quantitative, providing a precise basis for risk stratification. Clustering based on the risk assessment matrix enables refined differentiation of case risk characteristics, breaking away from the traditional "one-size-fits-all" grouping model and ensuring a high degree of match between grouping results and actual case risk levels. Personalized plans are generated by combining grouped risk characteristics with treatment needs, ensuring that treatment strategies and follow-up arrangements are precisely adapted to individual patient risks and needs, improving treatment effectiveness and the rationality of follow-up. This not only improves the accuracy of pigmentary disease diagnosis but also provides directly applicable personalized treatment guidance for clinicians, contributing to precision medicine while reducing the waste of medical resources caused by incompatible treatment plans.

[0090] In one embodiment, the deviation correction factor can be calculated using the following formula:

[0091]

[0092] in, The deviation correction factor is represented, and its value range is [not specified]. , This represents the average deviation coefficient of historical cases corresponding to the current diagnostic category. This indicates the confidence level of the current diagnostic classification result. This represents the mean deviation rate between the actual values ​​of the core lesion features in the current dimensionality-reduced feature vector and the feature values ​​of the diagnostic classification criteria. This represents the bias sensitivity coefficient.

[0093] This embodiment's deviation correction coefficient calculation formula achieves precise correction of the deviation of the dimensionality-reduced feature vector by integrating historical patterns with current diagnostic information and dynamically adjusting the influence of deviation. On the one hand, it introduces the average deviation coefficient of historical cases corresponding to the current diagnostic classification, anchoring the inherent deviation pattern of this type of diagnosis and ensuring that the correction direction conforms to clinical historical experience. At the same time, it combines the confidence level of the current diagnostic classification result, so that the higher the confidence level, the greater the weight of the historical average deviation coefficient, avoiding correction deviation caused by the uncertainty of the current diagnosis and balancing the reliability of historical data and the specificity of the current diagnosis. On the other hand, it quantifies the difference between the current feature and the standard value by the mean deviation rate of the core lesion features, and achieves dynamic adjustment of "the larger the deviation, the smaller the influence of the non-confidence correlation term on the correction coefficient" by combining the exponential term of the deviation sensitivity coefficient. This effectively suppresses over-correction caused by abnormal deviation of the current feature (such as data noise), and the deviation correction coefficient is limited to a range of 0.8-1.2, further ensuring that the correction range is reasonable and controllable, avoiding over-correction or under-correction. Overall, this formula can provide a precise and stable basis for the parameter calibration of dimensionality-reduced feature vectors, reduce the interference of feature bias on subsequent malignancy assessment, risk grouping and treatment plan generation, and improve the accuracy of diagnosis and treatment plan formulation for pigmentary diseases from the data calibration stage.

[0094] In one embodiment, such as Figure 2 As shown, this application also provides a diagnostic device for pigmentary diseases based on multimodal data fusion, the device may include:

[0095] The multimodal data processing module 501 is used to acquire multimodal data of pigmentary diseases, and to perform resolution and wavelength range unification processing on the multimodal data to obtain a standardized multimodal dataset.

[0096] The feature extraction optimization module 502 is used to extract modal features and calculate data quality scores based on a standardized multimodal dataset using a convolutional neural network, denoise low-quality data and determine alternative weights for missing modalities to obtain a multimodal feature dataset.

[0097] The lesion feature fusion module 503 is used to fuse lesion features from a multimodal feature dataset using a weighted average method, and then process the data using a principal component analysis algorithm to obtain a dimensionality-reduced feature vector.

[0098] The diagnostic plan generation module 504 is used to judge the dimensionality reduction feature vector through a preset malignant association threshold to obtain a diagnostic classification, group cases with similar risk characteristics according to the diagnostic classification, and generate personalized diagnosis and treatment plans by combining the risk characteristics of the groups and the diagnosis and treatment needs.

[0099] The aforementioned multimodal data fusion diagnostic device for pigmentary diseases first acquires multimodal data of pigmentary diseases (covering dermoscopic images, histopathological sections, spectral analysis data, and other lesion-related multi-source information) through its multimodal data processing module. Then, it performs preprocessing to address the format differences of different modalities: for image data such as dermoscopic images and histopathological sections, an interpolation algorithm is used to unify the resolution, ensuring that the pixel size matches the image ratio; for spectral analysis data, wavelength calibration technology is used to unify the wavelength measurement range, eliminating wavelength deviations caused by equipment differences. Finally, a standardized multimodal dataset with a unified format, directly usable for subsequent analysis, is output. This dataset serves as the input to the feature extraction and optimization module. The feature extraction optimization module is based on a standardized multimodal dataset and uses a convolutional neural network (CNN) adapted to different modalities to extract features: for image modalities, features such as edges, textures, and cell morphology are extracted through convolutional layers and pooling layers; for spectral modalities, feature wavelength signal intensity and peak distribution are extracted through 1D-CNN. At the same time, the data quality score of each set of feature data is calculated (combining the feature validity ratio and signal-to-noise ratio); for low-quality data with scores below a preset threshold, Gaussian filtering, wavelet denoising and other techniques are used to remove interference; if a modality is missing (such as some cases without pathological data), replacement weights are calculated based on the correlation between the missing modality and the complete modality, and supplementary data is generated using the correlation features of the complete modality, and finally, a multimodal feature dataset is output. The lesion feature fusion module takes a multimodal feature dataset as input, first extracts lesion features related to pigmentary diseases such as asymmetry, diameter variation, and pigment distribution differences, then sets weights according to the diagnostic contribution of each feature (determined through historical case statistics), and uses a weighted average method to fuse features and generate a fused feature vector. Subsequently, a standardization algorithm (such as Z-score, Min-Max) is used to unify the numerical range of each feature and eliminate dimensional interference. Finally, the principal component analysis (PCA) algorithm is applied to reduce the dimensionality of the standardized fused feature vector, retaining principal components with a cumulative variance contribution of ≥85% to remove redundant dimensions, and outputting a dimensionality-reduced feature vector. After receiving the dimensionality-reduced feature vector, the diagnostic plan generation module calls a preset malignancy association threshold (determined based on labeled case training) to determine the diagnostic classification (e.g., benign, suspected malignancy). Based on the diagnostic classification results, and combined with quantitative indicators such as the malignancy probability value and disease progression rate in the dimensionality-reduced feature vector, a clustering algorithm (e.g., K-means) is applied to group cases with similar risk characteristics. Finally, based on the risk characteristics of each group (e.g., high malignancy probability and rapid progression rate for high-risk groups) and treatment needs (e.g., treatment sensitivity parameters: responsiveness to laser treatment; follow-up period: once a month for high-risk groups and once every 3 months for low-risk groups), a personalized treatment plan adapted to different patients is generated.

[0100] This device ensures data quality from the source through the coordinated operation of four modules. The multimodal data processing module eliminates interference from format differences through standardized processing, while the feature extraction optimization module improves the purity and completeness of feature data through quality assessment and missing data completion mechanisms, providing high-quality input for subsequent analysis. It also enhances the effectiveness of features and diagnosis. The lesion feature fusion module weights and fuses key lesion features according to their diagnostic contribution, and combines PCA dimensionality reduction to simplify data dimensions. This highlights the impact of core features on diagnosis while avoiding model bias caused by the curse of dimensionality, ensuring the diagnostic discriminative power of the reduced feature vectors. Finally, it enables precise diagnosis and personalized treatment. The diagnostic plan generation module, based on quantitative indicators and clustering, breaks through the limitations of traditional "uniform conclusions." The generated personalized plans can match the treatment and follow-up needs of patients with different risk levels. Simultaneously, standardized processes and intelligent processing reduce the reliance on individual physician experience in diagnosis, improving the diagnostic level of primary healthcare institutions and non-specialist scenarios, effectively alleviating the disparities in treatment capabilities caused by uneven distribution of medical resources, and facilitating the implementation of precision medicine in clinical practice.

[0101] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0102] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the multimodal data fusion diagnostic method, apparatus, device, and medium for pigmentary diseases as described above.

[0103] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0104] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0105] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A diagnostic method for pigmentary disorders based on multimodal data fusion, characterized in that, The method includes: Multimodal data of pigmentary diseases are acquired, and the resolution and wavelength range of the multimodal data are uniformly processed to obtain a standardized multimodal dataset. Based on the standardized multimodal dataset, a convolutional neural network is used to extract modal features and calculate data quality scores. Low-quality data is denoised and replacement weights for missing modalities are determined to obtain a multimodal feature dataset. After fusing lesion features using a weighted average method on the multimodal feature dataset, a dimensionality-reduced feature vector is obtained by processing it using principal component analysis algorithm; The dimensionality-reduced feature vector is judged by a preset malignant association threshold to obtain a diagnostic classification. Cases with similar risk characteristics are grouped according to the diagnostic classification, and personalized treatment plans are generated by combining the risk characteristics of the groups and the treatment needs.

2. The method according to claim 1, characterized in that, The process involves extracting modal features and calculating data quality scores using a convolutional neural network based on the standardized multimodal dataset, denoising low-quality data, and determining alternative weights for missing modalities to obtain a multimodal feature dataset, including: Based on the standardized multimodal dataset, convolutional neural networks are used to extract feature information of each modality to obtain a modality feature vector set; For each set of feature data in the modal feature vector set, calculate the corresponding data quality score to obtain the quality assessment result; Based on the quality assessment results, the feature data with a quality score lower than a preset threshold are filtered out, and the low-quality feature data is denoised to obtain a denoised modal feature dataset. The modal feature dataset is judged. If each modal feature in the modal feature dataset does not cover the preset key feature dimension and the proportion of missing feature data is higher than the preset threshold, the substitution weight corresponding to the missing modality is calculated to obtain the feature dataset with the substitution weight of the missing modality. The key feature dimensions include the lesion boundary dimension of dermoscopy images, the cell structure dimension of histopathological sections, and the characteristic wavelength dimension of spectral analysis data. Based on the feature dataset, the alternative weights are used to generate supplementary features by fusing the feature data associated with the missing modality and the complete modality. The supplementary features are then filled into the missing positions to obtain a complete multimodal feature dataset.

3. The method according to claim 2, characterized in that, The data quality score is calculated using the following formula: in, This represents the data quality score, with a range of values. , This represents the weighting coefficients, which are preset based on modal characteristics. This represents the number of features in the modal feature vector set that meet the preset feature validity threshold. This represents the total number of features in the modal feature vector set. The energy value of the noise signal in the feature data is represented by a filter and calculated. This represents the energy value of the effective signal in the feature data.

4. The method according to claim 1, characterized in that, After fusing lesion features using a weighted average method on the multimodal feature dataset, the dimensionality-reduced feature vector is obtained by processing it using principal component analysis algorithm, including: Lesion features related to pigmentary diseases are extracted from the multimodal feature dataset to obtain a lesion feature set; the lesion features include asymmetry, diameter variation, and pigment distribution differences; A preset weighted average method is used to fuse all lesion features in the lesion feature set to obtain a fused feature vector; the weights of the weighted average method are set according to the diagnostic contribution of each lesion feature. Extract disease attributes that have key discriminative power for disease diagnosis from the fused feature vector to generate a disease attribute set; the disease attributes include malignancy risk association attributes and disease progression characteristic attributes; Based on the aforementioned lesion attribute set, a standardization algorithm is used to uniformly process the numerical range of each attribute to obtain a standardized feature vector. The standardized feature vector is subjected to dimensionality reduction by applying principal component analysis algorithm, which retains the main feature information and removes redundant dimensions to obtain the dimensionality-reduced feature vector.

5. The method according to claim 1, characterized in that, The process involves determining the diagnostic classification by judging the dimensionality-reduced feature vector through a preset malignant association threshold, grouping cases with similar risk characteristics according to the diagnostic classification, and generating personalized treatment plans by combining the risk characteristics of the groups and treatment needs. The feature dimension data and feature value distribution are extracted from the reduced feature vector, and a preset malignant association threshold is used to perform diagnostic classification judgment to obtain the diagnostic classification result. Based on the diagnostic classification results, the matching deviation correction coefficient is calculated, and the initial calibration parameters are obtained through parameter mapping transformation; Based on the initial calibration parameters, an iterative optimization algorithm is used to calibrate the reduced feature vector to obtain the calibrated feature vector. The malignancy assessment results of pigmentary diseases are determined by analyzing the quantitative indicators related to the degree of malignancy in the calibrated feature vector; the quantitative indicators include the probability value of malignancy and the rate of disease progression. Risk probability distribution data for different disease stages are extracted from the malignancy assessment results, and a risk assessment matrix is ​​constructed according to preset dimensions. Based on the risk assessment matrix, a clustering algorithm is applied to group cases with similar risk characteristics to obtain disease risk clustering groups; Personalized treatment plans are generated by combining the risk characteristics and treatment needs of each group in the disease risk clustering group; the treatment needs include treatment sensitivity parameters and follow-up period.

6. The method according to claim 5, characterized in that, The deviation correction factor is calculated using the following formula: in, The deviation correction factor is represented, and its value range is [not specified]. , This represents the average deviation coefficient of historical cases corresponding to the current diagnostic category. This indicates the confidence level of the current diagnostic classification result. This represents the mean deviation rate between the actual values ​​of the core lesion features in the current dimensionality-reduced feature vector and the feature values ​​of the diagnostic classification criteria. This represents the bias sensitivity coefficient.

7. A diagnostic device for pigmentary disorders based on multimodal data fusion, characterized in that, The device includes: A multimodal data processing module is used to acquire multimodal data of pigmentary diseases, and to perform resolution and wavelength range unification processing on the multimodal data to obtain a standardized multimodal dataset; The feature extraction optimization module is used to extract modal features and calculate data quality scores based on the standardized multimodal dataset using a convolutional neural network, denoise low-quality data and determine the alternative weights for missing modalities to obtain a multimodal feature dataset. The lesion feature fusion module is used to fuse lesion features in the multimodal feature dataset using a weighted average method, and then process the data using a principal component analysis algorithm to obtain a dimensionality-reduced feature vector. The diagnostic scheme generation module is used to determine the diagnostic classification by judging the dimensionality reduction feature vector through a preset malignant association threshold, group cases with similar risk characteristics according to the diagnostic classification, and generate personalized diagnosis and treatment schemes by combining the risk characteristics of the groups and the diagnosis and treatment needs.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images