Acne grading diagnosis auxiliary system and method based on multispectral image

Abstract

The invention discloses an acne grading diagnosis auxiliary system and method based on a multispectral image, and the system comprises a data obtaining module which is used for obtaining multispectral imaging data and user feature data of a to-be-diagnosed object; the data processing module is used for acquiring feature vectors and feature weights of different wavebands based on the multispectral imaging data, and performing weighted fusion on the feature vectors based on the feature weights to generate fusion vectors; the model optimization module is used for determining a model weight according to the user feature data and establishing a target diagnosis model according to the model weight; and the output module is used for importing the fusion vector into a target diagnosis model and determining a grading diagnosis result according to an output parameter of the target diagnosis model. According to the method, dynamic fusion of multispectral imaging data is realized by introducing a dynamic updating mechanism of feature weights, dynamic fusion of a plurality of classification models is realized by introducing a dynamic updating mechanism of model weights, and the accuracy and reliability of acne classification diagnosis results can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent medical treatment, and in particular to a multi-spectral image-based acne grading diagnosis auxiliary system and method. BACKGROUND

[0002] Acne is a chronic and inflammatory skin problem that occurs in the pilosebaceous follicle, mainly occurring in the facial, chest, back and other areas where sebum is excreted. The main cause of acne is the excessive secretion of sebum by the sebaceous gland, which leads to abnormal keratinization of the follicular orifice, causing a closed environment, breeding anaerobic bacteria, and then inducing inflammation, resulting in damaged skin barrier. The main features of acne lesions include comedones, papules, pustules, nodules and cysts.

[0003] In the prior art, acne evaluation and diagnosis methods mainly include visual assessment and image recording. The existing visual assessment usually adopts the Pillsbury grading method, which divides acne into four levels according to the severity of the lesions: mild (grade I) only has comedones, moderate (grade II) has inflammatory papules, moderate (grade III) appears pustules, severe (grade IV) has nodules and cysts, and the observer gives a typing diagnosis result based on visual observation, which has the following problems: the current grading method lacks specific and quantifiable grading standards, and requires a high level of experience for the observer, is limited by the differences in cognitive levels of different observers, and the diagnosis results given by different observers differ greatly, resulting in poor diagnosis reliability. The existing image recording method records acne changes through facial photography, which has the following problems: the images used for evaluation are easily affected by lighting conditions and patient skin color, and the post-analysis still relies on manual implementation, resulting in low accuracy of the diagnosis results and affecting the subsequent diagnosis and treatment effect. SUMMARY

[0004] The present application provides a multi-spectral image-based acne grading diagnosis auxiliary system and method to solve the problem of existing acne evaluation and diagnosis methods relying on manual implementation, lacking specific and quantifiable grading standards, poor diagnosis reliability, and images used for evaluation being easily affected by environmental and skin color factors, resulting in low diagnosis accuracy.

[0005] According to an aspect of the present application, a multi-spectral image-based acne grading diagnosis auxiliary system is provided, comprising: a data acquisition module for acquiring multi-spectral imaging data and user feature data of a subject to be diagnosed; the user feature data includes at least one of the following: skin color feature, age feature and gender feature; a data processing module for acquiring feature vectors and feature weights of different wavebands based on the multi-spectral imaging data, and weighting and fusing the feature vectors based on the feature weights to generate a fusion vector; a model optimization module for determining a model weight according to the user feature data, and establishing a target diagnosis model according to the model weight; wherein the target diagnosis model comprises a plurality of classification models, and the classification models correspond one-to-one to the model weight; an output module connected with the data processing module and the model optimization module respectively, for importing the fusion vector into the target diagnosis model, and determining a grading diagnosis result according to an output parameter of the target diagnosis model.

[0006] Optionally, the classification model at least includes: a non-inflammatory morphology classification model, an inflammation activity classification model, a lesion depth classification model and a sebum secretion degree classification model; the model optimization module is configured to: weight and fuse the non-inflammatory morphology classification model, the inflammation activity classification model, the lesion depth classification model and the sebum secretion degree classification model based on the model weight, to establish the target diagnosis model.

[0007] Optionally, the output parameter S of the target diagnosis model satisfies: ; wherein, , , and represents the weight value of the model weight corresponding to the classification model, ; represents a quantitative evaluation result generated based on non-inflammatory morphology features; represents a quantitative evaluation result generated based on inflammation activity; represents a quantitative evaluation result generated based on lesion depth; represents a quantitative evaluation result generated based on sebum secretion degree.

[0008] Optionally, the non-inflammatory morphology classification model sets at least two levels of acne area proportion quantitative indicators; the inflammation activity classification model sets at least four levels of erythema quantitative indicators and at least four levels of inflammatory lesion area indicators; the lesion depth classification model sets at least three levels of lesion depth quantitative indicators and at least three levels of lesion volume quantitative indicators; and the sebum secretion degree classification model sets at least three levels of fluorescence area quantitative indicators.

[0009] Optionally, the data acquisition module includes a multispectral imaging module with a built-in standard color chart and a fluorescence reference unit; the standard color chart is used to perform color restoration and white balance processing on the multispectral imaging data; the fluorescence reference unit is used to calibrate the fluorescence spectral intensity of the multispectral imaging data.

[0010] Optionally, the multispectral imaging module further includes: a multispectral light source and an image acquisition unit; wherein, the multispectral light source has the following wavelengths: visible light, green or yellow light, ultraviolet or blue light, red light, and near-infrared light; the image acquisition unit is used to acquire imaging data of the object to be diagnosed under different light source conditions.

[0011] Optionally, the data processing module includes: a preprocessing unit, used to perform registration and noise reduction enhancement processing on the multispectral imaging data, and to perform image segmentation on the processed data using a segmentation model to obtain image segmentation sub-regions; the image segmentation sub-regions include: acne-dominant region, inflammation-dominant region, and nodule-dominant region; a feature extraction unit, used to extract acne-related features from the image segmentation sub-regions, and to establish the feature vector based on the extracted acne-related features; and a feature fusion unit, used to perform preliminary classification based on the image segmentation sub-regions, determine the feature weights based on the preliminary classification results, and perform weighted fusion of the feature vectors based on the feature weights.

[0012] Optionally, the feature vector includes at least: a visible light band feature vector, a green or yellow light band feature vector, an ultraviolet or blue light band feature vector, and a near-infrared band feature vector.

[0013] Optionally, the output module further includes: a visualization unit; the visualization unit is used to visualize the fusion vector and the hierarchical diagnosis result; wherein the visualization of the fusion vector uses a visualization image with at least two colors.

[0014] According to another aspect of the present invention, a method for assisting in the graded diagnosis of acne based on multispectral imaging is provided, comprising: acquiring multispectral imaging data and user feature data of a subject to be diagnosed; the user feature data including at least one of the following: skin color feature, age feature, and gender feature; acquiring feature vectors and feature weights of different bands based on the multispectral imaging data, and performing weighted fusion of the feature vectors based on the feature weights to generate a fusion vector; determining model weights according to the user feature data, and establishing a target diagnostic model according to the model weights; wherein the target diagnostic model includes multiple classification models, and the classification models correspond one-to-one with the model weights; importing the fusion vector into the target diagnostic model, and determining the graded diagnosis result according to the output parameters of the target diagnostic model.

[0015] Based on the above embodiments, the technical solution of the present invention has the following technical effects:

[0016] Firstly, by combining multispectral images to dynamically update feature weights, and then using the dynamically updated feature weights to perform weighted fusion of feature vectors in different bands, key information in different bands can be highlighted in different scenarios. Furthermore, the imaging data of different bands can compensate for each other, solving the problem that sampled images are easily affected by environmental factors and improving the robustness of the system.

[0017] Secondly, by introducing a classification model to quantify and analyze the feature vectors of different bands, and by introducing user feature data to dynamically update the model weights, and by using the dynamically updated model weights to dynamically fuse multiple classification models, it is beneficial to improve the accuracy and reliability of acne grading diagnosis results. This solves the problems of existing acne assessment and diagnosis methods relying on manual implementation, lacking specific and quantifiable grading standards, having poor diagnostic reliability, and having low diagnostic accuracy due to the images used for assessment being easily affected by skin color factors.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the structure of an acne grading diagnosis auxiliary system based on multispectral imaging, provided in an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of the structure of a data acquisition module provided in an embodiment of the present invention;

[0022] Figure 3 This is a schematic diagram of the structure of an output module provided in an embodiment of the present invention;

[0023] Figure 4 A flowchart of an acne grading diagnosis auxiliary method based on multispectral imaging provided in an embodiment of the present invention;

[0024] Figure 5 A schematic diagram of the electronic device used to implement the acne grading diagnosis auxiliary method based on multispectral imaging, according to an embodiment of the present invention. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] Figure 1 This is a schematic diagram of a multispectral imaging-based acne grading diagnostic auxiliary system provided in an embodiment of the present invention, suitable for applications involving automatic classification of acne severity. Figure 1 As shown, the acne grading diagnosis auxiliary system based on multispectral imaging of the present invention specifically includes: a data acquisition module 100, a data processing module 200, a model optimization module 300, and an output module 400.

[0028] The data acquisition module 100 of this invention is used to acquire multispectral imaging data and user characteristic data of a subject to be diagnosed. The subject to be diagnosed can be understood as a user who has or may have acne. Multispectral imaging data can be understood as images of acne areas (including but not limited to the face, back, and chest) acquired using a multispectral camera. Specifically, by capturing the reflection, absorption, and fluorescence characteristics of the skin in multiple specific wavelengths (from ultraviolet to near-infrared), multispectral imaging data can extract pathological information from the skin surface and subepidermal layers, achieving accurate identification and quantification of different acne types. Typically, multispectral imaging data includes, but is not limited to, at least one of the following: visible light band imaging data, green or yellow light band imaging data, near-infrared band imaging data, and ultraviolet or blue light band imaging data. Visible light imaging data is used to monitor the surface morphology of acne (such as the color and size of papules and pustules); in the green or yellow light band, oxyhemoglobin and deoxyhemoglobin have strong absorption peaks, and green or yellow light imaging data can be used to quantify the degree of inflammation; the near-infrared band can penetrate the skin surface (1-3 mm), and light penetrates deeper into tissues, so near-infrared imaging data can be used to assess structural changes and the depth of inflammation in the dermis; under ultraviolet or blue light excitation, porphyrins produced by Propionibacterium acnes emit a characteristic orange-red fluorescence, while clogged hair follicles and excess sebum may also have specific fluorescent signals, and ultraviolet or blue light imaging data is used to detect the degree of hair follicle blockage.

[0029] User characteristic data can be understood as data representing a user's inherent physiological or social attributes. This user characteristic data is used to adjust the dependence of the grading diagnostic model on different pathological information. Typically, user characteristic data includes at least one of the following: skin color characteristics, age characteristics, and gender characteristics. Among them, skin color characteristics directly affect the interaction between light and skin. Its core role in acne diagnosis is to adjust the dependence of the grading diagnostic model on parameters such as hemoglobin measurement and erythema index based on the influence of skin color on parameters such as hemoglobin measurement and erythema index. The core roles of age and gender characteristics in acne diagnosis are to adjust the dependence of the grading diagnostic model on parameters such as skin lesion type and sebum secretion based on the attention paid to different symptoms (such as skin lesion type) and causes (such as hormone levels) by age and gender. For example, adolescent acne is strongly correlated with androgen-driven sebum secretion, while acne in adult women may be related to hormone fluctuations and stress.

[0030] The data processing module 200 of this invention is used to acquire feature vectors and feature weights for different bands based on multispectral imaging data, and to perform weighted fusion of the feature vectors based on the feature weights to generate a fused vector. The feature vector can be understood as a set of information established based on acne-related features extracted from imaging data of a specific band. Typically, acne-related features include at least one of the following: non-inflammatory morphological features (including but not limited to at least one of the following: number of comedones, comedone area, comedone density), inflammatory activity features (including but not limited to at least one of the following: erythema index, area of ​​inflammatory region, intensity of inflammation), lesion depth features (including but not limited to at least one of the following: lesion depth, lesion volume), and sebum secretion features (including but not limited to: area of ​​sebaceous region). In some optional embodiments, the feature vector includes at least: a visible light band feature vector, a green or yellow light band feature vector, an ultraviolet or blue light band feature vector, and a near-infrared band feature vector. The visible light band feature vectors include, but are not limited to, comedo area and number, primarily used to assess skin surface morphology; the green or yellow light band feature vectors include, but are not limited to, the following information: mean erythema index, erythema index standard deviation, and inflamed area, primarily used to assess inflammatory activity; the ultraviolet or blue light band feature vectors include, but are not limited to, the following information: total number of fluorescent spots, total fluorescent spot intensity, and average fluorescent spot area, primarily used to assess follicular blockage and sebum secretion; the near-infrared band feature vectors include, but are not limited to, the following information: area of ​​deep lesions, average gray level of deep lesions, and texture roughness, primarily used to assess lesion depth. Feature weights are used to adjust the attention given to feature vectors of different bands in different scenarios (such as preliminary classification levels). Weighted fusion of feature vectors of different bands based on dynamically updated feature weights can highlight key information of different bands in different scenarios.

[0031] The model optimization module 300 of this invention is used to determine model weights based on user feature data and to establish a target diagnostic model based on these model weights. The target diagnostic model includes multiple classification models, with each classification model corresponding to a specific model weight. Model weights can be understood as the weight coefficients of the output layer or attention layer of the target diagnostic model. In this embodiment, each classification model adopts an independent neural network model architecture. During the model training phase, it is trained using a set of acne feature vectors with specific skin color, age, and gender characteristics, along with their corresponding acne grading labels. Specifically, a model weight database is established to address the dependence of acne diagnosis on pathological information in groups with different skin color, age, and gender characteristics. For example, because skin melanin can mask erythema and absorb fluorescence, for patients with dark skin, it is necessary to reduce the diagnostic model's dependence on pathological information such as erythema index and inflammatory area. For adolescent patients, it is necessary to increase the diagnostic model's dependence on sebum secretion; for adult women, it is appropriate to reduce the diagnostic model's dependence on sebum secretion while increasing the diagnostic model's dependence on comedone area. During the model application phase, the user characteristic data of the subject to be diagnosed is compared with skin color, age, and gender in the model weight database. The model weights of the data sets with consistent skin color, age, and gender are determined as the final model weights. Furthermore, the model weights are used to adjust the proportion of the classification model in the evaluation system of the target diagnostic model.

[0032] The output module 400 of this invention is connected to the data processing module 200 and the model optimization module 300, respectively, and is used to import the fused vector into the target diagnostic model and determine the graded diagnostic result based on the output parameters of the target diagnostic model. In this embodiment, the graded diagnostic result can be represented by a diagnostic score.

[0033] Specifically, the fusion vector is imported into the target diagnostic model as an input parameter. Each classification model calculates a classification score based on the pathological information (such as erythema index, fluorescence area, sebum secretion, and comedo area) in the fusion vector. Then, the classification scores given by different classification models are weighted and summed to obtain the final diagnostic score, which reflects the severity of acne.

[0034] Therefore, the technical solution of this invention combines multispectral images to dynamically update feature weights, and performs weighted fusion of feature vectors of different bands based on the dynamically updated feature weights. This enables the highlighting of key information of different bands in different scenarios, and the imaging data of different bands can compensate for each other, solving the problem that sampled images are easily affected by environmental factors and improving the robustness of the system. By introducing a classification model to perform quantitative analysis of feature vectors of different bands, and by introducing user feature data to dynamically update the model weights, and using the dynamically updated model weights to dynamically fuse multiple classification models, it is beneficial to improve the accuracy and reliability of acne grading diagnosis results. This solves the problems of existing acne assessment and diagnosis methods relying on manual implementation, lacking specific and quantifiable grading standards, having poor diagnostic reliability, and having low diagnostic accuracy due to the susceptibility of the images used for assessment to skin color factors.

[0035] In some optional embodiments, the classification model includes at least: a non-inflammatory morphology classification model, an inflammation activity classification model, a lesion depth classification model, and a sebum secretion classification model; the model optimization module 300 is configured to: perform weighted fusion of the non-inflammatory morphology classification model, the inflammation activity classification model, the lesion depth classification model, and the sebum secretion classification model based on model weights to establish a target diagnostic model. Specifically, the non-inflammatory morphology classification model outputs quantitative evaluation results (such as scores) based on quantitative indicators of non-inflammatory morphologies such as open comedones (blackheads) and closed comedones (whiteheads); the inflammation activity classification model outputs quantitative evaluation results based on quantitative indicators of the inflammation activity of inflammatory lesions such as papules and pustules; the lesion depth classification model outputs quantitative evaluation results based on quantitative indicators of the lesion depth of inflammatory lesions such as papules, pustules, nodules, and cysts; and the sebum secretion classification model outputs quantitative evaluation results based on quantitative indicators of sebum secretion.

[0036] Specifically, if we define the visible light band feature vector as... The feature vector for the green or yellow light band is The corresponding feature weights are The characteristic vector of the ultraviolet or blue light band is The corresponding feature weights are The near-infrared band feature vector is The corresponding feature weights are , Then the fusion vector output by the data processing module 200 can be expressed as: .

[0037] Define the target diagnostic model as After the fusion vector is imported into the target diagnostic model by the output module 400, the non-inflammatory morphological classification model adopts... and The non-inflammatory morphological evaluation score was calculated using the input parameter; the inflammatory activity classification model was adopted. , and The inflammation activity score is calculated using the input parameter; the lesion depth classification model uses... , and The lesion depth evaluation score is calculated using the input parameter; the sebum secretion classification model is adopted. , and The sebum secretion level score is calculated using this as an input parameter.

[0038] Furthermore, by combining the weighted summation of the non-inflammatory morphology evaluation score, inflammatory activity evaluation score, lesion depth evaluation score, and sebum secretion evaluation score using model weights, a diagnostic score reflecting the severity of acne is obtained.

[0039] Preferably, the output parameter S of the target diagnostic model satisfies: ;in, , , and The weight values ​​represent the model weights corresponding to the classification model. ; This represents the quantitative assessment results based on non-inflammatory morphological generation. This represents a quantitative assessment result generated based on inflammatory activity. This represents a quantitative assessment result generated based on lesion depth; This represents a quantitative assessment result based on sebum secretion levels.

[0040] Specifically, For non-inflammatory morphological classification models, and The non-inflammatory morphological evaluation score is calculated using the input parameter; For the inflammation activity classification model, , and The inflammation activity evaluation score is calculated using the input parameter; For lesion depth classification models, , and The lesion depth evaluation score is calculated using the input parameter; The sebum secretion classification model was adopted , and The sebum secretion score is calculated using the input parameter. The final diagnostic score output by the target diagnostic model.

[0041] In some optional embodiments, the non-inflammatory morphology classification model sets at least two levels of quantification indicators for the proportion of comedo area; the inflammatory activity classification model sets at least four levels of quantification indicators for erythema index and at least four levels of quantification indicators for inflammatory lesion area; the lesion depth classification model sets at least three levels of quantification indicators for lesion depth and at least three levels of quantification indicators for lesion volume; and the sebum secretion classification model sets at least three levels of quantification indicators for fluorescence area.

[0042] For example, Table 1 shows specific scale indicators for a different classification model.

[0043]

[0044] As shown in Table 1 above, the non-inflammatory morphology classification model calculates the non-inflammatory morphology evaluation score based on the quantitative indicators that the area of ​​Grade I comedones is less than 5% and the area of ​​Grade II comedones is 5-8%. The inflammation activity classification model calculates the inflammation activity score based on the following quantitative indicators: Grade I erythema index less than threshold T1, inflammation area ratio less than 5%; Grade II erythema index [T1, T2], inflammation area ratio [5%, 15%]; Grade III erythema index (T2, T3), inflammation area ratio (15%, 30%); Grade IV erythema index greater than T3, inflammation area ratio greater than 30%. The lesion depth classification model calculates the lesion depth score based on the following quantitative indicators: Grade II lesion depth less than 0.5 mm, lesion volume less than 5 mm³; Grade III lesion depth [0.5 mm, 1.5 mm], lesion volume [5 mm³, 20 mm³]; Grade IV lesion depth greater than 1.5 mm, lesion volume greater than 20 mm³. The sebum secretion classification model calculates the sebum secretion score based on the following quantitative indicators: Grade I fluorescent area ratio [1%, 5%]; Grade II fluorescent area ratio (5%, 10%); Grade III fluorescent area ratio greater than 10%.

[0045] Therefore, the embodiments of the present invention simplify the model's computational workload and improve the efficiency of hierarchical diagnosis by establishing a segmented evaluation system.

[0046] Figure 2 This is a schematic diagram of the structure of a data acquisition module provided in an embodiment of the present invention.

[0047] Optionally, the data acquisition module 100 includes a multispectral imaging module with a built-in standard color chart 101 and a fluorescence reference unit 102; the standard color chart 101 is used to perform color restoration and white balance processing on the multispectral imaging data; the fluorescence reference unit 102 is used to calibrate the fluorescence spectral intensity of the multispectral imaging data.

[0048] Specifically, the standard color chart 101 of this invention has a preset spectral reflectance. Each time the multispectral imaging module is powered on or the lighting conditions change, the standard color chart is placed in the imaging area (e.g., the acne area of ​​the subject to be diagnosed), and a multispectral image containing the entire color chart is captured using the same light source and camera settings (e.g., exposure time, gain) as when imaging the acne area of ​​the subject. Further, a computer vision algorithm is used to automatically identify and locate the four corner points of the color chart in the visible light band image. Based on the known number of rows and columns, each color block area is automatically segmented, and the average pixel value of all pixels within each color block area is calculated. A calibration model (e.g., a linear calibration model) is constructed using the preset spectral reflectance and average pixel value of the same band. After acquiring multispectral imaging data of the acne area of ​​the subject to be diagnosed in the same band, the pixel value of each pixel is corrected using the calibration model, and the corrected pixel value is recorded as the reflectance of that pixel. Specifically, by using the white and neutral gray blocks on the color chart to adjust the gain of each channel (such as R, G, B, or each spectral band), the reflectance values ​​of these neutral color blocks are made equal (or proportional) across all wavelengths, thereby eliminating color shift of the light source. This reflectance reflects the skin tissue's ability to absorb and scatter light of different wavelengths.

[0049] The fluorescence reference unit 102 of this invention emits light of the same or similar wavelength as the target fluorescence signal (such as the orange-red fluorescence of porphyrin). Before each fluorescence imaging, the fluorescence reference unit 102 is placed in the imaging area (such as the acne area of ​​the subject to be diagnosed), and an image of the fluorescence reference unit 102 is captured using the exact same light source and camera settings (exposure time, gain) as when imaging the acne area of ​​the subject to be diagnosed. The average pixel value in the image of the fluorescence reference unit 102 is calculated, and a correlation between the pixel value and the standard fluorescence intensity value is established. After acquiring the clinical fluorescence image of the acne area of ​​the subject to be diagnosed, the pixel value of each pixel in the clinical fluorescence image is calculated, and the pixel value is converted into fluorescence intensity based on the correlation between the pixel value and the standard fluorescence intensity value. Fluorescence intensity is related to bacterial load and activity and is an important indicator of sebum secretion and acne activity.

[0050] In some optional embodiments, a complex nonlinear calibration map can be established by setting multiple fluorescence reference units (with high, medium and low intensities) to dynamically correct the multispectral imaging data.

[0051] Therefore, by setting up a standard color chart and a fluorescence reference unit, the multispectral image data collected at different times, with different devices, and under different lighting conditions are normalized, and the original pixel values ​​are converted into reflectance and fluorescence intensity to quantify acne-related pathological information.

[0052] Continue to refer to Figure 2As shown, the multispectral imaging module of the present invention further includes a multispectral light source 103 and an image acquisition unit 104. The multispectral light source 103 has the following wavelengths: visible light (e.g., 400 to 700 nm), green light (e.g., 540 to 580 nm) or yellow light (e.g., 570 to 590 nm), ultraviolet light (e.g., 315 to 400 nm) or blue light (e.g., 380 to 470 nm), red light (e.g., 600-700 nm), and near-infrared light (e.g., 700-1000 nm). The image acquisition unit 104 is used to acquire imaging data of the object to be diagnosed under different light source conditions.

[0053] Specifically, the multispectral light source can be formed by assembling several modular LED light source arrays, with each LED driven by an independent constant current driver. The operation of the multispectral light source is as follows: Each constant current driver is controlled by a drive control signal (e.g., pulse width modulation signal PWM), independently and precisely controlling the illumination of each LED light source array, and adjusting parameters such as the luminous intensity (e.g., adjustable from 0-100%) and luminous duration of each LED light source array. In some preferred embodiments, by controlling the illumination intensity and shooting distance of LED light sources in different wavelength bands to be consistent, motion blur can be avoided, ultimately acquiring image data of the same scene in different wavelength bands. Preferably, the resolution of the multispectral imaging data can be set to 512×512 pixels.

[0054] In some optional embodiments, the data processing module 200 includes: a preprocessing unit 201, used to perform registration and noise reduction enhancement processing on the multispectral imaging data, and to perform image segmentation on the processed data using a segmentation model to obtain image segmentation sub-regions; the image segmentation sub-regions include: acne-dominant region, inflammation-dominant region, and nodule-dominant region; a feature extraction unit 202, used to extract acne-related features from the image segmentation sub-regions, and to establish feature vectors based on the extracted acne-related features; and a feature fusion unit 203, used to perform preliminary classification based on the image segmentation sub-regions, determine feature weights based on the preliminary classification results, and perform weighted fusion of the feature vectors based on the feature weights.

[0055] Specifically, the comedone-dominant region is the image area dominated by comedones. The acne-related features extracted from the comedone-dominant region mainly include comedone area and number. Based on this, the feature vector can be represented as a visible light band feature vector. The inflammation-dominant region is the image area dominated by inflammatory skin lesions. Acne-related features extracted from this region mainly include the mean erythema index, the standard deviation of the erythema index, and the area of ​​the inflammatory region. Based on these features, the resulting feature vector can be represented as a feature vector in the green or yellow light bands. The nodule-dominant region is an image area primarily characterized by fusion. Acne-related features extracted from the nodule-dominant region mainly include lesion depth and lesion area. Based on this, the feature vector can be represented as a near-infrared band feature vector. The feature fusion unit 203 can import image segmentation sub-regions into a pre-trained classification model to perform preliminary classification, and determine feature weights by referring to the preliminary classification results output by the pre-trained classification model and the preset weights corresponding to different classification levels.

[0056] Figure 3 This is a schematic diagram of an output module provided in an embodiment of the present invention. See also... Figure 3 As shown, the output module 400 of the present invention further includes a visualization unit 401. This visualization unit 401 is used to visualize the fusion vector and the grading diagnosis results. The visualization of the fusion vector uses a visualization image with at least two colors. For example, regions of different colors can be superimposed on a visible light image, with the comedone-dominant area 401A marked in blue, the inflammation-dominant area 401B in red, and the deep nodule-dominant area 401C in purple. In some preferred embodiments, all calculated acne-related feature values ​​and their normal reference ranges can also be labeled in the visualization image. This visualization facilitates intuitive display and viewing of the acne grading diagnosis results.

[0057] Based on the same inventive concept as the above embodiments, the present invention provides an acne grading diagnosis assistance method based on multispectral imaging. The acne grading diagnosis assistance method based on multispectral imaging provided by the present invention can execute the control strategy of the acne grading diagnosis assistance system based on multispectral imaging provided in any embodiment of the present invention, and has the corresponding beneficial effects of executing the control strategy.

[0058] Figure 4 This is a flowchart illustrating an auxiliary method for acne grading diagnosis based on multispectral imaging, provided as an embodiment of the present invention. Figure 4 As shown, this acne grading diagnostic aid method based on multispectral imaging includes the following steps:

[0059] S101: Acquire multispectral imaging data and user characteristic data of the object to be diagnosed.

[0060] The user characteristic data includes at least one of the following: skin color characteristics, age characteristics, and gender characteristics.

[0061] S102: Based on multispectral imaging data, obtain feature vectors and feature weights for different bands, and perform weighted fusion of the feature vectors based on the feature weights to generate a fused vector.

[0062] S103: Determine the model weights based on user feature data, and establish a target diagnostic model based on the model weights.

[0063] The target diagnostic model includes multiple classification models, with each classification model corresponding to a specific model weight.

[0064] S104: Import the fusion vector into the target diagnostic model and determine the graded diagnostic result based on the output parameters of the target diagnostic model.

[0065] Optionally, the classification model may include at least: a non-inflammatory morphology classification model, an inflammatory activity classification model, a lesion depth classification model, and a sebum secretion classification model. Model weights are determined based on user feature data, and a target diagnostic model is established based on these model weights. Specifically, this includes weighted fusion of the non-inflammatory morphology classification model, the inflammatory activity classification model, the lesion depth classification model, and the sebum secretion classification model based on their respective model weights to establish the target diagnostic model.

[0066] Optionally, the output parameter S of the target diagnostic model satisfies: ;in, , , and The weight values ​​represent the model weights corresponding to the classification model. ; This represents the quantitative assessment results generated based on non-inflammatory morphological features; This represents a quantitative assessment result generated based on inflammatory activity. This represents a quantitative assessment result generated based on lesion depth; This represents a quantitative assessment result based on sebum secretion levels.

[0067] Optionally, the non-inflammatory morphology classification model can be configured with at least two levels of quantitative indicators for the proportion of comedo area; the inflammatory activity classification model can be configured with at least four levels of quantitative indicators for erythema index and at least four levels of quantitative indicators for inflammatory lesion area; the lesion depth classification model can be configured with at least three levels of quantitative indicators for lesion depth and at least three levels of quantitative indicators for lesion volume; and the sebum secretion classification model can be configured with at least three levels of quantitative indicators for fluorescence area.

[0068] Optionally, multispectral imaging data and user feature data of the object to be diagnosed can be acquired, specifically including: color restoration and white balance processing of the multispectral imaging data; and calibration of the fluorescence spectral intensity of the multispectral imaging data.

[0069] Optionally, the imaging bands of the multispectral imaging module include: visible light band, green or yellow light band, ultraviolet or blue light band, red light band, and near-infrared band.

[0070] Optionally, feature vectors and feature weights for different bands are obtained based on multispectral imaging data, and the feature vectors are weighted and fused based on the feature weights to generate a fused vector. Specifically, this includes: performing registration and noise reduction enhancement processing on the multispectral imaging data, and using a segmentation model to perform image segmentation on the processed data to obtain image segmentation sub-regions; the image segmentation sub-regions include: comedo-dominant region, inflammation-dominant region, and nodule-dominant region; extracting acne-related features from the image segmentation sub-regions, and establishing feature vectors based on the extracted acne-related features; performing preliminary classification based on the image segmentation sub-regions, determining feature weights based on the preliminary classification results, and weighted fusion of the feature vectors based on the feature weights.

[0071] Optionally, the feature vector may include at least: a visible light band feature vector, a green or yellow light band feature vector, an ultraviolet or blue light band feature vector, and a near-infrared band feature vector.

[0072] Optionally, the acne grading diagnosis assistance method based on multispectral images of the present invention further includes: visualizing the fusion vector and the grading diagnosis results; wherein the visualization of the fusion vector uses a visualization image with at least two colors.

[0073] Figure 5 This is a schematic diagram of an electronic device for implementing the acne grading diagnosis assistance method based on multispectral imaging, as described in this embodiment of the invention. The electronic device is intended to represent various forms of digital computers, such as laptops, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0074] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0075] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0076] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, central processing unit (CPU), graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the aforementioned acne grading diagnostic aid method based on multispectral imagery.

[0077] In some embodiments, the above-described acne grading diagnosis assistance method based on multispectral imagery can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the acne grading diagnosis assistance method based on multispectral imagery described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the above-described acne grading diagnosis assistance method based on multispectral imagery by any other suitable means (e.g., by means of firmware).

[0078] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0079] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0080] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0081] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0082] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0083] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0084] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0085] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A multispectral imaging-based acne grading diagnostic auxiliary system, characterized in that, include: The data acquisition module is used to acquire multispectral imaging data and user characteristic data of the object to be diagnosed; the user characteristic data includes at least one of the following: skin color characteristics, age characteristics, and gender characteristics; The data processing module is used to obtain feature vectors and feature weights of different bands based on the multispectral imaging data, and to perform weighted fusion of the feature vectors based on the feature weights to generate a fused vector; The model optimization module is used to determine model weights based on the user feature data and to establish a target diagnostic model based on the model weights; wherein, the target diagnostic model includes multiple classification models, and the classification models correspond one-to-one with the model weights; The output module, connected to the data processing module and the model optimization module respectively, is used to import the fusion vector into the target diagnostic model and determine the graded diagnostic result based on the output parameters of the target diagnostic model.

2. The acne grading and diagnostic auxiliary system based on multispectral imaging according to claim 1, characterized in that, The classification model includes at least: a non-inflammatory morphology classification model, an inflammatory activity classification model, a lesion depth classification model, and a sebum secretion classification model; The model optimization module is configured to: perform weighted fusion of the non-inflammatory morphology classification model, the inflammatory activity classification model, the lesion depth classification model, and the sebum secretion classification model based on the model weights to establish the target diagnostic model.

3. The acne grading and diagnostic auxiliary system based on multispectral imaging according to claim 2, characterized in that, The output parameter S of the target diagnostic model satisfies: ； in, , , and The weight values ​​represent the model weights corresponding to the classification model. ; This represents the quantitative assessment results generated based on non-inflammatory morphological features; This represents a quantitative assessment result generated based on inflammatory activity. This represents a quantitative assessment result generated based on lesion depth; This represents a quantitative assessment result based on sebum secretion levels.

4. The acne grading and diagnostic auxiliary system based on multispectral imaging according to claim 2, characterized in that, The non-inflammatory morphological classification model is equipped with at least two levels of quantitative indicators for the proportion of acne area. The inflammation activity classification model is configured with at least four levels of erythema index quantification indicators and at least four levels of inflammatory lesion area indicators; The lesion depth classification model is equipped with at least three levels of lesion depth quantification index and at least three levels of lesion volume quantification index. The sebum secretion classification model is configured with at least three levels of fluorescence area quantification indicators.

5. The acne grading and diagnostic auxiliary system based on multispectral imaging according to claim 1, characterized in that, The data acquisition module includes a multispectral imaging module with a built-in standard color chart and fluorescence reference unit; The standard color chart is used for color restoration and white balance processing of the multispectral imaging data; The fluorescence reference unit is used to calibrate the fluorescence spectral intensity of the multispectral imaging data.

6. The acne grading and diagnostic auxiliary system based on multispectral imaging according to claim 5, characterized in that, The multispectral imaging module further includes: a multispectral light source and an image acquisition unit; The multispectral light source includes the following wavelengths: visible light, green or yellow light, ultraviolet or blue light, red light, and near-infrared light. The image acquisition unit is used to acquire imaging data of the object to be diagnosed under different light source conditions.

7. The acne grading and diagnostic auxiliary system based on multispectral imaging according to claim 1, characterized in that, The data processing module includes: The preprocessing unit is used to perform registration and noise reduction enhancement processing on the multispectral imaging data, and to perform image segmentation on the processed data using a segmentation model to obtain image segmentation sub-regions; the image segmentation sub-regions include: acne-dominant region, inflammation-dominant region, and nodule-dominant region; The feature extraction unit is used to extract acne-related features from the image segmentation sub-regions and to establish the feature vector based on the extracted acne-related features; The feature fusion unit is used to perform preliminary classification based on the image segmentation sub-regions, determine the feature weights based on the preliminary classification results, and perform weighted fusion of the feature vectors based on the feature weights.

8. The acne grading and diagnostic auxiliary system based on multispectral imaging according to claim 7, characterized in that, The feature vectors include at least: visible light band feature vectors, green or yellow light band feature vectors, ultraviolet or blue light band feature vectors, and near-infrared band feature vectors.

9. The acne grading diagnostic auxiliary system based on multispectral imaging according to any one of claims 1-8, characterized in that, The output module further includes: a visualization unit; The visualization unit is used to visualize the fusion vector and the hierarchical diagnosis results; The visualization of the fused vector uses a visualization image with at least two colors.

10. A method for auxiliary diagnosis of acne grading based on multispectral imaging, characterized in that, include: Acquire multispectral imaging data and user characteristic data of the subject to be diagnosed; the user characteristic data includes at least one of the following: skin color characteristics, age characteristics, and gender characteristics; Based on the multispectral imaging data, feature vectors and feature weights for different bands are obtained, and the feature vectors are weighted and fused based on the feature weights to generate a fused vector. The model weights are determined based on the user feature data, and a target diagnostic model is established based on the model weights; wherein, the target diagnostic model includes multiple classification models, and each classification model corresponds one-to-one with the model weights; The fusion vector is imported into the target diagnostic model, and the graded diagnostic result is determined based on the output parameters of the target diagnostic model.

Images