Strong pulsed light skin beauty local area identification method and system

Abstract

The invention provides an intense pulsed light skin beauty local area identification method and system, and relates to the technical field of phototherapy, and the method comprises the steps: collecting an original face image of a patient, obtaining a historical skin treatment segmentation data set, carrying out the preprocessing of the original face image, and obtaining a standard face image; based on image feature information in the skin treatment segmentation data set, constructing a skin region division model, inputting the facial standard image into the skin region division model for target recognition, and obtaining a plurality of to-be-treated region images in the facial standard image; performing independent component analysis on the to-be-treated area image to obtain a melanin channel and a hemoglobin channel in the to-be-treated area image; constructing a multi-modal attention network model, inputting the melanin channel and the hemoglobin channel into the multi-modal attention network model for feature extraction and segmentation, and respectively obtaining melanin segmentation information and hemoglobin segmentation information; and fusing the melanin segmentation information and the hemoglobin segmentation information to obtain a skin treatment segmentation map, collecting treatment features in the skin treatment segmentation map, classifying the treatment features according to a random forest classifier, obtaining a skin treatment type corresponding to each to-be-treated area image, and obtaining the skin treatment type corresponding to each to-be-treated area image. And a treatment scheme corresponding to the skin treatment type is given. According to the method, the local detail sensitivity and the global information processing efficiency of the skin region division model can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of phototherapy, and in particular to a method and system for identifying a local area of ​​intense pulsed light skin beauty. Background Art

[0002] Intense pulsed light (IPL), also known as intense pulsed light, is a high-intensity light emitted by a special light source. After focusing and filtering, it forms light of a specific wavelength range. This light is incoherent ordinary light, which is different from laser, and its wavelength is usually between 500 and 1200nm. In the field of epidermal treatment, intense pulsed light is widely used, mainly because it can effectively treat a variety of skin problems, especially skin diseases related to phototherapy and photoaging. Intense pulsed light is mainly used in the treatment of pigmented skin diseases, vascular skin diseases and diseases related to changes in the structure of dermal collagen tissue.

[0003] A Chinese patent with announcement number CN115100107B discloses a dermoscopic image segmentation method and system. The method includes obtaining a dermoscopic image and preprocessing the dermoscopic image; based on the preprocessed dermoscopic image, using a segmentation model to obtain a segmentation result, the segmentation model includes: using a ResNet network to extract a first feature map; using the first feature map as the input of a Transformer, adding a boundary gate in the Transformer to extract local details to process fuzzy boundaries, and obtaining a second feature map; based on the second feature map, using a DenseASPP network to enhance feature representation and process multi-scale information to obtain a third feature map, and upsampling the obtained result to restore the resolution to obtain a segmentation result. However, the solution provided in the above application can only produce a limited receptive field, and cannot capture the dependencies between global relationships, which leads to the inability to effectively extract enough local details to process fuzzy boundaries. Therefore, it is very necessary to provide an intense pulsed light skin beauty local area recognition method and system to improve the local detail sensitivity and global information processing efficiency of the skin area division model. Summary of the invention

[0004] In view of this, the present invention proposes an intense pulsed light skin beauty local area recognition method and system, which combines the advantages of the skin area division model and independent component analysis to improve the local detail sensitivity and global information processing efficiency of the skin area division model.

[0005] The present invention provides an intense pulsed light skin beauty local area identification method, the method comprising:

[0006] Collecting the original facial image of the patient and obtaining a historical skin treatment segmentation dataset, preprocessing the original facial image to obtain a standard facial image;

[0007] Based on the image feature information in the skin treatment segmentation data set, a skin area division model is constructed, the facial standard image is input into the skin area division model for target recognition, and a plurality of images of areas to be treated in the facial standard image are obtained;

[0008] Performing independent component analysis on the image of the area to be treated to obtain a melanin channel and a hemoglobin channel in the image of the area to be treated;

[0009] Constructing a multimodal attention network model, and inputting the melanin channel and the hemoglobin channel into the multimodal attention network model for feature extraction and segmentation, to obtain melanin segmentation information and hemoglobin segmentation information respectively;

[0010] The melanin segmentation information and the hemoglobin segmentation information are fused to obtain a skin treatment segmentation map, and the repair features in the skin treatment segmentation map are collected. The repair features are classified according to a random forest classifier to obtain the skin treatment type corresponding to each image of the area to be treated, and a treatment plan corresponding to the skin treatment type is given.

[0011] On the basis of the above technical solution, preferably, the preprocessing of the original facial image to obtain the standard facial image specifically includes:

[0012] Converting the original facial image into RBG format, and adjusting the facial area in the original facial image to a horizontal state to obtain a facial transition image;

[0013] The facial transition image is sequentially subjected to color correction, noise removal, illumination balance and detail enhancement processing to obtain a facial standard image.

[0014] On the basis of the above technical solution, preferably, the detail enhancement specifically includes:

[0015] Segmenting the facial transition image after color correction, noise removal and illumination equalization to obtain a facial transition segmentation map consisting of a plurality of segmentation blocks;

[0016] Obtaining a single-channel grayscale image of each segmentation block in the facial transition segmentation image, obtaining the grayscale level of each pixel in each segmentation block according to a grayscale level standard, and treating the pixel points whose grayscale level is lower than a preset level threshold as under-enhanced pixel points;

[0017] The pixel value of the pixel point at the corresponding position of the under-enhanced pixel point in the facial transition image is adjusted to the corresponding expected pixel value to obtain the facial standard image, wherein the expected pixel value is obtained based on the grayscale level of the corresponding pixel point, the total number of pixel values ​​of the segmented block where the corresponding pixel point is located, and the total number of grayscale levels.

[0018] More preferably, the skin area segmentation model is constructed based on the image feature information in the skin treatment segmentation dataset, specifically comprising:

[0019] Acquire unimodal image data and multimodal image data in a skin treatment segmentation data set, divide the unimodal image data and the multimodal image data into first image data and second image data at a preset ratio, and extract data features in the first image data and the second image data based on a feature extraction function;

[0020] Based on the data features in the first image data, construct an initial multimodal attention fusion network, and use multi-scale feature fusion to increase the receptive field of the initial multimodal attention fusion network;

[0021] Extracting image depth features from the initial multimodal attention fusion network, and calibrating the image depth feature channels in the initial multimodal attention fusion network using an attention mechanism to obtain calibrated image depth features;

[0022] The depth features of the calibrated image are fused with the original facial image to construct a multimodal attention fusion network, and the multimodal attention fusion network is trained according to the data features and loss function in the second image data to obtain the skin area division model.

[0023] More preferably, the expressions of the feature extraction function and the loss function are respectively:

[0024]

[0025] Among them, M c (F) represents the feature extraction function, F represents the feature map of the input attention mechanism module, σ() represents the activation function, MLP() represents the built-in optimization function of the three-layer perceptron, AvgPool() represents the average pooling function, W0 represents the first weight matrix, W1 represents the second weight matrix, represents the feature after average pooling, L total represents the loss function, α represents the weight loss parameter, q represents the true label value output by the model, p represents the predicted value output by the model, δ represents the marginal adjustment parameter, N represents the total number of samples input to the model, n represents the nth sample, ω represents the hyperparameter, and q n Represents the true label value of the nth sample, p n Represents the predicted value of the nth sample.

[0026] More preferably, performing independent component analysis on the image of the area to be treated to obtain a melanin channel and a hemoglobin channel in the image of the area to be treated specifically includes:

[0027] Based on the skin color density function, a skin optical model is constructed, and the skin optical model and the predicted pure color density vector are introduced into the skin optical model to obtain a synthetic skin color density function;

[0028] Based on the synthetic skin color density function and the least square method, the melanin channel and the hemoglobin channel in the image of the area to be treated are solved respectively.

[0029] More preferably, the expression of the synthetic skin color density function is:

[0030]

[0031] Among them, I x,y represents the skin color density function, x represents the horizontal coordinate of the pixel point, y represents the horizontal coordinate of the pixel point, c m Represents the pure color characteristics per unit density of melanin, c h Represents the pure color characteristic of hemoglobin per unit density, Represents the relative amount of melanin in each pixel. represents the relative amount of hemoglobin in each pixel, Δh represents the stationary column vector generated by other skin pigments and structures, and S x,y represents the synthetic skin color density function, represents the predicted pure color density vector, K represents the first density control parameter, and j represents the second density control parameter.

[0032] More preferably, the random forest classifier includes a feature integration unit, a data set partitioning unit, a decision tree construction unit, a multiple decision tree training unit and a classification prediction unit, wherein:

[0033] The feature integration unit is used to integrate the repair features collected in the skin treatment segmentation map and form a repair feature data set;

[0034] The data set division unit is used to divide the restoration feature data set into a plurality of different restoration feature sub-data sets by random sampling;

[0035] The decision tree construction unit is used for performing decision tree generation and recursive splitting;

[0036] The multiple decision tree training unit is used to establish different decision trees on different restoration feature sub-data sets respectively;

[0037] The classification prediction unit is used to perform sample classification and output the skin treatment type corresponding to each image of the area to be treated.

[0038] More preferably, the skin treatment types include pigmentary diseases, vascular diseases, keratosis pilaris, acne and its sequelae, and skin texture problems.

[0039] In a second aspect of the present application, an intense pulsed light skin beauty local area recognition system is provided, wherein the local area operation system comprises a data acquisition module, an information extraction module and a treatment operation module, wherein:

[0040] The data acquisition module is used to collect the original facial image of the patient and obtain the historical skin treatment segmentation data set, pre-process the original facial image, and obtain the standard facial image;

[0041] The information extraction module is used to construct a skin area division model based on the image feature information in the skin treatment segmentation data set, input the facial standard image into the skin area division model for target recognition, obtain multiple images of the area to be treated in the facial standard image, perform independent component analysis on the image of the area to be treated, obtain the melanin channel and the hemoglobin channel in the image of the area to be treated, construct a multimodal attention network model, and input the melanin channel and the hemoglobin channel into the multimodal attention network model for feature extraction and segmentation, and obtain melanin segmentation information and hemoglobin segmentation information respectively;

[0042] The treatment operation module is used to fuse the melanin segmentation information and the hemoglobin segmentation information to obtain a skin treatment segmentation map, collect repair features in the skin treatment segmentation map, classify the repair features according to a random forest classifier, obtain the skin treatment type corresponding to each image of the area to be treated, and provide a treatment plan corresponding to the skin treatment type.

[0043] The method and system for identifying local areas of intense pulsed light skin beauty provided by the present invention have the following beneficial effects compared with the prior art:

[0044] (1) By collecting original facial images and combining them with historical treatment data sets to build a skin region segmentation model, multiple facial regions to be treated can be accurately identified and segmented. At the same time, independent component analysis is performed on the regions to be treated to separate the melanin channel and the hemoglobin channel, which can more deeply analyze the pigmentation and vascular characteristics of the skin. The multimodal attention network model is used to extract and segment the melanin and hemoglobin channels, which can simultaneously consider both pigment and vascular characteristics. This multimodal analysis can provide more comprehensive diagnostic information, thereby improving the accuracy of segmentation. In addition, the features extracted by the independent component analysis can be further optimized by utilizing the advantages of the long-term dependent modeling of the skin region segmentation model to complete the task of image segmentation of the region to be treated. Combining the advantages of the skin region segmentation model and independent component analysis, it helps to improve the local detail sensitivity and global information processing efficiency of the skin region segmentation model, thereby more comprehensively understanding the image content.

[0045] (2) By using feature extraction functions to process different types of image data, more comprehensive skin feature information can be captured. Multi-scale feature fusion is used to increase the network's receptive field, allowing the model to focus on local details and global structures at the same time. The attention mechanism is used to calibrate the image's deep feature channels, highlight important features, and suppress irrelevant information, so as to improve the skin region segmentation model's ability to recognize key skin features, thereby improving the accuracy of the segmentation.

[0046] (3) By constructing a skin optical model, the distribution of melanin and hemoglobin can be separated more accurately. The introduction of the skin optical model and the predicted pure color density vector takes into account the complexity and multi-layered nature of the skin structure, making the analysis results closer to the actual skin condition. At the same time, the melanin and hemoglobin channels can be accurately separated and the distribution of melanin and hemoglobin can be accurately quantified, so that the parameters of intense pulsed light therapy can be selected more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 A schematic diagram of a flow chart of a method for identifying local areas of intense pulsed light skin beauty provided by the present invention;

[0049] Figure 2 This is a schematic diagram of the structure of the local area operation system provided by the present invention.

[0050] Explanation of the accompanying drawings: 1. Local area operation system; 11. Data acquisition module; 12. Information extraction module; 13. Treatment operation module. DETAILED DESCRIPTION

[0051] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] The present application embodiment discloses a method for identifying a local area of ​​intense pulsed light skin beauty treatment, such as Figure 1 As shown, the method comprises steps S1 to S5.

[0053] Step S1, collecting the original facial image of the patient and obtaining the historical skin treatment segmentation data set, preprocessing the original facial image, and obtaining a standard facial image.

[0054] In this embodiment, the original image of the patient's face can be collected using a high-resolution digital camera or professional medical imaging equipment, such as a Canon EOS R5 full-frame micro-single camera equipped with a 45MP high-resolution sensor or a VISIA skin analyzer, and the image is taken under standard lighting conditions to ensure consistent image quality and to capture comprehensive information from multiple angles.

[0055] This step includes steps S11 to S12.

[0056] Step S11, converting the original facial image into RBG format, and adjusting the facial area in the original facial image to a horizontal state to obtain a facial transition image.

[0057] In this step, the original facial image is read using the image processing library, and the current format of the original facial image is checked. If it is already in RGB format, the conversion step is skipped; if it is not in RGB format, the corresponding color space conversion function is called to convert the original facial image into RGB format, and the converted RGB format image is saved. The facial detection algorithm is applied to the RGB format image to locate the facial area. Within the detected facial area, the feature point detection algorithm is used to locate the position of both eyes. The facial inclination angle is calculated based on the coordinates of the left and right eyes. The center of the RGB format image is used as the rotation center. According to the calculated angle, the entire RGB format image is rotated. After rotation, the facial area is re-detected to ensure that the face has been adjusted to a horizontal state. If necessary, the image is cropped to remove the blank area caused by the rotation, and the adjusted image is saved as a facial transition image.

[0058] Step S12, color correction, noise removal, illumination balance and detail enhancement are sequentially performed on the facial transition image to obtain a facial standard image.

[0059] In this step, the gray world algorithm is used to calculate the average values ​​of the R, G, and B channels in the image. Assume that the average values ​​of R are 120, G are 140, and B are 160. Calculate the adjustment coefficient K R =140 / 120=1.17, K G =1, K B =140 / 160=0.875. Multiply the R value of each pixel in the image by 1.17, the B value by 0.875, and the G value remains unchanged. Convert the image to the HSV color space, analyze the average value of the S channel (saturation), assume it is 0.4 (range 0-1), increase the value of the S channel by 20% overall, and convert the adjusted image back to the RGB space.

[0060] Analyze the image noise characteristics to determine whether it is Gaussian noise, salt and pepper noise, or other types. Use Gaussian filtering or bilateral filtering for Gaussian noise, median filtering for salt and pepper noise, and advanced noise reduction algorithms such as non-local means (NLM) or BM3D for complex noise. Adjust the parameters of the noise reduction algorithm based on the image characteristics and the degree of noise to strike a balance between retaining details and removing noise.

[0061] Analyze the image brightness histogram, evaluate the degree of uneven lighting, use histogram equalization or adaptive histogram equalization (CLAHE) to adjust the overall brightness, apply local contrast enhancement algorithms such as the Retinex algorithm or multi-scale adaptive enhancement, identify shadow and highlight areas in the image, perform brightness restoration and detail enhancement respectively, and analyze and correct the brightness differences in different facial areas caused by the position of the light source.

[0062] This step also includes steps S13 to S15.

[0063] Step S13, segmenting the facial transition image after color correction, noise removal and illumination equalization processing to obtain a facial transition segmentation map composed of a plurality of segmentation blocks.

[0064] In this step, the 68-point facial feature detection model of the dlib library is used to locate 68 key feature points on the facial image, including eyes, eyebrows, nose, mouth, and facial contours. The face is divided into the following main areas according to the feature points: from the upper edge of the eyebrows to the hairline is divided into the forehead area, the left and right eyebrows are divided into the eyebrow area, the area around the left and right eyes is divided into the eye area, from the center of the eyebrows to the tip of the nose is divided into the nose area, the left and right cheeks are divided into the cheek area, the upper and lower lips are divided into the mouth area, and from the lower lip to the bottom of the chin is divided into the chin area.

[0065] Further, the forehead area can be divided into 3 blocks horizontally and 2 blocks vertically, for a total of 6 blocks; the eyebrow area can be divided into 3 blocks on each side of the eyebrow, for a total of 6 blocks; the eye area can be divided into 4 blocks around each eye, for a total of 8 blocks; the nose area can be divided into 3 blocks: upper, middle, and lower; the cheek area can be divided into 4 blocks on each side of the cheek, for a total of 8 blocks; the mouth area can be divided into 2 blocks on the upper lip and 2 blocks on the lower lip, for a total of 4 blocks; the chin area can be divided into 3 blocks. The boundary of each area is calculated using the coordinates of the feature points. For example, the boundary of the eye area can be determined by the feature points of the corner of the eye and the lower edge of the eyebrow.

[0066] Create a blank mask image of the same size as the original image. Use the cv2.fillPoly() function to fill the corresponding polygonal area for each defined area, and assign a unique identification number to each area. For the facial contour, use the feature points to fit the curve to ensure smooth segmentation edges. In key areas such as eyes and mouth, use finer segmentation to retain details.

[0067] Step S14, obtaining a single-channel grayscale image of each segmentation block in the facial transition segmentation image, obtaining the grayscale level of each pixel in each segmentation block according to the grayscale level standard, and taking the pixel whose grayscale level is lower than the preset level threshold as the under-enhanced pixel.

[0068] Step S15, adjusting the pixel value of the pixel point at the corresponding position of the under-enhanced pixel point in the facial transition image to the corresponding expected pixel value, to obtain a facial standard image, wherein the expected pixel value is obtained based on the grayscale level of the corresponding pixel point, the total number of pixel values ​​of the segmented block where the corresponding pixel point is located, and the total number of grayscale levels.

[0069] In an example, assume that the right cheek area of ​​a facial image is divided into a 50x50 pixel block, the block size is 50x50 pixels, the total number of pixels (TP) is 2500, the total number of gray levels (TGL) is 10, the preset level threshold is 4 (pixels below level 4 are considered under-enhanced), and the range of each gray level is 255 / 10=25.5.

[0070] In this segmented block, a pixel is determined whose RGB value is (70, 65, 60). Gray is calculated by the formula Gray = 0.299R + 0.587G + 0.114B, Gray = 0.299*70 + 0.587*65 + 0.114*60≈66, and the current gray level is determined to be 66 / 25.5≈2.59, which is rounded down to level 2. Since 2<4 (preset threshold), this pixel is identified as an under-enhanced pixel, and the target level is determined, target level = current level + 2 = 2 + 2 = 4 (not exceeding the highest level 10). The minimum value of level 4 is (4-1)*25.5=76.5, and the maximum value of level 4 is 4*25.5=102. The relative position of the current value in level 2 is (66-25.5) / (51-25.5)≈0.59. The expected value can be represented by E, E=76.5+0.59*(102-76.5)≈91.5. The original gray value 66 is adjusted to the new expected value 92 (rounded), keeping the color ratio unchanged. The new RGB value is approximately R: 70*(92 / 66)≈97, G: 65*(92 / 66)≈90, B: 60*(92 / 66)≈83. In the facial transition image, the value of this pixel is updated from (70, 65, 60) to (97, 90, 83), and this process is repeated for all pixels identified as under-enhanced in the segmented block.

[0071] Step S2, based on the image feature information in the skin treatment segmentation data set, a skin area division model is constructed, the facial standard image is input into the skin area division model for target recognition, and multiple images of the area to be treated in the facial standard image are obtained.

[0072] This step also includes steps S21 to S24.

[0073] Step S21, obtaining unimodal image data and multimodal image data in the skin treatment segmentation data set, and dividing the unimodal image data and the multimodal image data into first image data and second image data in a preset ratio, and extracting data features in the first image data and the second image data based on a feature extraction function.

[0074] In this step, the skin treatment segmentation dataset can be a large-scale skin lesion image dataset provided by the International Skin Imaging Collaboration (ISIC). Use Python's image processing library (such as PIL, OpenCV) to read the image files in the skin treatment segmentation dataset, traverse the dataset directory, and load single-modal and multi-modal images respectively.

[0075] Step S22: construct an initial multimodal attention fusion network based on the data features in the first image data, and use multi-scale feature fusion to increase the receptive field of the initial multimodal attention fusion network.

[0076] In this step, a two-branch network structure is constructed to process RGB images and depth images respectively. Each branch contains feature extraction, attention mechanism and multi-scale feature extraction modules, and finally the features of the two branches are combined through the fusion module. For the RGB branch, the pre-trained ResNet50 is used as the backbone network. For the depth map branch, the modified ResNet50 (the number of input channels is changed to 1) is used. The last fully connected layer of the two ResNet50s is removed, and the feature map output is retained. A spatial attention module is added after the ResNet50 output of each branch, and the feature map is compressed to a single channel using a 1x1 convolution layer. The Sigmoid activation function is applied to obtain the attention weight, and the attention weight is multiplied by the original feature map. A multi-scale feature extraction module is designed in each branch, and 1x1, 3x3, and 5x5 convolution layers are used to process the feature maps in parallel. The dilated convolution is used to increase the receptive field, and the dilation rates are 1, 2, and 4 respectively, and the feature maps of different scales are spliced. The multi-scale features of the RGB branch and the depth branch are concatenated, a 1x1 convolution layer is used to reduce the number of feature channels, and a global average pooling is applied to obtain a fixed-length feature vector. The cross entropy loss function is used. The Adam optimizer is used, the initial learning rate is set to 0.001, and the learning rate scheduler is used. The training is performed for 200 epochs, and the learning rate is reduced every 50 epochs.

[0077] Step S23, extract image depth features from the initial multimodal attention fusion network, and use the attention mechanism to calibrate the image depth feature channels in the initial multimodal attention fusion network to obtain calibrated image depth features.

[0078] In this step, a channel attention module is added to the feature map after ResNet50 feature extraction, and the feature maps are processed using global average pooling and global maximum pooling to obtain two channel descriptors. The two descriptors are processed by a shared multi-layer perceptron (MLP). The MLP contains two fully connected layers, and the ReLU activation function is used in the middle. The outputs of the MLP are added and the channel attention weights are obtained by the Sigmoid function. The obtained channel attention weights are multiplied with the original feature map to achieve weighted importance of different channels. After the channel attention, a spatial attention module is added, and maximum pooling and average pooling are performed along the channel dimension to obtain two 2D feature maps. The two 2D feature maps are concatenated and processed through a 7x7 convolution layer. The spatial attention weight map is obtained using the Sigmoid function, and the spatial attention weight map is multiplied with the feature map processed by the channel attention. The processed RGB features and depth features are concatenated, and a 1x1 convolution layer is used to reduce the number of feature channels. A multi-scale feature extraction module is applied to the fused features, including convolutions with different kernel sizes and dilation rates. The multi-scale features are concatenated, and 1x1 convolution is used again to reduce the number of channels. Global average pooling is applied to obtain a feature vector of fixed length.

[0079] Step S24, fusing the depth features of the calibrated image with the original facial image, constructing a multimodal attention fusion network, and training the multimodal attention fusion network according to the data features and loss function in the second image data to obtain a skin area division model.

[0080] In this embodiment, the expressions of the feature extraction function and the loss function are respectively:

[0081]

[0082] Among them, M c (F) represents the feature extraction function, F represents the feature map of the input attention mechanism module, σ() represents the activation function, MLP() represents the built-in optimization function of the three-layer perceptron, AvgPool() represents the average pooling function, W0 represents the first weight matrix, W1 represents the second weight matrix, represents the feature after average pooling, L total represents the loss function, α represents the weight loss parameter, q represents the true label value output by the model, p represents the predicted value output by the model, δ represents the marginal adjustment parameter, N represents the total number of samples input to the model, n represents the nth sample, ω represents the hyperparameter, and q n Represents the true label value of the nth sample, p n Represents the predicted value of the nth sample.

[0083] Step S3, performing independent component analysis on the image of the area to be treated to obtain a melanin channel and a hemoglobin channel in the image of the area to be treated.

[0084] This step also includes steps S31 to S32.

[0085] Step S31, constructing a skin optical model based on the skin color density function, and introducing the skin optical model and the predicted pure color density vector into the skin optical model to obtain a synthetic skin color density function.

[0086] In this embodiment, the expression of the synthetic skin color density function is:

[0087]

[0088] Among them, I x,y represents the skin color density function, x represents the horizontal coordinate of the pixel point, y represents the horizontal coordinate of the pixel point, c m Represents the pure color characteristics per unit density of melanin, c h Represents the pure color characteristic of hemoglobin per unit density, Represents the relative amount of melanin in each pixel. represents the relative amount of hemoglobin in each pixel, Δh represents the stationary column vector generated by other skin pigments and structures, and S x,y represents the synthetic skin color density function, represents the predicted pure color density vector, K represents the first density control parameter, and j represents the second density control parameter.

[0089] Step S32, based on the synthetic skin color density function and the least square method, respectively solve the melanin channel and the hemoglobin channel in the image of the area to be treated.

[0090] In one example, it is assumed that the skin includes three skin clusters: normal skin, pigmented skin, and erythema. For each skin cluster, a reference value vector is learned by selecting a group of pixels from the image. In order to learn the reference values ​​of the three skin clusters, the user is required to manually outline the areas of normal skin, pigmented skin, and vascular system in the selected training pixels. The selected image is processed by independent component analysis to extract the hemoglobin channel. Subsequently, the mean and standard deviation of the R, G, and B channels of the hemoglobin components of normal skin, pigmented skin, and erythema are calculated. The reference values ​​obtained above are stored as prior information, and a threshold framework is designed based on the Mahalanobis distance on the skin hemoglobin component to classify the red area. For each pixel of the image, the Mahalanobis distance of the three skin clusters (normal skin, pigmented skin, and blood vessels) in the hemoglobin component is calculated, and the pixel is classified into the group with the closest distance.

[0091]

[0092] in, Respectively represent the Mahalanobis distance between the pixel (x, y) and normal skin, pigmented skin and blood vessels, and the subscripts ri, gi, and bi are the RGB values ​​of the reference cluster. Then the pixel belongs to the erythema cluster. By performing the same analysis on each pixel of each image, each pixel is classified into one of the three mentioned skin clusters to achieve segmented skin regions.

[0093] Furthermore, shape information can be considered at the same time as color, mainly considering two shape categories, namely tubular and circular skin blood vessels. In order to measure the tubularity of a pixel, the pixel X = (x, y) is plotted at different scales s = {s1, s2, ..., s k} can be expressed using the following formula:

[0094]

[0095] Among them, R and S are the measures of spot degree and second-order structure degree, λ i (X,s), i = 1, 2 (|λ1| ≤ |λ2|) is the Hessian matrix eigenvalue of the image i at scale s, β and c are control parameters that control the filter's sensitivity to R and S, R reaches its maximum at the clumping structure, while S will be very low in the absence of structure and with a small eigenvalue. The presence of tubular structure increases the contrast. This function represents the class probability estimate that the image is mapped to a tubular shape.

[0096] According to the direction of subcutaneous blood vessels, some skin blood vessels may be circular structures (dots). For a pixel to belong to a circular structure, the eigenvalues ​​of the two Hessian matrices should be in the same order of magnitude. Therefore, the class probability estimate of the image mapping to a circle can be expressed as:

[0097]

[0098] Using the melanin and hemoglobin channels generated by the segmentation step, 12 vascular features were defined and extracted for each pathology in the historical skin treatment segmentation dataset. These features include maximum vessel length, mean vessel length, length standard deviation, maximum vessel area, mean vessel area, vessel area standard deviation, vessel area to lesion area ratio, maximum vessel width, mean vessel width, vessel width standard deviation, number of vessel branches, and branch to lesion area ratio.

[0099] Step S4, construct a multimodal attention network model, and input the melanin channel and the hemoglobin channel into the multimodal attention network model for feature extraction and segmentation, so as to obtain melanin segmentation information and hemoglobin segmentation information respectively.

[0100] In this embodiment, independent input branches are designed for the melanin channel and the hemoglobin channel. This separate initial processing allows the model to capture the unique features of each channel separately, laying the foundation for subsequent fusion and analysis. Each branch is equipped with a feature extraction network, such as ResNet50 or DenseNet as the backbone network. These deep convolutional neural networks can extract rich multi-scale features from their respective channels, including low-level texture information and high-level semantic information. After feature extraction, a multi-level attention mechanism is introduced, including a channel attention mechanism and a spatial attention mechanism. The channel attention mechanism can adaptively adjust the importance of different feature maps, and the spatial attention mechanism is used to highlight the key areas in the image. These attention mechanisms are not only applied within a single channel, but also extended across channels, forming a cross-modal attention layer to establish a dynamic connection between melanin and hemoglobin information and capture the complex interaction between them.

[0101] Furthermore, the U-Net architecture is used as the decoder network. The U-Net architecture contains multiple upsampling blocks and fine-grained jump connections. These jump connections are not just simple feature transfers, but are weighted by attention, ensuring that only the most relevant low-level features are passed to the corresponding layers of the decoder, improving the model's ability to retain and utilize detailed information. The output layer of the multimodal attention network model designs independent but interrelated output heads for melanin segmentation and hemoglobin segmentation, respectively. Each output head not only contains a series of convolutional layers but also introduces a feedback mechanism to allow information exchange between the two segmentation tasks.

[0102] In this embodiment, the melanin channel segmentation results include pigmentation areas, that is, areas showing higher melanin content in the skin; hypopigmentation areas, that is, areas marking lower melanin content; pigment distribution patterns, that is, showing the distribution of melanin in the skin, such as uniform distribution or irregular distribution; boundary information, that is, showing the boundary contour of pigmentation treatment; structural features, such as specific melanin distribution patterns such as reticular structure and spherical structure. The hemoglobin channel segmentation results include vascular structure, that is, showing the vascular network and distribution in the skin; erythema area, that is, marking the area of ​​blood filling or inflammation in the skin; vascular morphology, that is, vascular structures of different forms such as linear, dot-shaped, and dendritic; vascular density, that is, showing the density of blood vessels in different areas; abnormal vascular structure, such as dilated blood vessels or abnormal vascular patterns. These segmentation results are usually presented in the form of binary images or multi-valued label maps.

[0103] Step S5, fuses the melanin segmentation information and the hemoglobin segmentation information to obtain a skin treatment segmentation map, collects the repair features in the skin treatment segmentation map, classifies the repair features according to the random forest classifier, obtains the skin treatment type corresponding to each image of the area to be treated, and gives a treatment plan corresponding to the skin treatment type.

[0104] In this step, different weights are assigned to the melanin and hemoglobin segmentation results, which can be determined based on clinical importance or specific treatment goals. For example, the final segmentation = α*melanin segmentation information + β*hemoglobin segmentation information, where α+β=1. Create a multi-level classification system to reflect different treatment needs. Use color coding to visualize different areas of treatment needs. For example, green indicates normal, yellow indicates mild, orange indicates moderate, and red indicates severe. And use morphological operations (such as dilation and erosion) to enhance segmentation boundaries, perform connected region analysis, and identify and mark independent treatment areas.

[0105] In one example, the melanin segmentation map and the hemoglobin segmentation map are extracted from the melanin segmentation information and the hemoglobin segmentation information, and the pixel value range of both is 0-255. The two segmentation maps are normalized to the range of 0-1, and the images are adjusted to the same size to ensure that the two images are of the same size. Weights are assigned to melanin and hemoglobin, for example, the melanin weight α=0.6 and the hemoglobin weight β=0.4. The weight assignment can be adjusted according to the specific skin lesion type and clinical importance. Thresholds are set to divide the fusion results into multiple levels, where level 0 (normal) fusion value <0.3; level 1 (mild) 0.3≤fusion value <0.5; level 2 (moderate) 0.5≤fusion value <0.7; level 3 (severe) fusion value ≥0.7. Different levels are represented by different colors, level 0 is green (RGB: 0, 255, 0); level 1 is yellow (RGB: 255, 255, 0); level 2 is orange (RGB: 255, 165, 0); level 3 is red (RGB: 255, 0, 0). Morphological operations (such as dilation) are used to enhance the boundaries of each area. The fused image is subjected to connected region analysis, a unique identifier is assigned to each independent area, the total area and relative proportion of each level are calculated, and the proportion of the total area requiring treatment (levels 1-3) is calculated.

[0106] In this embodiment, the random forest classifier includes a feature integration unit, a data set partitioning unit, a decision tree construction unit, a multiple decision tree training unit and a classification prediction unit, wherein:

[0107] The feature integration unit is used to integrate the repair features collected in the skin treatment segmentation map and form a repair feature data set; the data set division unit is used to divide the repair feature data set into several different repair feature sub-data sets by random sampling; the decision tree construction unit is used to generate decision trees and recursively split them; the multiple decision tree training unit is used to establish different decision trees on different repair feature sub-data sets; the classification prediction unit is used to classify samples and output the skin treatment type corresponding to each image of the area to be treated.

[0108] Skin treatment types include pigmentary disorders, vascular disorders, keratosis pilaris, acne and its sequelae, and skin texture problems.

[0109] By collecting original facial images and combining them with historical treatment data sets to build a skin region segmentation model, multiple facial areas to be treated can be accurately identified and divided. At the same time, independent component analysis is performed on the areas to be treated to separate the melanin channel and the hemoglobin channel, which can more deeply analyze the pigmentation and vascular characteristics of the skin. The multimodal attention network model is used to extract and segment the melanin and hemoglobin channels, which can consider both pigment and vascular characteristics. This multimodal analysis can provide more comprehensive diagnostic information, thereby improving the accuracy of segmentation, and can use the advantages of long-term dependent modeling of the skin region segmentation model to further optimize the features extracted by independent component analysis to complete the task of image segmentation of the area to be treated. Combining the advantages of the skin region segmentation model and independent component analysis, it helps to improve the local detail sensitivity and global information processing efficiency of the skin region segmentation model, thereby more comprehensively understanding the image content.

[0110] Based on the above method, the present application embodiment discloses an intense pulsed light skin beauty local area recognition system, referring to Figure 2 The local area operation system 1 includes a data acquisition module 11, an information extraction module 12 and a treatment operation module 13, wherein:

[0111] The data acquisition module 11 is used to collect the original facial image of the patient and obtain the historical skin treatment segmentation data set, pre-process the original facial image, and obtain the standard facial image;

[0112] The information extraction module 12 is used to construct a skin area division model based on the image feature information in the skin treatment segmentation data set, input the facial standard image into the skin area division model for target recognition, obtain multiple images of the area to be treated in the facial standard image, perform independent component analysis on the image of the area to be treated, obtain the melanin channel and the hemoglobin channel in the image of the area to be treated, construct a multimodal attention network model, and input the melanin channel and the hemoglobin channel into the multimodal attention network model for feature extraction and segmentation, and obtain melanin segmentation information and hemoglobin segmentation information respectively;

[0113] The treatment operation module 13 is used to fuse the melanin segmentation information and the hemoglobin segmentation information, obtain a skin treatment segmentation map, collect repair features in the skin treatment segmentation map, classify the repair features according to the random forest classifier, obtain the skin treatment type corresponding to each image of the area to be treated, and provide a treatment plan corresponding to the skin treatment type.

[0114] In one example, the data acquisition module 11 is used to convert the original facial image into RBG format, and adjust the facial area in the original facial image to a horizontal state to obtain a facial transition image; the facial transition image is sequentially subjected to color correction, noise removal, lighting balance, and detail enhancement processing to obtain a facial standard image.

[0115] In one example, the data acquisition module 11 is used to segment the facial transition image after color correction, noise removal and illumination equalization processing to obtain a facial transition segmentation map composed of multiple segments; obtain a single-channel grayscale image of each segmentation block in the facial transition segmentation map, and obtain the grayscale level of each pixel in each segmentation block according to the grayscale level standard, and use the pixel points with grayscale levels lower than the preset level threshold as under-enhanced pixels; adjust the pixel values ​​of the pixel points at the corresponding positions of the under-enhanced pixels in the facial transition image to the corresponding expected pixel values ​​to obtain a facial standard image, wherein the expected pixel value is obtained based on the grayscale level of the corresponding pixel point, the total number of pixel values ​​of the segmentation block where the corresponding pixel point is located, and the total number of grayscale levels.

[0116] In one example, the information extraction module 12 is used to obtain unimodal image data and multimodal image data in a skin treatment segmentation data set, and divide the unimodal image data and the multimodal image data into first image data and second image data in a preset ratio, and extract data features in the first image data and the second image data based on a feature extraction function; based on the data features in the first image data, an initial multimodal attention fusion network is constructed, and multi-scale feature fusion is used to increase the receptive field of the initial multimodal attention fusion network; image depth features are extracted from the initial multimodal attention fusion network, and the image depth feature channels in the initial multimodal attention fusion network are calibrated using an attention mechanism to obtain calibrated image depth features; the calibrated image depth features are fused with the original facial image to construct a multimodal attention fusion network, and the multimodal attention fusion network is trained according to the data features in the second image data and the loss function to obtain a skin area segmentation model.

[0117] In an example, the expressions of the feature extraction function and the loss function are:

[0118]

[0119] Among them, M c (F) represents the feature extraction function, F represents the feature map of the input attention mechanism module, σ() represents the activation function, MLP() represents the built-in optimization function of the three-layer perceptron, AvgPool() represents the average pooling function, W0 represents the first weight matrix, W1 represents the second weight matrix, represents the feature after average pooling, L total represents the loss function, α represents the weight loss parameter, q represents the true label value output by the model, p represents the predicted value output by the model, δ represents the marginal adjustment parameter, N represents the total number of samples input to the model, n represents the nth sample, ω represents the hyperparameter, and q n Represents the true label value of the nth sample, p n Represents the predicted value of the nth sample.

[0120] In one example, the information extraction module 12 is used to construct a skin optical model based on the skin color density function, and introduce the skin optical model and the predicted pure color density vector into the skin optical model to obtain a synthetic skin color density function; based on the synthetic skin color density function and the least squares method, the melanin channel and the hemoglobin channel in the image of the area to be treated are solved respectively.

[0121] In one example, the expression of the composite skin color density function is:

[0122]

[0123] Among them, I x,y represents the skin color density function, x represents the horizontal coordinate of the pixel point, y represents the horizontal coordinate of the pixel point, c m Represents the pure color characteristics per unit density of melanin, c h Represents the pure color characteristic of hemoglobin per unit density, Represents the relative amount of melanin in each pixel. represents the relative amount of hemoglobin in each pixel, Δh represents the stationary column vector generated by other skin pigments and structures, and S x,y represents the synthetic skin color density function, represents the predicted pure color density vector, K represents the first density control parameter, and j represents the second density control parameter.

[0124] In one example, a random forest classifier includes a feature integration unit, a data set partitioning unit, a decision tree construction unit, a multiple decision tree training unit, and a classification prediction unit, wherein:

[0125] The feature integration unit is used to integrate the repair features collected in the skin treatment segmentation map and form a repair feature data set;

[0126] The data set division unit is used to divide the restoration feature data set into a plurality of different restoration feature sub-data sets by random sampling;

[0127] The decision tree building unit is used to generate decision trees and perform recursive splitting;

[0128] The multiple decision tree training unit is used to establish different decision trees on different restoration feature sub-datasets;

[0129] The classification prediction unit is used to classify samples and output the skin treatment type corresponding to each image of the area to be treated.

[0130] In one example, skin treatment types include pigmentary disorders, vascular disorders, keratosis pilaris, acne and its sequelae, and skin texture problems.

[0131] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for identifying local areas of intense pulsed light skin beauty, characterized in that: The method comprises: Collecting the original facial image of the patient and obtaining a historical skin treatment segmentation dataset, preprocessing the original facial image to obtain a standard facial image; Based on the image feature information in the skin treatment segmentation data set, a skin area division model is constructed, the facial standard image is input into the skin area division model for target recognition, and a plurality of images of areas to be treated in the facial standard image are obtained; Performing independent component analysis on the image of the area to be treated to obtain a melanin channel and a hemoglobin channel in the image of the area to be treated; Constructing a multimodal attention network model, and inputting the melanin channel and the hemoglobin channel into the multimodal attention network model for feature extraction and segmentation, to obtain melanin segmentation information and hemoglobin segmentation information respectively; The melanin segmentation information and the hemoglobin segmentation information are fused to obtain a skin treatment segmentation map, and the repair features in the skin treatment segmentation map are collected. The repair features are classified according to a random forest classifier to obtain the skin treatment type corresponding to each image of the area to be treated, and a treatment plan corresponding to the skin treatment type is given.

2. The method according to claim 1, characterized in that The preprocessing of the original facial image to obtain a standard facial image specifically includes: Converting the original facial image into RBG format, and adjusting the facial area in the original facial image to a horizontal state to obtain a facial transition image; The facial transition image is sequentially subjected to color correction, noise removal, illumination balance and detail enhancement processing to obtain a facial standard image.

3. The method according to claim 2, characterized in that The detail enhancement specifically includes: Segmenting the facial transition image after color correction, noise removal and illumination equalization to obtain a facial transition segmentation map consisting of a plurality of segmentation blocks; Obtaining a single-channel grayscale image of each segmentation block in the facial transition segmentation image, obtaining the grayscale level of each pixel in each segmentation block according to a grayscale level standard, and treating the pixel points whose grayscale level is lower than a preset level threshold as under-enhanced pixel points; The pixel value of the pixel point at the corresponding position of the under-enhanced pixel point in the facial transition image is adjusted to the corresponding expected pixel value to obtain the facial standard image, wherein the expected pixel value is obtained based on the grayscale level of the corresponding pixel point, the total number of pixel values ​​of the segmented block where the corresponding pixel point is located, and the total number of grayscale levels.

4. The method according to claim 1, characterized in that The step of constructing a skin area segmentation model based on the image feature information in the skin treatment segmentation dataset specifically includes: Acquire unimodal image data and multimodal image data in a skin treatment segmentation data set, divide the unimodal image data and the multimodal image data into first image data and second image data at a preset ratio, and extract data features in the first image data and the second image data based on a feature extraction function; Based on the data features in the first image data, construct an initial multimodal attention fusion network, and use multi-scale feature fusion to increase the receptive field of the initial multimodal attention fusion network; Extracting image depth features from the initial multimodal attention fusion network, and calibrating the image depth feature channels in the initial multimodal attention fusion network using an attention mechanism to obtain calibrated image depth features; The depth features of the calibrated image are fused with the original facial image to construct a multimodal attention fusion network, and the multimodal attention fusion network is trained according to the data features and loss function in the second image data to obtain the skin area division model.

5. The method according to claim 4, characterized in that The expressions of the feature extraction function and the loss function are respectively: Among them, M c (F) represents the feature extraction function, F represents the feature map of the input attention mechanism module, σ() represents the activation function, MLP() represents the built-in optimization function of the three-layer perceptron, AvgPool() represents the average pooling function, W0 represents the first weight matrix, W1 represents the second weight matrix, represents the feature after average pooling, L total represents the loss function, α represents the weight loss parameter, q represents the true label value output by the model, p represents the predicted value output by the model, δ represents the marginal adjustment parameter, N represents the total number of samples input to the model, n represents the nth sample, ω represents the hyperparameter, and q n Represents the true label value of the nth sample, p n Represents the predicted value of the nth sample.

6. The method according to claim 1, characterized in that The performing independent component analysis on the image of the area to be treated to obtain a melanin channel and a hemoglobin channel in the image of the area to be treated specifically includes: Based on the skin color density function, a skin optical model is constructed, and the skin optical model and the predicted pure color density vector are introduced into the skin optical model to obtain a synthetic skin color density function; Based on the synthetic skin color density function and the least square method, the melanin channel and the hemoglobin channel in the image of the area to be treated are solved respectively.

7. The method according to claim 6, characterized in that The expression of the synthetic skin color density function is: Among them, I x,y represents the skin color density function, x represents the horizontal coordinate of the pixel point, y represents the horizontal coordinate of the pixel point, c m Represents the pure color characteristics per unit density of melanin, c h Represents the pure color characteristic of hemoglobin per unit density, Represents the relative amount of melanin in each pixel. represents the relative amount of hemoglobin in each pixel, Δh represents the stationary column vector generated by other skin pigments and structures, and S x,y represents the synthetic skin color density function, represents the predicted pure color density vector, K represents the first density control parameter, and j represents the second density control parameter.

8. The method according to claim 1, characterized in that The random forest classifier includes a feature integration unit, a data set division unit, a decision tree construction unit, a multiple decision tree training unit and a classification prediction unit, wherein: The feature integration unit is used to integrate the repair features collected in the skin treatment segmentation map and form a repair feature data set; The data set division unit is used to divide the restoration feature data set into a plurality of different restoration feature sub-data sets by random sampling; The decision tree construction unit is used for performing decision tree generation and recursive splitting; The multiple decision tree training unit is used to establish different decision trees on different restoration feature sub-data sets respectively; The classification prediction unit is used to perform sample classification and output the skin treatment type corresponding to each image of the area to be treated.

9. The method according to claim 1, characterized in that The skin treatment types include pigmentary diseases, vascular diseases, keratosis pilaris, acne and its sequelae, and skin texture problems.

10. An intense pulsed light skin beauty local area recognition system, characterized in that: The local area operation system (1) comprises a data acquisition module (11), an information extraction module (12) and a treatment operation module (13), wherein: The data acquisition module (11) is used to acquire the original facial image of the patient and obtain a historical skin treatment segmentation data set, pre-process the original facial image, and obtain a standard facial image; The information extraction module (12) is used to construct a skin area division model based on the image feature information in the skin treatment segmentation data set, input the facial standard image into the skin area division model for target recognition, obtain multiple images of the area to be treated in the facial standard image, perform independent component analysis on the image of the area to be treated, obtain the melanin channel and the hemoglobin channel in the image of the area to be treated, construct a multimodal attention network model, and input the melanin channel and the hemoglobin channel into the multimodal attention network model for feature extraction and segmentation, and obtain melanin segmentation information and hemoglobin segmentation information respectively; The treatment operation module (13) is used to fuse the melanin segmentation information and the hemoglobin segmentation information to obtain a skin treatment segmentation map, collect repair features in the skin treatment segmentation map, classify the repair features according to a random forest classifier, obtain the skin treatment type corresponding to each image of the area to be treated, and provide a treatment plan corresponding to the skin treatment type.

Images