Image Processing-Based Peristostomy Skin Analysis Method and System

Abstract

This application relates to the field of stoma skin analysis technology, and discloses a method and system for analyzing skin around the stoma based on image processing. First, it acquires and preprocesses an original image containing the stoma, surrounding skin, and part of the stoma bag base. Next, it accurately separates the skin and base regions through multi-objective semantic segmentation. Then, using the base as a chance reference, it performs color calibration on the image by comparing its actual observed color with a standard color to eliminate differences in lighting and equipment. Finally, it extracts quantified color and texture features from the calibrated skin region to form a feature vector, and uses this vector to perform skin condition analysis and generate a report. In this way, without the need for an additional color chart, it effectively solves the problem of color consistency in image analysis in a home environment by utilizing inherent elements within the image for color calibration, achieving automated, objective, and standardized assessment of the skin condition around the stoma.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of stoma skin analysis, and more particularly, to a stoma surrounding skin analysis method and system based on image processing. BACKGROUND

[0002] An intestinal stoma is a drainage opening set on the abdominal wall of a patient after a partial surgical operation, and the patient needs to wear a stoma bag for a long time. Due to factors such as excrement stimulation and base plate damage, the skin around the stoma is prone to complications such as dermatitis, which seriously affects the quality of life of the patient. Therefore, timely and accurate monitoring of the skin around the stoma is crucial for guiding daily care and preventing complications. At present, the evaluation of the skin in this area mainly relies on patient self-examination or visual inspection by nurses, which is highly subjective, lacks quantitative standards, and is difficult to record and track standardized, especially for home patients, it is difficult to discover early skin deterioration in time. Therefore, developing an objective and convenient quantitative analysis scheme has become an urgent technical demand.

[0003] Using a portable device such as a smart phone to collect images and combining computer vision technology for analysis provides the possibility for automated health monitoring. However, there are two major technical challenges in applying general image processing techniques to this specific scenario, resulting in no mature solution in existing technology. First, the composition of the stoma area image is complex, and it contains multiple targets such as stoma mucosa, surrounding skin, and stoma bag base plate. If these areas cannot be accurately segmented, subsequent skin feature analysis will be impossible. The second, and more critical challenge, is the color consistency problem. Images taken by patients in uncontrolled environments such as homes are severely affected by lighting conditions and shooting devices, resulting in a huge color difference in the same skin problem in different images, making quantitative analysis based on color (such as evaluating the degree of skin redness) extremely unreliable. Although standard color cards can be used for color calibration in professional medical imaging, it is cumbersome to require patients to routinely use color cards in daily selfie monitoring, and the compliance is poor, which does not have the feasibility of practical promotion. SUMMARY

[0004] In view of the problems of strong subjectivity and lack of quantitative standards in the existing skin state evaluation of stoma care, and the difficulties in image segmentation and severe color distortion when applying existing image analysis techniques to this scenario, the embodiments of the present application propose a stoma surrounding skin analysis method and system based on image processing, thereby providing an objective and accurate home self-monitoring tool for stoma patients through a non-invasive and convenient technical means, helping them to discover potential skin complications in time, and providing reliable quantitative basis for remote guidance and precise nursing of medical staff, thereby improving the nursing quality and life quality of stoma patients.

[0005] According to an aspect of the present application, a method for peristomal skin analysis based on image processing is provided, comprising: obtaining an original image containing a stoma, peristomal skin and part of a pouch base plate; pre-processing the original image to obtain a pre-processed image; performing multi-target semantic segmentation on the pre-processed image to obtain a stoma mask, a peristomal skin mask and a base plate mask; performing color calibration based on a chance reference based on the standard base plate color and the base plate mask to obtain a calibrated image; performing skin region feature extraction on the calibrated image to obtain a skin feature vector; and performing skin state analysis based on the skin feature vector to obtain an analysis report.

[0006] In a possible implementation, the pre-processing of the original image to obtain the pre-processed image comprises: performing size standardization and denoising processing on the original image to obtain the pre-processed image.

[0007] In a possible implementation, the multi-target semantic segmentation on the pre-processed image to obtain the stoma mask, the peristomal skin mask and the base plate mask comprises: inputting the pre-processed image into a pre-trained multi-class semantic segmentation deep learning model to obtain the stoma mask, the peristomal skin mask and the base plate mask.

[0008] In a possible implementation, the pre-trained multi-class semantic segmentation deep learning model is a U-Net network with an attention mechanism.

[0009] In a possible implementation, the color calibration based on a chance reference based on the standard base plate color and the base plate mask to obtain the calibrated image comprises: performing color space conversion on the pre-processed image to obtain a LAB image; extracting a reference pixel set from the LAB image based on the base plate mask to obtain an original base plate pixel set; performing stable observation color calculation on the original base plate pixel set to obtain an observed base plate color; performing color calibration gain parameter calculation based on the observed base plate color and the standard base plate color to obtain a calibration gain set; and performing full-image pixel-level color correction and color space inverse transformation on the LAB image based on the calibration gain set to obtain the calibrated image.

[0010] In a possible implementation, the stable observation color calculation on the original base plate pixel set to obtain the observed base plate color comprises: extracting only the luminance channel values of all pixels from the original base plate pixel set to obtain a luminance value list; sorting the luminance value list and calculating the 5% quantile and the 95% quantile of the luminance value list to obtain an effective luminance interval; performing luminance effective interval screening on the original base plate pixel set based on the effective luminance interval to obtain a filtered base plate pixel set; and calculating the arithmetic mean values of the L channel, the A channel and the B channel of all pixels in the filtered base plate pixel set to obtain the observed base plate color.

[0011] In a possible implementation, the stable observed color calculation on the original baseplate pixel set to obtain an observed baseplate color comprises: extracting only the luminance channel values of all pixels from the original baseplate pixel set to obtain a luminance value list; sorting the luminance value list and determining an effective luminance interval based on the relative probability of the median and mean of the luminance values and the absolute deviation distribution thereof; performing luminance effective interval screening on the original baseplate pixel set based on the effective luminance interval to obtain a filtered baseplate pixel set; and calculating the arithmetic mean of the L channel, the A channel and the B channel of all pixels in the filtered baseplate pixel set to obtain the observed baseplate color.

[0012] In a possible implementation, the color calibration gain parameter calculation based on the observed baseplate color and the standard baseplate color to obtain a calibration gain set comprises: performing color calibration gain parameter calculation to obtain the calibration gain set by using the following formula:

[0013]

[0014]

[0015]

[0016] wherein, , and comprise the standard baseplate color, , and comprise the observed baseplate color, , and comprise the calibration gain set.

[0017] In a possible implementation, the skin region feature extraction on the calibrated image to obtain a skin feature vector comprises: converting the calibrated image to a CIELAB space to obtain a calibrated LAB image; extracting a calibrated skin pixel set from the calibrated LAB image based on a peristomal skin mask; and extracting color features and texture features from the calibrated skin pixel set to obtain the skin feature vector.

[0018] According to another aspect of the present application, there is provided an image processing based peristomal skin analysis system, comprising: an image acquisition module configured to acquire an original image containing a stoma, peristomal skin and part of a pouch base plate; an image preprocessing module configured to preprocess the original image to obtain a preprocessed image; a multi-target semantic segmentation module configured to perform multi-target semantic segmentation on the preprocessed image to obtain a stoma mask, a peristomal skin mask and a base plate mask; an image color calibration module configured to perform a chance reference based color calibration on the preprocessed image based on a standard base plate color and the base plate mask to obtain a calibrated image; a skin feature extraction module configured to perform skin region feature extraction on the calibrated image to obtain a skin feature vector; and a skin state analysis module configured to perform skin state analysis based on the skin feature vector to obtain an analysis report.

[0019] Compared with the prior art, the image processing based peristomal skin analysis method and system provided by the present application firstly introduces a color calibration mechanism based on the pouch base plate as a chance reference, greatly improving the accuracy and reliability of skin color analysis, making the analysis results obtained at different times, under different lighting and different devices comparable, and effectively overcoming the defect that the prior art cannot perform effective quantitative analysis due to color distortion. Secondly, the multi-target semantic segmentation technology is adopted to realize pixel-level accurate extraction of the peristomal skin region, avoiding the interference of the stoma mucosa or the base plate region on the skin feature analysis, and ensuring the purity of the analysis object. Thirdly, the entire analysis process converts subjective human eye observation into objective computer quantitative analysis, outputs a standardized skin feature vector and an analysis report, and provides reliable data support for long-term tracking and change trend warning of the skin state. Finally, the present application does not require the user to use any additional auxiliary tools (such as a color card), and can be completed by only one daily photographed photo, the operation process is extremely simple, and the practicability and user compliance of the scheme are greatly improved. BRIEF DESCRIPTION OF DRAWINGS

[0020] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description of embodiments of the present application taken in conjunction with the accompanying drawings. The drawings provided in the present application are used to provide a further understanding of embodiments of the present application, and constitute a part of the specification, and are used to explain the present application together with embodiments of the present application, and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0021] Figure 1 FIG. 1 illustrates a schematic flowchart of an image processing based peristomal skin analysis method according to an embodiment of the present application.

[0022] Figure 2Fig. 6 shows a schematic flow chart of step S4 in the image processing based peristomal skin analysis method according to an embodiment of the present application.

[0023] Figure 3 Fig. 7 shows a schematic flow chart of step S43 in the image processing based peristomal skin analysis method according to an embodiment of the present application.

[0024] Figure 4 Fig. 8 shows a schematic flow chart of step S5 in the image processing based peristomal skin analysis method according to an embodiment of the present application.

[0025] Figure 5 Fig. 9 shows a schematic block diagram of the image processing based peristomal skin analysis system according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In the following, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of embodiments of the present application, rather than all embodiments of the present application. It should be understood that the present application is not limited to the described example embodiments.

[0027] Figure 1 Fig. 1 shows a schematic flow chart of the image processing based peristomal skin analysis method and system according to an embodiment of the present application. As shown in Fig. 1, the present application provides an image processing based peristomal skin analysis method, which comprises the following steps. Figure 1 S1, obtaining an original image containing a stoma, peristomal skin and part of a pouch base plate; S2, pre-processing the original image to obtain a pre-processed image; S3, performing multi-target semantic segmentation on the pre-processed image to obtain a stoma mask, a peristomal skin mask and a base plate mask; S4, performing chance reference based color calibration on the pre-processed image based on a standard base plate color and the base plate mask to obtain a calibrated image; S5, performing skin region feature extraction on the calibrated image to obtain a skin feature vector; and S6, performing skin state analysis based on the skin feature vector to obtain an analysis report.

[0028] Exemplarily, in step S1, an original image containing the stoma, the surrounding skin and part of the ostomy bag base is acquired. That is, by a simple shooting action, all the visual information necessary for subsequent intelligent analysis is captured completely. Specifically, the acquired image not only needs to contain the analyzed stoma surrounding skin, but also must contain the stoma itself and the ostomy bag base as a color calibration key reference at the same time. Only when these three elements are obtained in one image, the subsequent multi-object segmentation and color calibration based on opportunistic reference can be realized, thereby providing a reliable original data source for the final generation of objective and accurate skin analysis report. Specifically, in the present application, the original image refers to the digital image data generated directly by the image acquisition device without any algorithm processing. This original image must meet specific requirements in content, that is, the three key target areas must be clearly presented in its field of view: one is the stoma, that is, the opening of the intestinal tract on the abdominal wall; the second is the surrounding skin, which refers to the skin covered by the ostomy bag base and any auxiliary fixing tape within 3-4 inches (about 10 cm) around the stoma according to the World Confederation for Enterostomy Therapists (WCET) guidelines; the third is part of the ostomy bag base, that is, part of the annular adhesive used to fix the ostomy bag and directly contact with the skin.

[0029] Specifically, the user can hold an electronic device with a photographing function, aim the camera at the stoma area of himself / herself in a suitable lighting environment. The user needs to adjust the shooting distance and angle to ensure that the aforementioned three targets of stoma, surrounding skin and part of ostomy bag base are completely and clearly captured in the lens frame, and then trigger the shooting function to complete the acquisition of the image. This process does not require the user to master professional photography skills, nor does it require the use of any external auxiliary equipment such as color cards, special light sources, etc., aiming to minimize the user's use threshold, so that he / she can easily perform daily self-monitoring in a home environment.

[0030] In one specific embodiment, the user uses a smart phone to take a picture of the stoma area under normal indoor lighting conditions. When shooting, ensure that the photo clearly contains the following three parts: 1) the stoma itself; 2) the stoma surrounding skin, ensuring that the photo clearly covers a sufficient analysis area (e.g. 2-3 cm) extending outward from the stoma edge; 3) part of the ostomy bag base adhered to the skin. Finally, an original image in RGB format is obtained.

[0031] Exemplarily, in step S2, the original image is pre-processed to obtain a pre-processed image. It should be understood that different smartphone cameras have different resolutions, and the distance between the user and the stoma area when taking a picture is also different, which will result in different sizes of the original image; at the same time, in the home lighting environment, the image sensor may introduce random electronic noise. These inconsistent sizes and noise interference will seriously affect the performance and stability of the subsequent deep learning model algorithm, therefore, the original image is standardized and purified through the pre-processing step to eliminate the interference of these uncertain factors.

[0032] In one embodiment, the original image is pre-processed to obtain a pre-processed image, including: performing size standardization and denoising processing on the original image to obtain the pre-processed image. Specifically, first, the size of all input images is standardized to a fixed specification. Size standardization can be achieved by image scaling or cropping algorithm, ensuring that the input dimension received by the subsequent processing module is consistent. Then, denoising processing is performed on the size-standardized image. The purpose of denoising processing is to smooth the image and reduce random noise pixels introduced by the sensor or lighting conditions. A common implementation is to apply an image filter, such as a Gaussian filter, which suppresses noise by performing a weighted average of each pixel and its neighborhood, while preserving the main edges and structural information in the image as much as possible.

[0033] In a specific embodiment, the original image is pre-processed to obtain a pre-processed image, including: first, size standardization: uniformly scaling or cropping the original image to a fixed size of 512x512 pixels. Then, denoising processing: applying a Gaussian filter to the size-standardized image to smooth the image and reduce the interference of random noise. After the above processing, a pre-processed image with uniform size and less noise is obtained. This pre-processed image will be directly input to the next stage of multi-target semantic segmentation.

[0034] Exemplarily, in step S3, the pre-processed image is subjected to multi-target semantic segmentation to obtain a stoma mask, a peristomal skin mask, and a flange mask. It should be understood that the image content of the stoma area is complex, containing at least three targets that are significantly different in shape, color, and texture: stoma, surrounding skin, and flange. Without accurate segmentation, the subsequent color calibration step will not be able to accurately find the flange area as a reference, and the skin feature extraction step may also incorrectly include pixels of the stoma mucosa or flange in the calculation, resulting in serious deviation of the entire analysis result. Therefore, through multi-target semantic segmentation, different areas mixed together in the image can be clearly separated, providing pure and accurate target areas for each subsequent analysis step.

[0035] In one embodiment, the pre-processed image is subjected to multi-target semantic segmentation to obtain the stoma mask, the peristomal skin mask and the flange mask, including: inputting the pre-processed image into a pre-trained multi-class semantic segmentation deep learning model to obtain the stoma mask, the peristomal skin mask and the flange mask, and the pre-trained multi-class semantic segmentation deep learning model is a U-Net network with attention mechanism.

[0036] Specifically, the U-Net network with attention mechanism is based on a classic U-Net architecture, which contains an encoder (shrinkage path) for feature extraction and a decoder (expansion path) for precise localization. The encoder gradually reduces the spatial dimensions of the feature map while increasing its channel number through a series of convolutional layers and max-pooling layers, in order to capture the context and high-level semantic information of the image. The decoder then gradually recovers the spatial resolution of the feature map through a series of up-sampling (or transposed convolution) layers and convolutional layers. The key to the U-Net architecture lies in its skip-connection design, which directly concatenates the feature maps of the corresponding levels in the encoder to the feature maps of the decoder, thereby combining high-resolution shallow features with high-semantic deep features, greatly improving the precision of segmentation. The attention mechanism used in this application is an improvement on the standard U-Net, which introduces an attention gate module in the skip connection. This module will automatically learn to generate an attention weight map based on the high-level semantic information of the decoder before concatenating the feature maps, and apply it to the shallow feature maps of the encoder. This process enables the model to pay more attention to the target regions related to the current segmentation task when fusing features, while suppressing the feature responses of irrelevant regions such as the background, thereby further improving the accuracy and robustness of segmentation.

[0037] It should be understood that the U-Net network with attention mechanism needs to be trained before being applied to inference. The specific training process includes: first, construct a large-scale image dataset that is accurately labeled by medical experts with stoma, peristomal skin and flange regions. Then, input the dataset into the network for supervised learning, and use a composite loss function such as the Dice loss function combined with the cross-entropy loss function to measure the difference between the model's prediction results and the expert's labeling (i.e. the "gold standard"). By using optimizers such as Adam, the network's internal weight parameters are continuously iteratively adjusted through the backpropagation algorithm until the model's performance on the validation set converges. After sufficient training, the model has the ability to automatically and accurately perform multi-target semantic segmentation on new images. When a pre-processed image is input into the trained model, the model will perform a forward inference, and finally output the probability of each pixel belonging to the three categories. By thresholding the probability map, the three binary masks required can be generated.

[0038] To more specifically illustrate the implementation of the multi-target semantic segmentation step, the present application provides a specific embodiment which adopts a pre-trained U-Net network model with attention mechanism as the multi-class semantic segmentation deep learning model. The pre-processed image is input into the model, and the model will output three independent binary masks (Masks), which are: 1. Stoma mask: in this mask, the pixel value corresponding to the stoma area in the original image is 1, and the rest is 0. 2. Peristomal skin mask: in this mask, the pixel value corresponding to the peristomal skin area is 1, and the rest is 0. 3. Base plate mask: in this mask, the pixel value corresponding to the base plate area of the pouch is 1, and the rest is 0. These three masks will be passed to the subsequent steps for color calibration and skin feature extraction, respectively.

[0039] Exemplarily, in step S4, the pre-processed image is subjected to chance reference-based color calibration based on the standard base plate color and the base plate mask to obtain a calibrated image. It should be understood that since different lighting conditions and shooting devices can cause image color distortion, which can seriously affect subsequent color-based quantitative analysis. For example, the degree of redness of the skin is an important indicator for judging inflammation, but the image color of healthy skin taken under warm-toned light may be redder than that of mildly inflamed skin taken under cool-toned light. Such inconsistency in color makes images taken under different conditions incomparable, thus making automated longitudinal disease tracking meaningless. The present application introduces a color calibration mechanism without the need for additional auxiliary tools, aiming to eliminate these color deviations caused by the environment and devices, and restore the true color information of the skin area.

[0040] In one embodiment, as shown in Figure 2 S41, performing color space conversion on the pre-processed image to obtain a LAB image; S42, performing reference object pixel set extraction on the LAB image based on the base plate mask to obtain an original base plate pixel set; S43, performing stable observation color calculation on the original base plate pixel set to obtain an observed base plate color; S44, performing color calibration gain parameter calculation based on the observed base plate color and the standard base plate color to obtain a calibration gain set; S45, performing full-image pixel-level color correction and color space inverse transformation on the LAB image based on the calibration gain set to obtain the calibrated image.

[0041] Specifically, first, the pre-processed image is converted to LAB color space. Converting the image from device-dependent RGB color space to CIELAB color space, which is more consistent with human visual perception and separates luminance from chrominance, is a prerequisite for effective color correction. Second, the reference pixel set is extracted from the LAB image based on the mask to obtain the original base pixel set. Using the base mask generated in the previous step, all pixels belonging to the base area can be accurately extracted from the LAB image to form a data set for analysis.

[0042] Next, the original base pixel set is subjected to stable observation color calculation to obtain the observed base color. It should be understood that the stable observation color calculation can calculate a value that can stably represent the color of the base under the current lighting from the base pixels that may contain reflections and shadows. Specifically, as shown in Figure 3 The stable observation color calculation of the original base pixel set to obtain the observed base color includes: S431, extracting only the luminance channel values of all pixels from the original base pixel set to obtain a luminance value list; S432, sorting the luminance value list and calculating the 5% quantile and 95% quantile of the luminance value list to obtain the effective luminance interval; S433, based on the effective luminance interval, performing luminance effective interval filtering on the original base pixel set to obtain a filtered base pixel set; S434, calculating the arithmetic mean of the L channel, A channel and B channel of all pixels in the filtered base pixel set to obtain the observed base color.

[0043] Preferably, when defining the effective luminance interval, instead of using a fixed quantile method, a dynamic, data-driven method based on robust statistics can be used to define the effective luminance interval based on the relative probability of the median and mean of the luminance values and their absolute deviation distribution. That is, since the threshold of the fixed quantile is set by humans, there may be a weak correlation with the intrinsic properties of the image data, for example, in images with very uniform lighting, removing 10% of the data may be too aggressive and will discard valid pixels; while in images with large soft shadows (such as covering 20% of the base area), removing only 5% of the dark data may not be enough, because these shadow pixels will significantly lower the average value.

[0044] And because it cannot be adjusted according to the specific lighting conditions of each image, it lacks adaptability, and the robust method should be able to adaptively adjust its filtering range according to the actual distribution of the base area luminance values. And there is a vulnerability to skew distribution, for example, a strong highlight point caused by a flash produces a small number of extreme brightness pixels (forming a long tail on the right side of the distribution), then the 95% quantile itself may be a very high and unrepresentative value, that is, the interval center deviates from the most reliable data cluster.

[0045] That is, in another embodiment, the stable observed color calculation on the original palette pixel set to obtain the observed palette color comprises: extracting only the luminance channel values of all pixels from the original palette pixel set to obtain a luminance value list; sorting the luminance value list and determining an effective luminance interval based on the relative probability of the median and mean of the luminance values and the absolute deviation distribution thereof; performing luminance effective interval filtering on the original palette pixel set based on the effective luminance interval to obtain a filtered palette pixel set; and calculating the arithmetic mean of the L channel, A channel and B channel of all pixels in the filtered palette pixel set to obtain the observed palette color.

[0046] Specifically, the effective luminance interval is determined based on the relative probability of the median and mean of the luminance values and the absolute deviation distribution thereof, comprising: first, for the sorted luminance value list, obtaining the median Lmedian, calculating the mean Lmean thereof, and calculating the relative distribution Lrel=Lmedian / Lmean thereof, which has a high resistance to extreme values at both ends of the distribution.

[0047] Then, the robust dispersion degree of the luminance value distribution is measured, i.e. the distribution range of the effective data around the robust center Lmedian is quantified, for example, each luminance value is Li, and its absolute deviation di=|Li-Lmedian| from the median Lmedian is calculated, and then the median of the absolute deviation MAD is further determined, which represents the typical deviation degree of the data from the median.

[0048] Finally, the dynamic luminance interval is defined as Lmedian±k*Lrel*MAD, where k is a sensitivity parameter, for example, k=2 or k=3 is selected, or according to the distribution form, for example, for a normal distribution, the standard deviation σ is estimated by MAD as σ≈1.4826*MAD, therefore, selecting k=3 is roughly equivalent to the rule of μ±2σ, which is an effective way to filter out abnormal values while retaining most of the effective data, that is, k=3 can be selected to construct a robust filter. At the same time, the relative probability distribution of the relative probability of the median and mean of the relative probability of the median and mean is used to determine the effective deviation between the sequence sample distribution center and the numerical value distribution center, so as to maximize the consistency of the sequence sample distribution and the numerical value distribution.

[0049] That is, by using the median as a robust indicator to measure the central tendency, the influence of extreme outliers is reduced by the probability distribution relative to the mean, so that even in the presence of strong highlights or deep shadows, the luminance value of the most representative and normal part of the light on the palette can be reliably located, and by the absolute deviation of the median, the robustness of the data dispersion degree is measured for quantifying the distribution range of the data around the median, so as to define a dynamic luminance space to locate the center on the most reliable data, and its width is proportional to the actual variation degree of this part of reliable data.

[0050] Thus, the computed observed palette color will more accurately reflect the true color of the palette under the current ambient light, as it is derived from robustly selected pixels, while the system will perform more stably under a wider range of adverse lighting conditions (e.g. hard shadows cast by hands, specular highlights caused by windows or flashlights), making the final color calibration less likely to fail or produce severe errors. In addition, for photos taken at different times for the same patient, even if there is a difference in lighting quality, the computed observed palette color will be more stable and consistent, making long-term tracking analysis of skin color more reliable.

[0051] Subsequently, color calibration gain parameter computation is performed based on the observed palette color and the standard palette color to obtain a calibration gain set, including: performing color calibration gain parameter computation to obtain the calibration gain set with the following formula:

[0052]

[0053]

[0054]

[0055] wherein, , and comprise the standard palette color, , and comprise the observed palette color, , and comprise the calibration gain set.

[0056] Finally, full-image pixel-level color correction and color space inverse transformation are performed on the LAB image based on the calibration gain set to obtain the calibrated image. The computed calibration gain set is applied to each pixel of the entire LAB image, i.e. the (L, A, B) value of any pixel is transformed. After all pixels are corrected, the corrected LAB image is converted back to the general RGB space through color space inverse transformation, and the final result is the color-accurate calibrated image.

[0057] Exemplarily, in step S5, the calibrated image is subjected to skin region feature extraction to obtain a skin feature vector. It should be appreciated that although the calibrated image visually reflects the skin state, it is essentially a high-dimensional pixel matrix, which cannot be directly understood and processed by the subsequent machine learning classification model. The computer model needs structured and low-dimensional numerical input. Therefore, through the feature extraction process, the key visual information related to the skin health condition in the image is refined and quantified, and it is converted from a qualitative image representation to a set of quantitative numerical descriptors with clear physical or statistical meaning.

[0058] In one embodiment, as shown in FIG. 5, the skin feature vector is obtained by performing skin region feature extraction on the calibrated image, comprising: Figure 4 S51, converting the calibrated image to CIELAB space to obtain a calibrated LAB image; S52, extracting a calibrated skin pixel set from the calibrated LAB image based on the peristomal skin mask; S53, extracting color features and texture features from the calibrated skin pixel set to obtain the skin feature vector.

[0059] Specifically, first, the calibrated image is converted to CIELAB space to obtain a calibrated LAB image. Although the image has undergone LAB space transformation in the color calibration step, it is necessary to enter this color space again for feature extraction, because CIELAB space separates luminance and chrominance, making the calculation of color features more stable and consistent with human perception. Second, a calibrated skin pixel set is extracted from the calibrated LAB image based on the peristomal skin mask. Using the high-precision skin mask generated in the previous step, all pixels belonging to the peristomal skin region can be accurately separated from the calibrated LAB image like using a template, forming a pixel data pool for calculation, excluding any interference from non-skin regions. Finally, color features and texture features are extracted from the calibrated skin pixel set to obtain the skin feature vector. In this sub-step, a series of mathematical and statistical calculations will be performed on the extracted skin pixel set. For color features, statistical quantities such as mean and standard deviation of the pixel set in L, a, b channels are usually calculated, which reflect the average brightness, red-green / yellow-blue bias and color dispersion of the skin, respectively. For texture features, algorithms such as gray level co-occurrence matrix (GLCM) are used to analyze the spatial distribution and correlation of pixel gray levels, and calculate descriptors such as contrast, energy, homogeneity, etc. All the calculated numerical values are finally combined to form a multi-dimensional skin feature vector.

[0060] To further illustrate the implementation of skin region feature extraction on calibrated images, the present application provides a specific embodiment of quantitative feature extraction on peristomal skin region on color calibrated images. 1. Convert the calibrated image (in RGB format) obtained in step S4 to CIELAB space again to obtain a calibrated LAB image. 2. Extract all pixels belonging to skin region from the calibrated LAB image using the peristomal skin mask obtained in step S3 to form a calibrated skin pixel set. 3. Extract the following color and texture features from the calibrated skin pixel set: Color features: Calculate the mean and standard deviation of L, a, b channels of all pixels in the pixel set. The mean of a can reflect the redness of the skin. Texture features: Calculate the gray level co-occurrence matrix (GLCM) of L channel of the pixel set, and extract texture descriptors such as contrast, energy, homogeneity, etc. based on this matrix to represent the smoothness or roughness of the skin. 4. Combine all extracted feature values into a multi-dimensional skin feature vector. This final generated feature vector will be directly input to the next stage of skin state analysis.

[0061] Exemplarily, in step S6, skin state analysis is performed based on the skin feature vector to obtain an analysis report. It should be understood that the simple feature vector (e.g. a set of numbers about color mean, standard deviation and texture contrast) is difficult for non-professional users or even medical staff to directly understand and use. Therefore, this step aims to use an intelligent decision model to automatically map these complex numerical features to explicit skin state categories with clinical significance, and present the analysis results in an intuitive and friendly way, thereby realizing the closed loop of the entire analysis process and providing direct and feasible references for users' daily care.

[0062] In one embodiment, the skin feature vector is input into a pre-trained classification model for skin condition analysis to obtain an analysis report. The present application preferably uses Support Vector Machine (SVM) as the classifier. The model structure of SVM aims to find an optimal classification hyperplane in a high-dimensional feature space, which can separate different classes of sample points with the largest margin. For the nonlinear separable problem that may exist in the present application, SVM is processed by kernel trick, for example, using radial basis function (RBF) kernel, which can implicitly map the original feature space to a higher-dimensional or even infinite-dimensional space, so as to find a linear classification hyperplane in this high-dimensional space. The training method of the model is a supervised learning process. First, a large-scale training data set needs to be constructed, which contains a large number of peristomal skin images, and each image is accurately labeled by a dermatologist or a senior stoma therapist (for example, "healthy", "mild maceration", "dermatitis", "breakage", etc.). Subsequently, each image in the labeled image data set will go through all the processing steps (preprocessing, segmentation, calibration, feature extraction) of the preface of the present application, thereby generating a skin feature vector-state label data pair corresponding to each image. After obtaining the training data set, the SVM model can be trained. The goal of training is to solve a quadratic programming problem to find the hyperplane parameters that can maximize the classification margin. This process usually also includes optimizing the hyperparameters (such as the penalty coefficient C and the parameter gamma of the RBF kernel) of the model by cross-validation and other methods to obtain the best generalization ability. After training is completed, the SVM model solidifies the decision logic from the skin feature vector to its clinical state. When a new feature vector is input, the model will use the learned hyperplane to make a decision to determine the class to which the vector belongs. After obtaining this classification result, the system will further integrate the information, combine the classification conclusion with some key original feature values, and automatically generate a structured analysis report.

[0063] In summary, the stoma peristomal skin analysis method based on image processing provided in the present application firstly introduces a color calibration mechanism based on the opportunistic reference of the stoma pouch base plate, greatly improving the accuracy and reliability of skin color analysis, making the analysis results obtained under different times, different lighting, and different devices comparable, effectively overcoming the defect that the prior art cannot perform effective quantitative analysis due to color distortion. Secondly, the multi-target semantic segmentation technology is used to realize the pixel-level accurate extraction of the peristomal skin area, avoiding the interference of the stoma mucosa or the base plate area on the skin feature analysis, and ensuring the purity of the analysis object. Thirdly, the entire analysis process converts subjective human eye observation into objective computer quantitative analysis, outputs standardized skin feature vectors and analysis reports, and provides reliable data support for long-term tracking and change trend warning of skin state. Finally, the present application does not require the user to use any additional auxiliary tools (such as color cards), and can be completed by only one daily photographed photo, the operation process is extremely simple, and the practicality and user compliance of the scheme are greatly improved.

[0064] The present application also provides a stoma peristomal skin analysis system based on image processing for executing the above-mentioned stoma peristomal skin analysis method based on image processing. Figure 5 As shown in the figure, the stoma peristomal skin analysis system 500 based on image processing includes an image acquisition module 510 for acquiring an original image containing a stoma, peristomal skin, and part of a stoma pouch base plate; an image preprocessing module 520 for preprocessing the original image to obtain a preprocessed image; a multi-target semantic segmentation module 530 for performing multi-target semantic segmentation on the preprocessed image to obtain a stoma mask, a peristomal skin mask, and a base plate mask; an image color calibration module 540 for performing color calibration based on an opportunistic reference based on the standard base plate color and the base plate mask to obtain a calibrated image; a skin feature extraction module 550 for performing skin region feature extraction on the calibrated image to obtain a skin feature vector; and a skin state analysis module 560 for performing skin state analysis based on the skin feature vector to obtain an analysis report.

[0065] The present application also provides a computer readable storage medium having computer program code stored therein, which, when executed on a computer, causes the computer to perform the above-mentioned related method steps to realize the stoma peristomal skin analysis method based on image processing provided in the above-mentioned embodiments.

[0066] The present application also provides a computer program product, which, when executed on a computer, causes the computer to perform the above-mentioned related steps to realize the stoma peristomal skin analysis method based on image processing provided in the above-mentioned embodiments.

[0067] The system, computer-readable storage medium or computer program product provided by the embodiments of the present application are used to execute the corresponding method provided above, thus the beneficial effects achieved thereby can refer to the beneficial effects of the corresponding method provided above, which will not be described here again.

[0068] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments.

[0069] The processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some implementations, multitasking and parallel processing can be advantageous. The various embodiments described herein are described using a progression of embodiments, with the description of each successive embodiment building on the previous embodiments, however, the scope of the embodiments is not limited to the described orders.

Claims

1. An image processing-based peristomal skin analysis method, characterized by, The method comprises: obtaining an original image containing a stoma, surrounding skin and part of a stoma pouch base plate; preprocessing the original image to obtain a preprocessed image; performing multi-target semantic segmentation on the preprocessed image to obtain a stoma mask, a stoma surrounding skin mask and a base plate mask; performing chance reference-based color calibration on the preprocessed image based on a standard base plate color and the base plate mask to obtain a calibrated image, comprising: performing color space conversion on the preprocessed image to obtain a LAB image; performing reference pixel set extraction on the LAB image based on the base plate mask to obtain an original base plate pixel set; performing stable observation color calculation on the original base plate pixel set to obtain an observed base plate color; performing color calibration gain parameter calculation based on the observed base plate color and the standard base plate color to obtain a calibration gain set; performing full-image pixel-level color correction and color space inverse transformation on the LAB image based on the calibration gain set to obtain the calibrated image; performing skin region feature extraction on the calibrated image to obtain a skin feature vector; performing skin state analysis based on the skin feature vector to obtain an analysis report.

2. The image processing based peristomal skin analysis method according to claim 1, characterized in that, The preprocessing of the original image to obtain a preprocessed image comprises: performing size standardization and denoising processing on the original image to obtain the preprocessed image.

3. The image processing based peristomal skin analysis method according to claim 2, characterized in that, The multi-target semantic segmentation on the preprocessed image to obtain a stoma mask, a stoma surrounding skin mask and a base plate mask comprises: inputting the preprocessed image into a pre-trained multi-class semantic segmentation deep learning model to obtain the stoma mask, the stoma surrounding skin mask and the base plate mask.

4. The image processing-based peristomal skin analysis method according to claim 3, characterized in that, The pre-trained multi-class semantic segmentation deep learning model is a U-Net network with an attention mechanism.

5. The image processing based peristomal skin analysis method of claim 1, wherein, The stable observation color calculation on the original base plate pixel set to obtain an observed base plate color comprises: extracting only the luminance channel values of all pixels from the original base plate pixel set to obtain a luminance value list; sorting the luminance value list and calculating the 5% quantile and the 95% quantile of the luminance value list to obtain an effective luminance interval; based on the effective luminance interval, performing luminance effective interval filtering on the original base plate pixel set to obtain a filtered base plate pixel set; calculating the arithmetic mean of the L channel, the A channel and the B channel of all pixels in the filtered base plate pixel set to obtain the observed base plate color.

6. The image processing based peristomal skin analysis method of claim 1, wherein, The stable observation color calculation on the original base plate pixel set to obtain an observed base plate color comprises: extracting only the luminance channel values of all pixels from the original base plate pixel set to obtain a luminance value list; sorting the luminance value list and determining an effective luminance interval based on the relative probability of the median and mean of the luminance values and the absolute deviation distribution thereof; based on the effective luminance interval, performing luminance effective interval filtering on the original base plate pixel set to obtain a filtered base plate pixel set; calculating the arithmetic mean of the L channel, the A channel and the B channel of all pixels in the filtered base plate pixel set to obtain the observed base plate color.

7. The image processing based peristomal skin analysis method of claim 1, wherein, The color calibration gain parameter calculation based on the observed base plate color and the standard base plate color to obtain a calibration gain set comprises: performing color calibration gain parameter calculation to obtain the calibration gain set with the following formula: wherein, , and comprise the standard chassis color, , and comprise the observed chassis color, , and comprise the calibration gain set.

8. The image processing based peristomal skin analysis method of claim 1, wherein, extracting a skin feature vector from the calibrated image, comprising: converting the calibrated image to CIELAB space to obtain a calibrated LAB image; extracting a calibrated skin pixel set from the calibrated LAB image based on the peristomal skin mask; extracting color features and texture features from the calibrated skin pixel set to obtain the skin feature vector.

9. An image processing-based peristomal skin analysis system, characterized by, comprising: an image acquisition module, configured to acquire an original image containing a stoma, surrounding skin, and part of a pouch base plate; an image preprocessing module, configured to preprocess the original image to obtain a preprocessed image; a multi-target semantic segmentation module, configured to perform multi-target semantic segmentation on the preprocessed image to obtain a stoma mask, a peristomal skin mask, and a base plate mask; an image color calibration module, configured to perform chance reference-based color calibration on the preprocessed image based on a standard base plate color and the base plate mask to obtain a calibrated image, comprising: performing color space conversion on the preprocessed image to obtain a LAB image; extracting a reference pixel set from the LAB image based on the base plate mask to obtain an original base plate pixel set; performing stable observation color calculation on the original base plate pixel set to obtain an observed base plate color; performing color calibration gain parameter calculation based on the observed base plate color and the standard base plate color to obtain a calibration gain set; performing full-image pixel-level color correction and color space inverse transformation on the LAB image based on the calibration gain set to obtain the calibrated image; a skin feature extraction module, configured to extract skin region features from the calibrated image to obtain a skin feature vector; a skin state analysis module, configured to perform skin state analysis based on the skin feature vector to obtain an analysis report.

Images