Multispectral imaging assisted early detection system for skin damage after radiotherapy

Abstract

The application discloses a multispectral imaging assisted post-radiotherapy skin damage early detection system, belongs to the medical image processing and computer vision technical field, and includes a multispectral data acquisition module, a deep tissue feature extraction module, a three-dimensional lesion segmentation module and a space-time tracking evaluation module. Subcutaneous 2.5cm depth tissue information is acquired through multispectral imaging of a 400-1350nm wave band, physiological parameters such as blood perfusion, melanin concentration and collagen structure are inversed based on a radiation transmission theory, a three-dimensional medical image segmentation algorithm is adopted to realize stereoscopic accurate segmentation of a damage area, a space-time evolution model is established to predict a damage development trend, and a closed loop feedback mechanism is adopted to realize adaptive optimization of the system. The application can detect radioactive skin damage in a subclinical period, can shorten a detection time window by 5.2 days on average, can reduce the incidence of severe dermatitis by 65%, and can provide a basis for timely clinical intervention.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of medical image processing and computer vision, and particularly relates to a multispectral imaging assisted early detection system for skin damage after radiotherapy, and particularly relates to a technical scheme for three-dimensional space modeling of skin tissue and accurate identification of lesion areas by using medical image segmentation technology. BACKGROUND

[0002] Radiotherapy is one of the important means of clinical treatment of tumors, and about 60% to 70% of tumor patients need to receive radiotherapy during treatment. However, radiotherapy inevitably causes damage to the skin of patients. Statistical data shows that more than 90% of patients receiving radiotherapy will have different degrees of radiation-induced skin damage, of which about 30% to 40% will develop into moderate to severe dermatitis, seriously affecting the quality of life of patients and possibly leading to treatment interruption. Early detection of radiation-induced skin damage is of great significance for timely intervention and prevention of disease deterioration.

[0003] In the prior art, the evaluation of skin damage after radiotherapy mainly relies on the naked eye observation and experience judgment of doctors, and subjective scoring is performed using RTOG or CTCAE grading standards. This method has the following disadvantages: first, naked eye observation can only identify visible changes on the surface of the skin, such as erythema and desquamation, and cannot detect microscopic pathological changes in the subcutaneous tissue; second, manual evaluation has great subjectivity and individual differences, and different doctors may have a difference of 1 to 2 grades in the evaluation of the same patient; third, when the patient has visible symptoms, the skin tissue has often already been damaged seriously, missing the best opportunity for intervention.

[0004] CN118379261A discloses a clinical skin image automatic acquisition method and system, which detects and evaluates the image quality of the skin lesion site through a deep learning network, realizes automatic focusing of the lesion area and high-quality image acquisition. However, this technology only processes visible light RGB images and cannot obtain information of deep skin tissue; the lesion detection model can only identify lesions that have formed obvious visual features on the surface of the skin and does not have early detection capability; in addition, this system lacks quantitative analysis of physiological parameters of skin tissue and cannot evaluate physiological indicators closely related to radiation damage such as blood perfusion, melanin distribution and collagen structure, making it difficult to meet the clinical needs of early warning of skin damage after radiotherapy.

[0005] In recent years, multispectral imaging technology has shown great potential in the field of skin disease diagnosis. Multispectral imaging can reveal the structure and physiological state of the tissue within a depth range of 2.5 cm below the skin by collecting the reflectance spectrum information of the skin at different wavelengths. Studies have shown that in the 400-1350 nm wavelength range, the absorption and scattering characteristics of hemoglobin, melanin, water, collagen and other pigment groups in the skin tissue are significantly different at different wavelengths. By analyzing the spectral response of specific bands, the concentration and distribution of these physiological parameters can be quantitatively evaluated. However, existing multispectral imaging devices are mainly used for the diagnosis of skin cancer, melanoma and other diseases, and have not been systematically optimized for early detection of skin damage after radiotherapy.

[0006] The pathological evolution process of radioactive skin damage has unique spatiotemporal characteristics: in the spatial dimension, the damage area is highly correlated with the distribution of the radiation field, showing a dose-dependent gradient change; in the time dimension, the damage develops from microscopic vascular dilation and inflammatory response in the subclinical period to macroscopic manifestations such as erythema, edema and desquamation in the clinical period. This pathological evolution process provides an important window period for early detection, but a detection model that can capture the spatiotemporal dynamic changes is needed. The existing technology generally lacks systematic analysis of the spatiotemporal evolution of skin damage, making it difficult to achieve true early warning.

[0007] In addition, existing technology mainly uses two-dimensional segmentation methods for image segmentation, which can only obtain the planar information of the skin surface and cannot construct a three-dimensional model of the skin tissue. Radioactive skin damage involves multi-level changes in the structure of the skin from the epidermis to the deep dermis, and two-dimensional image analysis cannot accurately assess the depth and volume of the damage, limiting the detection accuracy and clinical application value.

[0008] In summary, the existing technology has the following outstanding problems in the early detection of skin damage after radiotherapy: detection timing lag, lack of deep tissue information, insufficient quantitative analysis capability, lack of spatiotemporal dynamic tracking, and lack of three-dimensional spatial modeling capability. An intelligent detection system that can detect microscopic pathological changes in the skin in the subclinical period and achieve true early warning is urgently needed. SUMMARY

[0009] The purpose of the present application is to overcome the shortcomings of the prior art and provide a multispectral imaging assisted early detection system for skin damage after radiotherapy. By constructing a deep coupling closed-loop collaborative system of multispectral data acquisition module, deep tissue feature extraction module, three-dimensional lesion segmentation module and spatiotemporal tracking and evaluation module, the subclinical early detection of skin damage after radiotherapy is realized, and the detection time window is moved forward to before the appearance of visible symptoms, which saves valuable time for clinical intervention.

[0010] To achieve the above object, the application provides a multi-spectral imaging assisted early detection system for skin damage after radiotherapy, which comprises a multi-spectral data acquisition module, a deep tissue feature extraction module, a three-dimensional lesion segmentation module and a space-time tracking evaluation module.

[0011] The multi-spectral data acquisition module is used for acquiring multi-band spectral images of a radiotherapy area of a patient and extracting reflectivity data of skin tissue at different wavelengths. The deep tissue feature extraction module receives the multi-band spectral images, determines subcutaneous blood perfusion parameters, melanin concentration distribution parameters and collagen structure parameters based on spectral response characteristic analysis, and generates a deep tissue feature map. The three-dimensional lesion segmentation module receives the deep tissue feature map, identifies and segments a potential damage area through a three-dimensional medical image segmentation algorithm, and outputs a three-dimensional lesion area mask and damage depth information. The space-time tracking evaluation module receives the three-dimensional lesion area mask and historical detection data, analyzes the time evolution trajectory and spatial expansion trend of the damage area, generates an early warning signal and feeds back to the deep tissue feature extraction module, and realizes dynamic optimization of the feature extraction strategy.

[0012] The application forms a complete technical chain from data acquisition to feature extraction, from lesion segmentation to space-time tracking through the deep coupling of the four core modules, establishes a parameter-level deep coupling and closed-loop feedback mechanism between the modules, and realizes the coordinated improvement of skin damage detection accuracy and early warning capability.

[0013] Compared with the prior art, the application has the following remarkable advantages and beneficial effects:

[0014] First, the real early detection of radioactive skin damage is realized. The application can detect microscopic pathological changes such as blood vessel dilation and inflammatory reaction at a subclinical stage by acquiring 2.5cm deep tissue information through multi-spectral imaging technology, moves the detection time window from the visible period of the prior art to the early stage of pathological changes, and finds damage signs 5-7 days earlier on average, which wins critical time for clinical intervention and effectively prevents the occurrence of severe dermatitis.

[0015] Second, the precise quantitative analysis of skin tissue physiological parameters is realized. The application establishes a quantitative calculation model of key physiological parameters such as hemoglobin, melanin and collagen based on the spectral response characteristics of the 400-1350nm band, and the detection accuracy reaches the medical level, which can objectively evaluate the functional state of skin tissue and overcome the individual difference problem of traditional subjective evaluation methods.

[0016] Third, the three-dimensional precise segmentation of the damage area is realized. The application uses medical image segmentation technology to construct a three-dimensional space model of skin tissue, not only identifies the planar range of damage, but also evaluates the damage depth and volume, and the segmentation accuracy rate is more than 90%, which provides precise three-dimensional anatomical information for formulating individualized treatment plans.

[0017] Fourthly, the dynamic space-time tracking of the damage evolution process is realized. The time evolution model and the space expansion prediction model of the damage area are established by analyzing the multiple detection data through the space-time tracking evaluation module, the damage development trend can be predicted, the passive detection is changed to active early warning, and the prevention of radioactive dermatitis is moved forward.

[0018] Fifthly, the closed-loop adaptive optimization of the detection system is realized. According to the damage evolution law, the feature extraction strategy is dynamically adjusted through the feedback mechanism of the space-time tracking module to the feature extraction module, so that the detection performance is continuously optimized in the continuous use process, the detection sensitivity and specificity are improved, and the system has good self-learning and adaptive ability.

[0019] Sixthly, the system adopts a non-invasive and non-contact detection method, the patient does not need to bear additional pain, the detection process is fast and convenient, the single detection time is not more than 3 minutes, which is suitable for the high-frequency monitoring demand during radiotherapy, and the patient compliance and clinical application feasibility are significantly improved. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 It is a whole structure schematic diagram of the early detection system of skin damage after radiotherapy assisted by the multispectral imaging of the application.

[0021] Figure 2 It is a structure schematic diagram of the multispectral data acquisition module of the application.

[0022] Figure 3 It is a processing flow schematic diagram of the deep tissue feature extraction module of the application.

[0023] Figure 4 It is a network structure schematic diagram of the three-dimensional lesion segmentation module of the application.

[0024] Figure 5 It is a working principle schematic diagram of the space-time tracking evaluation module of the application. DETAILED DESCRIPTION

[0025] Please refer to the accompanying Figures 1-5 The application will be further described in detail below in combination with the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and do not limit the protection scope of the application.

[0026] Referring to Figure 1 The early detection system of skin damage after radiotherapy assisted by the multispectral imaging provided by the application comprises a multispectral data acquisition module 1, a deep tissue feature extraction module 2, a three-dimensional lesion segmentation module 3 and a space-time tracking evaluation module 4.

[0027] Referring to Figure 2The multispectral data acquisition module 1 comprises a light source control unit, a spectral imaging unit and an image preprocessing unit.

[0028] The light source control unit adopts an LED array as the illumination light source, which can generate multiple monochromatic lights of specific wavelengths in the 400-1350 nm wavelength range. In the preferred embodiment, 12 characteristic wavelengths are selected, including 450 nm, 500 nm, 550 nm, 600 nm, 650 nm in the visible light band, 750 nm, 850 nm, 950 nm in the first near-infrared window, and 1050 nm, 1150 nm, 1250 nm, 1350 nm in the second near-infrared window. The selection of these wavelengths is based on the absorption spectral characteristics of skin pigment groups: 450-550 nm is sensitive to melanin, 550-600 nm is sensitive to oxyhemoglobin, 600-650 nm is sensitive to deoxyhemoglobin, 750-950 nm can penetrate to the dermis layer, and 1050-1350 nm is sensitive to water and collagen structure. The light source control unit switches the LED light source of different wavelengths in a predetermined time sequence, and the exposure time of each wavelength is 50-100 ms. The total time for completing the 12-wavelength cycle acquisition is about 1.2 s.

[0029] The spectral imaging unit adopts an InGaAs camera as the image sensor, which has high quantum efficiency in the 400-1700 nm wavelength band and can capture spectral information in the visible light to near-infrared extended band. The camera has a resolution of 1280×1024 pixels and a pixel size of 15 μm, and is equipped with a C lens with a focal length of 25 mm and a field of view angle of 30°. At a standard working distance of 30 cm from the skin surface, a single image can cover an area of about 15 cm×12 cm of the skin, meeting the imaging needs of common radiotherapy sites such as the head and neck, chest wall, etc. The camera is connected to the computer through the GigE interface, and the image data is transmitted to the backend processing unit in real time.

[0030] The image preprocessing unit performs standardization processing on the 12 original images of different wavelengths collected. First, dark current correction is performed to subtract the dark field image collected in the closed light source state, eliminating the inherent noise of the sensor. Then, flat field correction is performed, dividing each wavelength image by the standard whiteboard reflection image of the corresponding wavelength, eliminating light unevenness and lens vignetting effects. The corrected images are spatially registered, and a rigid transformation method based on feature point matching is used to align the different wavelength images to the same coordinate system. The registration accuracy is controlled at the sub-pixel level, and the registration error is less than 0.5 pixels. Finally, the reflectivity calibration is performed, and the linear mapping relationship between the image gray value and the true reflectivity is established using the standard reflectivity board to convert the image data to absolute reflectivity values with a range of 0% to 100%.

[0031] In an embodiment of the present application, to improve the efficiency of data acquisition and patient comfort, the multispectral data acquisition module 1 also integrates an automatic skin region recognition function. The system first acquires a panoramic image at a wavelength of 550 nm, and uses a deep learning semantic segmentation network to identify the skin region and non-skin regions such as clothing, background, and hair in the image. Then, according to the patient's radiotherapy plan, the radiation field outline is extracted from the DICOM RT Dose file, the radiation field is projected onto the skin surface, and the high-dose region that needs to be monitored is determined. The system automatically adjusts the camera field of view and focal length, preferentially acquiring high-resolution multispectral images of the high-dose region, while rapidly scanning the surrounding area to form multiscale image data with resolution grading, thereby shortening the overall acquisition time while ensuring the imaging quality of the key region.

[0032] The data format output by the multispectral data acquisition module 1 is a four-dimensional tensor with dimensions HxWx12xT, where H and W are the number of height and width pixels of the image, 12 represents the 12 wavelength channels, and T represents the time dimension, i.e., the multiple detection data of the patient at different time points. At the initial detection, T = 1; as the radiotherapy progresses, the system recommends 2-3 detections per week, and the T value gradually accumulates, providing a data basis for subsequent spatiotemporal tracking analysis.

[0033] Referring to Figure 3 , the deep tissue feature extraction module 2 receives multispectral images from the multispectral data acquisition module 1, extracts physiological parameters of the deep tissue of the skin through optical property modeling and physiological parameter inversion algorithms, and generates deep tissue feature maps.

[0034] The core of this module is a skin optical model based on radiation transfer theory. The optical properties of skin tissue can be described by the absorption coefficient and the scattering coefficient. At a wavelength , the total absorption coefficient of skin tissue can be represented as a linear combination of the absorption coefficients of each pigment group: .

[0035] where is the oxygenated hemoglobin concentration, is the deoxygenated hemoglobin concentration, is the melanin concentration, is the water content, is the lipid content, , , , , are the molar absorption coefficients of each pigment group at wavelength , with units of cm These molar absorption coefficients can be obtained from the bio-optical database and are known wavelength-dependent functions.

[0036] Scattering coefficient of skin tissue The Mie scattering model is used to approximate: .

[0037] where, is the scattering amplitude parameter, reflecting the density and size of scattering particles in the tissue, is the scattering exponent, describing the wavelength dependence of scattering, is the reference wavelength, usually taken as 600 nm. For normal skin tissue, the value is about 1.0-1.5; when the collagen structure changes due to radiation damage, the value will change significantly.

[0038] Diffuse reflectance of skin surface The relationship between the tissue optical parameters can be derived from the diffusion approximation theory: .

[0039] where, is the reduced albedo, is the scattering anisotropy factor, for skin tissue is about 0.8-0.9, is the boundary condition constant, related to the refractive index ratio of the tissue and air, with a value of about 2.5-3.0. This formula establishes a quantitative relationship between the measurable surface reflectance and the deep tissue optical parameters.

[0040] The deep tissue feature extraction module 2 uses a nonlinear optimization algorithm to invert the physiological parameters. Specifically, the objective function is defined as:

[0041] .

[0042] where, is the actually measured reflectance at wavelength is the theoretical reflectance calculated by the optical model, is the weight coefficient of the th wavelength, is the parameter vector to be inverted, is the regularization coefficient to prevent overfitting, represents the square of the L2 norm. This objective function is solved using the Levenberg-Marquardt algorithm, iteratively finding the parameter combination that minimizes .

[0043] ​In the preferred embodiment, to improve the robustness and computational efficiency of the inversion algorithm, the deep tissue feature extraction module 2 adopts a two-layer optimization strategy. The first layer optimization is performed for each pixel individually, using the nonlinear optimization method described above. The second layer optimization utilizes the spatial continuity constraint between neighboring pixels to smooth the first layer results and eliminate parameter jumps caused by noise. The spatial smoothing is implemented using a bilateral filter: .

[0044] where, is the original parameter value obtained from the first layer optimization at pixel , is the smoothed parameter value, is the neighborhood window centered at , is the normalization factor, and are the Gaussian kernel functions in spatial and value domains, respectively, and control the degree of spatial smoothing and value domain edge preservation, respectively. In the present embodiment, is set to 5 pixels, and is set to 10% of the parameter value range.

[0045] Based on the inversion algorithm described above, the deep tissue feature extraction module 2 calculates a number of physiological parameter maps:

[0046] Blood perfusion parameter map: Define the total hemoglobin concentration , unit g / L, reflecting the blood perfusion level of the skin tissue. Define the blood oxygen saturation , reflecting the oxygenation state of the tissue. Early radiation-induced skin damage often manifests as vasodilation, increased blood perfusion, and significantly elevated; as the damage worsens, microcirculation is impaired, and decreased.

[0047] Melanin concentration distribution parameter map: The unit of melanin concentration is mg / L. Radiation can stimulate melanocytes, leading to hyperpigmentation, and increased; it can also cause depigmentation, and decreased. The spatial distribution pattern of melanin concentration is an important basis for judging the nature of the damage.

[0048] Collagen structure parameter map: The change of scattering index reflects the structural changes of collagen fiber network. The value of normal skin is stable in the range of 1.2-1.4; radiation damage causes collagen fiber denaturation and rupture, making the value deviate from the normal range. In addition, the scattering amplitude parameter ratio This ratio comprehensively reflects the integrity of the tissue's microstructure and has high sensitivity for early damage detection.

[0049] The deep tissue feature extraction module 2 integrates the aforementioned physiological parameter maps into a multi-channel deep tissue feature map. The data format is H×W×N, where N is the number of feature channels. In this embodiment, N=5. The feature map includes total hemoglobin concentration, blood oxygen saturation, melanin concentration, scattering index, and scattering amplitude-index ratio. This feature map serves as the input for the subsequent three-dimensional lesion segmentation module 3.

[0050] In one embodiment of the present invention, the deep tissue feature extraction module 2 also integrates a multi-scale feature fusion mechanism. Considering the large scale differences in lesion areas caused by radiation-induced skin damage, ranging from millimeter-scale micro-erythema to centimeter-scale large-area lesions, single-scale feature extraction is insufficient to meet the detection needs of lesions at different scales. Therefore, the module performs multi-scale decomposition on the input multispectral image, constructing a three-layer image pyramid using a Gaussian pyramid, corresponding to the original resolution, half-resolution, and quarter-resolution, respectively. Physiological parameters are inverted at each scale layer to obtain feature maps at different scales. Then, the low-resolution feature map is restored to the original resolution through upsampling and fused pixel-by-pixel with the high-resolution feature map to obtain a multi-scale enhanced deep tissue feature map. Experiments show that the multi-scale feature fusion mechanism improves the detection rate of small lesions by 12%–15%.

[0051] Furthermore, the deep tissue feature extraction module 2 receives feedback signals from the spatiotemporal tracking evaluation module 4, enabling dynamic optimization of the feature extraction strategy. The spatiotemporal tracking module analyzes historical detection data to identify the physiological parameters most sensitive to the patient and the wavelength combinations with the highest predictive value, feeding this information back to the feature extraction module. Based on this, the feature extraction module adjusts the weighting coefficients of the parameter inversion algorithm. This increases the weight of sensitive wavelengths, improving the targeting of feature extraction. Simultaneously, the regularization coefficient is dynamically adjusted based on the damage evolution trend. Reduced during the rapid progression of injury To improve sensitivity, increase the value during the steady-state period. This reduces false alarms. This closed-loop feedback mechanism enables the system to adapt to individual needs, and its detection performance is continuously optimized with ongoing use.

[0052] Reference Figure 4 The three-dimensional lesion segmentation module 3 receives the deep tissue feature map from the deep tissue feature extraction module 2, and uses a deep learning-based three-dimensional medical image segmentation algorithm to identify and segment potential radiation-induced skin damage areas, and outputs a three-dimensional lesion area mask and damage depth information.

[0053] The network architecture of the module is improved based on 3D U-Net, adopting an encoder-decoder structure. The encoder path contains 4 down-sampling stages, each of which is composed of two 3x3x3 three-dimensional convolution layers, a batch normalization layer and a ReLU activation function, followed by a 2x2x2 max-pooling layer for down-sampling. The decoder path contains 4 up-sampling stages, each of which is first up-sampled through a 2x2x2 deconvolution layer, then the feature maps of the corresponding layer of the encoder are spliced through a jump connection, and then two 3x3x3 three-dimensional convolution layers are used for feature fusion. The last layer of the network is a 1x1x1 convolution layer, with an output channel number of 2, representing the background and lesion area respectively, and the probability of each voxel belonging to the lesion area is output through the Softmax activation function.

[0054] To construct the three-dimensional input data, the three-dimensional lesion segmentation module 3 first needs to expand the two-dimensional deep tissue feature map into three-dimensional body data. The specific method is: using the sensitivity difference of different parameters in the deep tissue feature map to different tissue depths to reconstruct the depth information of the skin tissue. For example, the wavelengths of 450-550 nm mainly reflect the epidermis layer information, the wavelengths of 750-950 nm can penetrate to the dermis layer, and the wavelengths of 1050-1350 nm are sensitive to the deep dermis and subcutaneous tissue. By analyzing the reflectance gradient of different wavelengths, a depth reconstruction algorithm is used to estimate the distribution of pigment groups and scatterers in the vertical depth direction, so as to reconstruct the two-dimensional feature map of HxWxN into three-dimensional body data of HxWxDxN, wherein D is the depth dimension, in this embodiment D=64, corresponding to the depth range from the skin surface to 3mm below the skin, and each layer represents about 50μm thickness.

[0055] The depth reconstruction algorithm is based on the extended form of Beer-Lambert's law. For the tissue layer at depth , the attenuation of light with wavelength satisfies: .

[0056] wherein is the incident light intensity, is the light intensity at depth , and is the absorption coefficient at depth . In the discrete form, it is assumed that the skin tissue is divided into layers along the depth direction, the absorption coefficient of the th layer is , and the surface reflectance can be represented as the weighted contribution of each layer: .

[0057] wherein is the weight of the th layer on the surface reflectance, which is related to the degree of attenuation of light at that depth, For the first The intrinsic reflectivity of a layer is related to the pigment cluster concentration and scattering characteristics of that layer. By establishing a set of multi-wavelength reflectivity measurement equations and using the constrained least squares method to solve for the pigment cluster concentration and scattering parameters of each layer, the depth dimension can be reconstructed.

[0058] To improve the accuracy of 3D segmentation, the 3D lesion segmentation module 3 introduces an attention mechanism. At the skip connection between the encoder and decoder, a spatial attention gating unit is inserted. This unit generates attention weights based on the feature map of the current layer in the decoder, and performs weighted modulation on the feature map transmitted from the encoder, highlighting regions related to the lesion and suppressing irrelevant background. Attention weights The calculation formula is: .

[0059] in, This is the feature map of the current layer of the decoder. The feature map of the corresponding layer of the encoder. and The weight matrix is ​​a learnable matrix. For bias terms, The Sigmoid activation function is used. The attention-weighted feature map is shown. ,in This represents element-wise product.

[0060] The 3D lesion segmentation module 3 is trained using a combination of the Dice loss function and the cross-entropy loss function. .

[0061] in, For Dice's loss, For cross-entropy loss, and As a trade-off factor, it is set to 0.5 and 0.5 respectively in this embodiment. The Dice loss is defined as: .

[0062] in, The total number of voxels in the volume data. For the first Individual factors predict the probability of a lesion area. For the first The true label of an individual element, which can be either 0 or 1. The cross-entropy loss is defined as:

[0063] .

[0064] Dice loss focuses on increasing the overlap between the segmented region and the ground truth label, while cross-entropy loss focuses on reducing pixel-level classification errors. Combining the two can achieve a more balanced segmentation performance.

[0065] To train the three-dimensional lesion segmentation network, the present application constructs a radiotherapy-induced skin injury multi-spectral image dataset. The dataset contains clinical data from 1200 patients in the radiotherapy department of 5 hospitals, covering radiotherapy patients of various tumor types such as head and neck cancer, breast cancer, lung cancer, etc., with a dose range of 40-70 Gy in 20-35 fractions. Each patient undergoes multi-spectral imaging detection twice a week during radiotherapy, from before radiotherapy to 4 weeks after radiotherapy, with an average of 12-15 time points per patient. An expert group consisting of two experienced radiotherapy physicians and one dermatologist conducts clinical rating on each image according to the RTOG radiation dermatitis grading standard, and manually labels the lesion area boundary on the multi-spectral image as the true value for segmentation network training. The dataset is divided into training set, validation set and test set in the ratio of 8:1:1, ensuring that there is no overlap between patients in the three subsets.

[0066] The network training uses the Adam optimizer with an initial learning rate of 0.0001 and a cosine annealing strategy to dynamically adjust the learning rate. The batch size is set to 4, and each batch contains 4 three-dimensional body data. To enhance the robustness of the model, data augmentation techniques are used during training, including random rotation (-15° to 15°), random flipping, random scaling (0.9-1.1 times), random contrast adjustment (0.8-1.2 times), and Gaussian noise addition (standard deviation of 1%-3% of signal intensity). Training is performed on an NVIDIA V100 GPU, and convergence requires about 150 epochs, taking about 36 hours.

[0067] The performance evaluation results of the three-dimensional lesion segmentation network after training on the test set are as follows: Dice similarity coefficient is 0.89, sensitivity is 0.92, specificity is 0.95, positive predictive value is 0.87, and negative predictive value is 0.96. Compared with the two-dimensional segmentation method, the Dice coefficient of three-dimensional segmentation is increased by 8%, especially in the detection rate of early micro lesions, the three-dimensional method is 15%-18% higher than the two-dimensional method.

[0068] The output of the three-dimensional lesion segmentation module 3 includes two parts: one is the three-dimensional lesion area mask, the data format is HxWxD binary data, the mask value of 1 represents the detected lesion area, and the mask value of 0 represents normal tissue; the second is the damage depth information, for each plane coordinate , the maximum depth and the average depth of the lesion area along the depth direction are counted, which have important reference value for evaluating the severity of the injury and developing treatment plans.

[0069] In an embodiment of the present application, the three-dimensional lesion segmentation module 3 also integrates an uncertainty estimation function. Using the Monte Carlo Dropout method, the Dropout layer activation is preserved during inference, and multiple forward propagations are performed on the same input to obtain multiple segmentation results. The standard deviation of the segmentation probability reflects the uncertainty of the model's prediction for the region. For regions with high uncertainty, the system will mark them in the output, prompting the clinician to manually review, thereby improving the reliability and clinical acceptability of the system.

[0070] In addition, the three-dimensional lesion segmentation module 3 receives prediction information about the spatial expansion trend of the lesion from the spatiotemporal tracking evaluation module 4, which is incorporated into the segmentation network as prior knowledge. Specifically, before the last convolution layer of the network, a spatial attention layer is introduced, and the attention map of this layer is initialized by the high-risk lesion area predicted by the spatiotemporal tracking module, guiding the segmentation network to focus on these areas and improving the sensitivity to new lesions. This cross-module information exchange embodies the design philosophy of the deep coupling and collaborative system of the present application.

[0071] Reference Figure 5 The spatiotemporal tracking evaluation module 4 receives the three-dimensional lesion region mask from the three-dimensional lesion segmentation module 3 and the patient's historical detection data, analyzes the temporal evolution trajectory and spatial expansion trend of the lesion area, generates early warning signals, and transmits key feedback information to the deep tissue feature extraction module 2 and the three-dimensional lesion segmentation module 3, achieving closed-loop optimization.

[0072] The core of the spatiotemporal tracking evaluation module 4 is the spatiotemporal sequence analysis algorithm. For the patient's multiple detection data, arrange them in chronological order into a sequence , where is the three-dimensional lesion region mask obtained by the th detection. The spatiotemporal tracking module first extracts features from the lesion region of each detection and calculates the following key indicators:

[0073] Lesion volume : Count the number of voxels with a mask value of 1 in , multiply by the actual volume of a unit voxel, and obtain the total volume of the lesion area, with a unit of mm 3 .

[0074] Lesion area : Project the three-dimensional mask in the depth direction to obtain a two-dimensional plane mask, count the number of pixels with a mask value of 1, multiply by the actual area of a unit pixel, and obtain the projected area of the lesion region, with a unit of mm 2 .

[0075] Lesion center position : Calculate the centroid coordinates of the three-dimensional mask to represent the spatial position of the lesion region.

[0076] Lesion shape index: sphericity of the lesion region , value range 0-1, the closer to 1, the more spherical the shape; eccentricity of the lesion region , reflecting the stretching degree of the lesion shape.

[0077] Lesion boundary roughness : using fractal dimension calculation method to evaluate the irregularity of the lesion region boundary, malignant lesions often have more irregular boundaries.

[0078] Based on the above feature sequence , the time-space tracking module establishes a time evolution model. The exponential growth model is used to fit the time variation of the lesion volume: .

[0079] wherein, is the initial lesion volume, is the time point when the lesion is first detected, is the lesion growth rate constant, unit: day . By least squares fitting of historical data points, the parameters and are estimated, and the lesion volume at future time is predicted. If , wherein is the preset growth rate threshold, the system determines that the lesion is in rapid progression, and generates a high-level warning signal. In the preferred embodiment, takes 0.15 day , corresponding to a rate of about 2 times per week increase in lesion volume.

[0080] For the time series of the lesion center position , the time-space tracking module calculates the position change of adjacent time points, and analyzes whether the lesion has spatial migration. Under normal circumstances, the position of radioactive skin damage should be relatively fixed, consistent with the distribution of the radiation field. If significant position drift is detected, it may indicate the presence of a new lesion or a misdiagnosis, and the system will mark the abnormality and suggest review.

[0081] The time-space tracking module also establishes a spatial expansion prediction model to predict the future expansion direction and range of the lesion region. Using the diffusion equation model, the lesion expansion is regarded as a diffusion process from the damaged area to the surrounding normal tissue: .

[0082] wherein, is the damage degree of position at time , value range 0-1, 0 represents normal tissue, and 1 represents complete damage, D is the diffusion coefficient, reflecting the speed of lesion propagation, L is the Laplace operator, R is the reaction term, describing the local growth and repair kinetics of the lesion. In a preferred embodiment, the form of the Fisher-KPP equation is adopted: .

[0083] where, D is the diffusion coefficient, reflecting the speed of lesion propagation, R is the reaction term, describing the local growth and repair kinetics of the lesion. In a preferred embodiment, the form of the Fisher-KPP equation is adopted: , , Then, the partial differential equation is numerically solved using finite difference method to predict the lesion distribution at future time . The prediction results are presented in the form of probability maps, indicating the risk of lesion occurrence at each location in the future.

[0084] The spatiotemporal tracking evaluation module 4 generates a graded warning signal based on the above analysis results:

[0085] Green (low risk): the rate of lesion volume growth day , the lesion area growth is less than 10% per week, and no new lesions are detected. It is recommended to maintain the current detection frequency.

[0086] Yellow (medium risk): the rate of lesion volume growth day , the lesion area growth is 10% to 30% per week, or the boundary expansion is detected. It is recommended to increase the detection frequency to 3 times per week and consider preventive intervention measures.

[0087] Orange (high risk): the rate of lesion volume growth day , the lesion area growth is 30% to 50% per week, or the depth increase is >200 μm per week. It is recommended to detect daily and start an active treatment program.

[0088] Red (extremely high risk): the rate of lesion volume growth day , the lesion area growth is >50% per week, or new multiple lesions are detected. It is recommended to consider suspending radiotherapy and emergency dermatology consultation.

[0089] The spatiotemporal tracking evaluation module 4 realizes closed-loop optimization through two feedback channels. The first feedback channel is connected to the deep tissue feature extraction module 2, and the information transmitted includes: the physiological parameter type most sensitive to the patient, the wavelength combination most valuable for prediction, and the current injury evolution stage. The feature extraction module adjusts the weight coefficients of the parameter inversion algorithm accordingly, assigns higher weights to sensitive parameters, and improves the relevance of feature extraction; according to the injury evolution stage, dynamically adjust the regularization coefficient, reduce the regularization strength in the rapid progression period to improve sensitivity, and increase the regularization in the stable period to reduce false positives. The second feedback channel is connected to the three-dimensional lesion segmentation module 3, and the probability map of the spatial expansion prediction is transmitted as the spatial prior of the segmentation network to guide the network to focus on high-risk areas and improve the detection rate of new small lesions.

[0090] Through this closed-loop feedback mechanism, the system continuously learns the injury evolution law of the individual patient during continuous use, and realizes the optimization of the personalized detection strategy. Clinical verification shows that after 3-4 times of adaptive learning, the false positive rate of the system is reduced by 20%-25%, the missed diagnosis rate is reduced by 15%-18%, and the detection performance is significantly better than that of the static system based on fixed parameters.

[0091] In an embodiment of the present application, the spatiotemporal tracking evaluation module 4 also integrates an interface with the radiotherapy planning system, which can import the patient's DICOM RT Dose data to obtain the cumulative radiation dose distribution received by each point on the skin surface. The spatiotemporal tracking module performs spatial registration and correlation analysis on the actual detected injury distribution and the dose distribution, and establishes an individualized dose-effect relationship model. For regions with similar doses, if the injury degree in some regions is significantly higher than that in other regions, it indicates that the patient has higher sensitivity to radiation or other aggravating factors. The system will mark the high-sensitivity area and remind the clinician to pay special attention to protection. The dose-effect model can also be used to predict the injury development trend in the remaining radiotherapy course, providing a quantitative reference for adjusting the radiotherapy plan.

[0092] The overall workflow of the system of the present application will be described below in conjunction with a complete clinical application embodiment.

[0093] Patient Zhang, a 62-year-old woman, received adjuvant radiotherapy after breast-conserving surgery for left breast cancer. The radiotherapy plan is conventional fractionation, total dose 50Gy, 25 times, 2Gy each time, 5 times a week, course of treatment 5 weeks. The patient underwent the first multispectral imaging detection before radiotherapy, established the baseline parameters of the skin tissue, and then underwent detection twice a week during radiotherapy.

[0094] Week 1, 1st detection (3 days after radiotherapy started): Multispectral data acquisition module 1 acquires 12-wavelength spectral images of the patient's left chest wall skin, covering an area of 15 cm x 12 cm, including the entire radiation field and a 3 cm boundary. Deep tissue feature extraction module 2 inverts the blood perfusion parameters map, showing that the total hemoglobin concentration in the central area of the radiation field is 4.2 g / L, and the blood oxygen saturation is 68%, which is a slight increase in blood perfusion and a slight decrease in blood oxygen saturation compared to the baseline detection (total hemoglobin 3.8 g / L, blood oxygen saturation 71%), which is consistent with the physiological characteristics of early inflammatory response during radiotherapy. Three-dimensional lesion segmentation module 3 does not detect a clear lesion area, and the system outputs a green warning signal. Spatiotemporal tracking evaluation module 4 records the detection data for this time as the basis for subsequent spatiotemporal analysis.

[0095] Week 2, 1st detection (10 days after radiotherapy started): Deep tissue feature extraction module 2 detects that the total hemoglobin concentration in the central area of the radiation field is elevated to 5.8 g / L, and the blood oxygen saturation is reduced to 58%, and the melanin concentration is increased from the baseline of 120 mg / L to 145 mg / L. Three-dimensional lesion segmentation module 3 identifies this area as a potential damage area, with a segmentation volume of about 180 mm 3 , and a maximum depth of 0.45 mm. During the clinical examination, there is no obvious erythema in this area of the skin, and the doctor's rating is RTOG 0 level. Spatiotemporal tracking evaluation module 4 analyzes the data from the two detections, calculates the lesion volume growth rate of about 0.08 day , and determines it as medium risk, outputting a yellow warning signal and suggesting increasing the detection frequency. The system feeds back to the feature extraction module, increasing the weight of the hemoglobin and melanin related wavelengths (550 nm, 600 nm).

[0096] Week 3, 1st detection (17 days after radiotherapy started): The damage area expands to 3.5 cm x 2.8 cm, and the three-dimensional segmentation volume increases to 620 mm 3 , with a maximum depth of 0.72 mm. During the clinical examination, a slight erythema is visible, and the doctor's rating is RTOG 1 level. Spatiotemporal tracking module calculates the growth rate up to 0.16 day , and determines it as high risk, outputting an orange warning signal. The system suggests starting active protective measures, including external moisturizers, avoiding friction, etc. The doctor adopts the suggestion and guides the patient to strengthen skin care.

[0097] Week 3, 2nd detection (20 days after radiotherapy started): After 3 days of active care, the damage area does not continue to expand rapidly, with an area of about 3.6 cm x 2.9 cm, a volume of 660 mm 3 , and a growth rate of 0.05 day , the spatiotemporal tracking module determines that the risk level is downgraded to yellow. Meanwhile, the spatial expansion prediction model shows that the expansion speed of the lesion area boundary slows down, and the expected expansion range in the next week is less than 5 mm, indicating that the nursing measures are effective.

[0098] Week 4 and Week 5 detection: The patient continues to follow the doctor's advice for skin care, and the lesion area remains relatively stable, with a growth rate of 0.03-0.06 day low growth rate. Until the end of radiotherapy, the patient's final rating is RTOG 1-2, no severe dermatitis occurs, and treatment interruption is avoided.

[0099] In this case, the system detected the subclinical lesion signal at week 2, 7 days earlier than the clinically visible symptoms, and issued a yellow warning, allowing the clinician to take timely intervention measures and effectively control the lesion progression, demonstrating the clinical value of early detection and warning of the present invention.

[0100] To comprehensively evaluate the performance of the present invention system, a prospective clinical study was conducted in the radiotherapy departments of 5 hospitals, involving 200 tumor patients receiving radiotherapy, who were randomly divided into the test group and the control group, each with 100 cases. Patients in the test group were subjected to multispectral imaging detection twice a week using the present invention system, while patients in the control group were subjected to doctor assessment once a week according to the standard clinical process. The primary endpoint was the incidence of severe radiation dermatitis (RTOG grade 3 and above), and the secondary endpoints included early lesion detection sensitivity, detection accuracy, patient satisfaction, and other indicators.

[0101] The results showed that the incidence of severe dermatitis in the test group was 8%, significantly lower than that in the control group of 23% (P<0.001, χ 2 test). In the test group, the system detected early lesions an average of 5.2±1.8 days earlier than the appearance of clinically visible symptoms. The lesion area segmentation accuracy of the system, measured by the Dice coefficient, was 0.82 for RTOG grade 1, 0.91 for RTOG grade 2, and 0.94 for RTOG grade 3, showing good segmentation performance. The early warning sensitivity of the system (the proportion of patients who will develop severe dermatitis) was 89%, the specificity (the proportion of patients who will not develop severe dermatitis) was 85%, the positive predictive value was 74%, and the negative predictive value was 94%, indicating that the system has high predictive value. Patient satisfaction surveys showed that the test group patients' satisfaction with skin care guidance was 92%, significantly higher than that of the control group of 78% (P<0.01).

[0102] In addition, the system maintains stable detection performance in different tumor types, different radiotherapy schemes, and different skin types of patients, showing good generalization ability and clinical practicability. The average time consumption of single detection is 2.5±0.4 min, including patient positioning, image acquisition, data processing and result generation, meeting the requirements of clinical rapid detection. The system is easy to operate, and the radiotherapy technician can operate it skillfully after 4h training without additional medical staff.

[0103] In summary, the multispectral imaging assisted radiotherapy post-skin damage early detection system provided by the present application realizes early detection, precise quantitative analysis, three-dimensional space modeling and dynamic space-time tracking of subclinical stage of radioactive skin damage through deep coupling and cooperation of four core modules of multispectral data acquisition, deep tissue feature extraction, three-dimensional lesion segmentation and space-time tracking evaluation, significantly improves the detection sensitivity and early warning ability, provides reliable basis for timely clinical intervention, and has important clinical application value and broad popularization prospect.

[0104] The above only describes the preferred embodiments of the present application and is not intended to limit the present application, and any modifications, equivalent replacements and improvements made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A multispectral imaging assisted system for early detection of skin damage after radiotherapy, characterized in that, The application relates to a multi-spectral data acquisition module for acquiring multi-wavelength spectral images of a patient's radiotherapy region to obtain reflectivity data of skin tissue in a 400-1350nm waveband range; a deep tissue feature extraction module connected with the multi-spectral data acquisition module, for receiving the multi-wavelength spectral images, establishing a skin optical model based on radiation transmission theory, calculating blood perfusion parameters, melanin concentration distribution parameters and collagen structure parameters through a parameter inversion algorithm, and generating a deep tissue feature map; a three-dimensional lesion segmentation module connected with the deep tissue feature extraction module, for receiving the deep tissue feature map, expanding the two-dimensional feature map into a three-dimensional body data, identifying and segmenting a damage region by using a three-dimensional medical image segmentation network, and outputting a three-dimensional lesion region mask and damage depth information; and a space-time tracking evaluation module connected with the three-dimensional lesion segmentation module, for receiving the three-dimensional lesion region mask and historical detection data, extracting damage region features, establishing a time evolution model to analyze a growth rate, establishing a spatial expansion prediction model to predict future distribution, generating a graded early warning signal, and feeding back information to the deep tissue feature extraction module and the three-dimensional lesion segmentation module to realize closed-loop adaptive optimization. The closed-loop adaptive optimization mechanism is realized by the following modes: the space-time tracking evaluation module analyzes historical detection data to identify the most sensitive physiological parameters of a patient, feeds back to the deep tissue feature extraction module, the deep tissue feature extraction module adjusts the weight coefficients of each wavelength in the nonlinear parameter inversion algorithm according to this, increases the weight of the sensitive wavelength, and dynamically adjusts the regularization coefficient according to the damage evolution stage, reduces the regularization strength to improve the sensitivity in the rapid progression period, and enhances the regularization to reduce false alarms in the stable period; the space-time tracking evaluation module feeds back the spatial expansion prediction result to the three-dimensional lesion segmentation module as a spatial priori to guide the network to focus on the high-risk area. The multi-spectral data acquisition module comprises a light source control unit, a spectral imaging unit and an image preprocessing unit, the light source control unit adopts an LED array to generate monochromatic light of 12 characteristic wavelengths, including 450nm, 500nm, 550nm, 600nm, 650nm, 750nm, 850nm, 950nm, 1050nm, 1150nm, 1250nm and 1350nm, and sequentially switches different wavelength light sources for exposure in a time sequence; the spectral imaging unit adopts an InGaAs camera to collect skin reflection images under each wavelength; and the image preprocessing unit performs dark current correction, flat field correction, spatial registration and reflectivity calibration on the collected original images to output absolute reflectivity values. ​ ​ ​ 2. The system of claim 1, wherein, ​ 3. The system of claim 1, wherein, In the deep tissue feature extraction module, the skin optical model represents the absorption coefficient of the skin tissue as a linear combination of five pigment groups of oxygenated hemoglobin, deoxygenated hemoglobin, melanin, water and lipid, and represents the scattering coefficient as a power function of wavelength, and establishes a quantitative relationship between surface reflectance and tissue optical parameters through diffusion approximation theory; the parameter inversion algorithm is a nonlinear parameter inversion algorithm, the nonlinear parameter inversion algorithm defines a target function containing the difference between the measured reflectance and the model reflectance, and uses the Levenberg-Marquardt algorithm to iteratively solve the physiological parameter combination that minimizes the target function, and uses a bilateral filter for spatial smoothing to improve robustness.

4. The system of claim 1, wherein, The deep tissue feature map generated by the deep tissue feature extraction module includes total hemoglobin concentration parameter map, blood oxygen saturation parameter map, melanin concentration parameter map, scattering index parameter map and scattering amplitude-index ratio parameter map, the total hemoglobin concentration parameter reflects the blood perfusion level, the blood oxygen saturation parameter reflects the tissue oxygenation state, the melanin concentration parameter reflects pigmentation or depigmentation, and the scattering index and ratio parameter reflects the integrity of the collagen fiber network structure.

5. The system of claim 1, wherein, The three-dimensional lesion segmentation module adopts an encoder-decoder network architecture improved based on 3D U-Net, the encoder path contains 4 down-sampling stages, the decoder path contains 4 up-sampling stages, and a spatial attention gate unit is inserted at the jump connection to highlight the lesion-related area; the depth reconstruction algorithm is based on the difference in penetration depth of different wavelengths of light in skin tissue, estimates the distribution of pigment groups in the depth direction by analyzing the reflectance gradient of different wavelengths, and reconstructs the two-dimensional feature map into three-dimensional volume data, with the depth dimension corresponding to the range from the skin surface to 3mm below the skin.

6. The system of claim 1 or 5, wherein, The three-dimensional lesion segmentation module uses a combination of Dice loss function and cross-entropy loss function for network training, the training data set contains at least 1000 cases of multi-time point multi-spectral images of radiotherapy patients and expert annotated damage area boundaries, and data enhancement technology is used to improve the robustness of the model; the three-dimensional lesion region mask data output by the segmentation network is in binary data format, and for each plane coordinate, the maximum depth and average depth along the depth direction are counted as damage depth information.

7. The system of claim 1, wherein, The time evolution model established by the spatio-temporal tracking evaluation module uses an exponential growth model to fit the change of lesion volume over time, estimates the growth rate constant by least squares method, and determines that the lesion is in rapid progression period and generates a high-level warning when the growth rate exceeds a preset threshold; the spatial expansion prediction model describes the expansion process of the damage from the damaged area to the surrounding normal tissue based on the diffusion equation, and uses finite difference method to numerically solve to obtain the damage distribution probability map at future time.

8. The system of claim 1 or 7, wherein, The hierarchical warning signal generated by the spatiotemporal tracking evaluation module includes four levels of green low risk, yellow medium risk, orange high risk and red extremely high risk, and the classification basis is lesion volume growth rate, lesion area growth percentage, damage depth increase speed and new lesion number; the information fed back by the spatiotemporal tracking evaluation module to the deep tissue feature extraction module includes the most sensitive physiological parameter type and the most predictive wavelength combination, and the information fed back to the three-dimensional lesion segmentation module includes a spatial expansion prediction probability map as a segmentation prior.

9. The system of claim 1, wherein, The system further comprises a display module and a report generation module, the display module is used for real-time display of the multispectral image, pseudo-color visualization of the deep tissue feature map, stereoscopic rendering of the three-dimensional lesion segmentation result, spatiotemporal evolution curve and warning level, and the report generation module is used for integrating analysis results of each module to generate a comprehensive evaluation report containing area and volume data of the damage region, physiological parameter quantitative value, spatial correlation analysis with the radiation dose distribution, damage development trend prediction and nursing suggestion.

Images