A prediction model training method for obtaining a blood perfusion map of a keloid

Abstract

The present application relates to a kind of for obtaining the training method of prediction model of blood perfusion map of keloid, comprising: obtaining sample data set, sample data set is preprocessed, obtain the training data set including training set and test set, each sample is pix2pix sample, including: true gross image, true blood perfusion map to which true gross image belongs;True segmentation image of gross image;The three images of each sample are input into the four generators of the corresponding TripleGAN neural network pre-constructed in three discriminators, and the total loss information is obtained based on the output of four generators and the corresponding image in sample;Further, TripleGAN neural network is trained and tested, obtain the trained TripleGAN neural network, the generator of TripleGAN neural network that receives gross image and outputs blood perfusion map is used as the trained prediction model.The above method solves the problem that the existing scale evaluation is not accurate and the cost is high and time-consuming and laborious when using LSCI evaluation.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method for training a predictive model for obtaining blood perfusion maps of keloids, a method for predicting blood perfusion results of keloids, and an electronic device. Background Technology

[0002] In the diagnosis and treatment of keloids, accurate assessment of the severity of its progression is essential for reminding patients to seek medical intervention, monitoring the effectiveness of treatments, and objectively comparing the efficacy of different interventions.

[0003] Current research on keloid assessment, both domestically and internationally, primarily relies on scale-based evaluations, with the Vancouver Scar Scale (VSS) being the most commonly used standard. These scales mainly assess subjectively aspects such as pigmentation, vascularization, thickness, flexibility, and roughness. However, the VSS is subject to observer subjectivity; its drawbacks include poor repeatability of observations and poor repeatability among observers.

[0004] Blood perfusion is an objective indicator for assessing keloids, but previous studies have not been consistent in evaluating blood flow within keloids. This may be because most current research focuses on the number of vessels. Since vessels within pathological scars (including hypertrophic scars and keloids) may exhibit endothelial hyperplasia, luminal stenosis, or even occlusion, the number of vessels may not accurately reflect the blood perfusion status within pathological keloids. Therefore, measuring and studying blood perfusion within pathological keloids as a whole has become a current research focus.

[0005] Laser speckle contrast imaging (LSCI) is a recently mature technique based on speckle contrast analysis. It's an innovative method for assessing tissue blood perfusion with high resolution and fast scan time, providing objective facts and non-contact measurements of blood flow perfusion in specific areas. LSCI has been used to assess blood flow in patients with microvascular diseases, port-wine stains, and burn scars, offering advantages such as high image resolution, fast imaging speed, large scanning range, and low spatial variability. Currently, there are relatively few studies applying LSCI to investigate blood flow perfusion levels in keloids. Although LSCI has become an objective method for assessing tissue blood flow perfusion, many hospitals lack the necessary LSCI equipment, and assessing keloids using LSCI still requires considerable time. Keloid delineation is particularly time-consuming, especially for multiple irregular keloids, where the delineation time is significantly longer than for single, regularly shaped keloids, causing inconvenience in clinical practice.

[0006] Therefore, there is an urgent need for a solution that uses gross images to automatically and objectively predict the blood perfusion results of keloids. Summary of the Invention

[0007] (a) Technical problems to be solved

[0008] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a method for training a predictive model for obtaining blood perfusion maps of keloids, a method for predicting blood perfusion results of keloids, and an electronic device, which solves the problems of inaccurate scale assessment and high cost and time-consuming and laborious use of LSCI assessment in the prior art.

[0009] (II) Technical Solution

[0010] To achieve the above objectives, the main technical solutions adopted by the present invention include:

[0011] In a first aspect, embodiments of the present invention provide a method for training a predictive model to obtain a blood flow perfusion map of keloids, comprising:

[0012] Obtain a sample dataset, wherein each sample in the sample dataset includes: a gross image of at least one keloid, a blood perfusion map corresponding to the gross image; and a gross image outlining the boundary of the keloid.

[0013] The sample dataset is preprocessed to obtain a training dataset including a training set and a test set. The training set includes paired samples, each paired sample being a pix2pix sample, including: a true gross image, the true blood flow perfusion map to which the true gross image belongs, and the true segmentation image of the gross image.

[0014] The three images of each sample are input into the four generators in the pre-built TripleGAN neural network, and the three discriminators of the TripleGAN neural network obtain the total loss information based on the output of the four generators and the corresponding images in the sample.

[0015] Based on the training set, test set, and pre-set target value of total loss information, the TripleGAN neural network is trained and tested to obtain the trained TripleGAN neural network. The generator in the TripleGAN neural network that receives the gross image and outputs the blood flow perfusion map is used as the prediction model after training.

[0016] Optionally, the step of inputting the three images of each sample into the four corresponding generators in the pre-constructed TripleGAN neural network includes:

[0017] For each paired sample:

[0018] The true macro image is input into the first generator G. I2P Output the first blood flow perfusion map;

[0019] The true blood flow perfusion map is input into the second generator G. P2I Output the first general image;

[0020] The true macro image is input into the third generator G. I2S Output the first segmented image;

[0021] The true segmented image is input into the fourth generator G. S2I Output the second large image;

[0022] The first blood flow perfusion map is input into the first generator G. I2P Output the third large image;

[0023] The first general image is input into the second generator G. P2I Output the second blood flow perfusion map;

[0024] The first segmented image is input into the third generator G. I2S Output the fourth large image;

[0025] The second large image is input into the fourth generator G. S2I Output the second segmented image.

[0026] Optionally, the three discriminators of the TripleGAN neural network obtain total loss information based on the outputs of the four generators and the corresponding images in the samples, including:

[0027] After each sample is input into the generator, the discriminator loss, consistency loss, and cycle loss are obtained respectively.

[0028] The total loss information L includes: the discriminator loss L of the four generators. D The cyclic loss L of the four generators cyc and two consistency losses L cons ;

[0029] ;

[0030] Among them, G I2P Represents the first generator, G P2I Indicates the second generator, G I2S Represents the third generator, G S2I Indicates the fourth generator;

[0031] D S Indicates the first discriminator, D I Indicates the second discriminator, D P This represents the third discriminator;

[0032] I, S, and P correspond to the gross image, segmentation image, and blood perfusion image, respectively.

[0033] G P2I (P) indicates that P is input into G. P2I The output result; the base e of the logarithm.

[0034] Optionally, the sample dataset is preprocessed to obtain a training dataset including a training set and a test set, including:

[0035] Retrieve the historical medical information of the patient ID to which the gross image belongs;

[0036] Based on historical medical information, gross images containing keloid treatment records, incomplete epidermal information, and blood perfusion maps of such gross images were removed from the sample dataset.

[0037] Remove gross images and blood perfusion maps of the specified defects from the historical medical information in the sample dataset;

[0038] All remaining gross images and blood perfusion maps are cropped and registered using a boundary registration method to obtain pix2pix true gross images and the true blood perfusion maps to which the true gross images belong;

[0039] The system automatically identifies and processes the general image with the keloid boundary to obtain a binarized true segmentation image.

[0040] The training set includes paired samples, and the test set includes paired samples and unpaired samples;

[0041] The true gross images and true blood flow perfusion maps in the unpaired samples are not paired.

[0042] Optionally, the pre-built TripleGAN neural network includes four generators and three discriminators, each generator being an adversarial neural network. The TripleGAN neural network is trained and tested based on the training set, the test set, and a pre-defined target value for the total loss information to obtain the trained TripleGAN neural network.

[0043] A phased training method is adopted. First, unpaired samples are input for training as the first phase, and then paired samples are input for training as the second phase. The first and second phases are alternated, with each cycle consisting of more than 200 rounds.

[0044] There is no consistency loss in the loss for training unpaired samples.

[0045] Optionally, the perfusion map of each paired sample is obtained by performing LSCI on the gross image within the sample; and the perfusion value corresponding to the perfusion map is obtained by LSCI.

[0046] Training the prediction model also includes:

[0047] For paired samples, obtain the blood perfusion value calculated by the prediction model based on the output blood perfusion map, and compare the blood perfusion value with the blood perfusion value in the paired samples;

[0048] The trained prediction model can output blood perfusion maps and blood perfusion values.

[0049] Secondly, embodiments of the present invention also provide a method for predicting blood perfusion results in keloids, comprising:

[0050] Obtain a gross image of the keloid to be analyzed at a specified size;

[0051] The general image is input into the trained generator to obtain the blood perfusion results output by the generator;

[0052] The trained generator is a generator obtained by training a predictive model for obtaining blood flow perfusion maps of keloids.

[0053] Optionally, the gross image is input into the trained generator to obtain the blood perfusion results output by the generator, including:

[0054] If the prediction interface receives a user-triggered instruction to predict the blood perfusion map, the gross image is input into the trained generator to obtain the blood perfusion map output by the generator, and the gross image and the predicted blood perfusion map are compared and displayed on the prediction interface.

[0055] If the prediction interface receives a user-triggered instruction to predict blood perfusion values, the general image is input into the trained generator to obtain the blood perfusion map output by the generator, and the blood perfusion value is automatically generated based on the blood perfusion map. The blood perfusion value is the average value of blood flow pixels within the boundary range of the keloid.

[0056] The prediction interface displays the general image, the predicted blood perfusion map, and the blood perfusion value; the blood perfusion map shows the boundary of the keloid.

[0057] Thirdly, embodiments of the present invention also provide an electronic device, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program stored in the memory and performs the steps of the method for predicting the blood perfusion results of keloids as described in any of the second aspects; or, performs the steps of the method for training a prediction model for obtaining blood perfusion maps of keloids as described in any of the first aspects.

[0058] (III) Beneficial Effects

[0059] The beneficial effects of this invention are as follows: The prediction model training method for obtaining blood flow perfusion maps of keloids in this invention uses paired samples from the training dataset to train a pre-constructed neural network with four generators and three discriminators, thereby obtaining a trained generator as a prediction model. This facilitates the direct output of blood flow perfusion maps from gross images, improving judgment efficiency while ensuring accuracy. It also reduces the cost of existing LSCI methods. Compared with existing technologies, this invention solves the shortcomings of existing LSCI methods, such as high cost, long evaluation time, and inability to achieve remote evaluation. Attached Figure Description

[0060] Figure 1 A flowchart illustrating a method for training a predictive model to obtain blood perfusion maps of keloids, provided in an embodiment of the present invention.

[0061] Figure 2 A flowchart illustrating a method for predicting blood perfusion results in keloids, as provided in an embodiment of the present invention;

[0062] Figure 3A and Figure 3B These are the architecture diagrams of the TripleGAN neural network provided in the embodiments of the present invention;

[0063] Figure 4 This is a schematic diagram of laser speckle contrast imaging and model prediction based on a macroscopic photograph in an embodiment of the present invention;

[0064] Figure 5 This is a schematic diagram showing the predicted and measured values ​​of the model in this embodiment of the invention before and after treatment. Detailed Implementation

[0065] To better explain and facilitate understanding of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0066] Scars are the product of natural healing after skin injury. Specifically, they are the normal repair process in which fibrous tissue replaces normal skin after the skin is damaged. They usually appear as flat or slightly raised light pink marks that gradually soften over time and do not extend beyond the original wound.

[0067] Keloids are a type of pathological hyperplasia that may continue to expand beyond the original injury area. They are an abnormal growth that presents as red or dark red lumps with a hard and tough texture. They are often accompanied by itching or stinging and can spread to the surrounding normal skin. They are commonly found in areas with high tension, such as the chest, back, and shoulders, and are related to factors such as genetics and immune abnormalities.

[0068] TripleGAN is an improved GAN model that includes four generators G and three discriminators D during the training phase.

[0069] Example 1

[0070] like Figure 1 As shown, the method of this embodiment can be implemented on any server and is used to train a prediction model to predict the blood perfusion results of keloids in gross images. The method of this embodiment may include the following steps:

[0071] 100. Obtain a sample dataset, wherein each sample in the sample dataset includes: a gross image of at least one keloid, a blood perfusion map corresponding to the gross image; and a gross image outlining the boundary of the keloid.

[0072] When preparing the actual sample dataset, sample pairs of gross images and blood flow perfusion maps can be constructed to generate a pixel-based image generation dataset.

[0073] In this embodiment, the gross image in the sample pair can be an image obtained from the hospital with the patient's consent, and the blood perfusion map is a blood perfusion map paired with the gross image after being output by LSCI.

[0074] 200. Preprocess the sample dataset to obtain a training dataset including a training set and a test set. The training set includes paired samples, each paired sample being a pix2pix sample (i.e., a sample belonging to pixel-level alignment), including: a true gross image, the true blood flow perfusion map to which the true gross image belongs; and the true segmentation image of the gross image.

[0075] For example, you can first obtain the historical medical information of the patient ID to which each gross image belongs;

[0076] Based on historical medical information, gross images containing keloid treatment records, incomplete epidermal information, and blood perfusion maps of such gross images were removed from the sample dataset.

[0077] Remove gross images and blood perfusion maps of the specified defects from the historical medical information in the sample dataset;

[0078] All remaining gross images and blood perfusion maps are cropped and registered using boundary registration to obtain pix2pix true gross images and the true blood perfusion maps to which the true gross images belong.

[0079] The system automatically identifies and processes the general image with the keloid boundary to obtain a binarized true segmentation image.

[0080] The training set includes paired samples, and the test set includes paired samples and unpaired samples;

[0081] The true gross images and true blood flow perfusion maps in the unpaired samples are not paired.

[0082] It should be noted that during the training phase, in order to better distinguish between paired samples belonging to the training dataset and the generator output during the training process, the paired samples are all distinguished as "true" during training. That is, all images belonging to the paired samples are considered true images input to the generator.

[0083] 300. Input the three images of each sample into the four generators in the pre-constructed TripleGAN neural network, and the three discriminators of the TripleGAN neural network obtain the total loss information based on the output of the four generators and the corresponding images in the sample.

[0084] The pre-built TripleGAN neural network includes four generators and three discriminators. Each generator is an adversarial neural network. Based on the training set, test set, and a pre-defined target value for total loss information, the TripleGAN neural network is trained and tested to obtain the trained TripleGAN neural network. In practical applications, a separate validation set can also be set, which is processed according to the actual situation; this embodiment does not limit it.

[0085] It should be noted that a phased training method can be used in this embodiment. First, unpaired samples are input for training as the first phase, and then paired samples are input for training as the second phase. The first and second phases are alternated, with each cycle having more than 200 rounds.

[0086] There is no consistency loss in the loss for training unpaired samples.

[0087] In other words, the inputs to the TripleGAN neural network in this embodiment include: a gross image, a perfusion image, and a scar segmentation image. Figure 3A and Figure 3B As shown.

[0088] The TripleGAN neural network mainly consists of four generators and three discriminators. The four generators are: G_I2P (gross image to blood perfusion map), G_P2I (blood perfusion map to gross image), G_I2S (gross image to segmented image), and G_S2I (segmented image to gross image). A real image is processed by the generators to produce a fake image; the fake image is then processed by the reverse generator to obtain a recursive image (rec).

[0089] There are three types of loss functions: the discriminator loss function (i.e., the loss in real / fake discrimination), the consistency loss function, and the recurrence loss function. For each fake image generated by the generator once (e.g., fake_SI), and the original image of that category (real_I), they are input into the discriminator for judgment, and its loss function L is calculated. D For a fake image (e.g., fake_SI) that is then processed by a generator to obtain a looped image (e.g., rec_S), the difference between the looped image and the original image is calculated to obtain the loss function L. C Since the input includes paired samples, meaning some samples can achieve pixel-level correspondence between the image and blood perfusion, a consistency loss function is calculated for the generated fake image (e.g., fake_P) and its corresponding original image (real_P). Simultaneously, for the paired input images, a consistency loss function is also calculated between fake_SI and fake_PI.

[0090] It is worth noting that the consistency loss function will not be calculated when unpaired images are used as input.

[0091] 400. Based on the training set, test set, and the target value of the pre-set total loss information, the TripleGAN neural network is trained and tested to obtain the trained TripleGAN neural network. The generator in the TripleGAN neural network that receives the gross image and outputs the blood flow perfusion map is used as the prediction model after training.

[0092] Furthermore, in this embodiment, the perfusion map of each paired sample in steps 100 and 200 above is obtained by performing LSCI on the gross image within the sample to obtain the perfusion map; and the perfusion value corresponding to the perfusion map is obtained by LSCI.

[0093] At this point, training the prediction model in step 400 further includes:

[0094] For paired samples, the blood perfusion value calculated by the prediction model based on the output blood perfusion map is obtained, and the blood perfusion value is compared with the blood perfusion value in the paired samples; the trained prediction model can output blood perfusion map and blood perfusion value.

[0095] In this embodiment, the blood perfusion value is obtained based on the pixel values ​​within a specified area of ​​the blood perfusion map.

[0096] That is, the prediction model can generate fine-grained blood perfusion images. By combining the blood perfusion map with the gross image, the blood perfusion is divided into regions (e.g., distinguishing between scar and non-scar areas; the scar area can be divided into fine grids, and pixel values ​​are calculated for each grid) to obtain the blood perfusion values ​​inside the keloid and the blood perfusion values ​​of the normal skin surrounding the keloid. Specifically, in an optional application, the scar boundary in the blood perfusion map can also be manually delineated to obtain the blood perfusion value.

[0097] The prediction model training method in this embodiment uses paired samples from the training dataset to train a pre-built neural network with four generators and three discriminators, thereby obtaining a trained generator as a prediction model. This facilitates the direct output of blood flow perfusion maps from macroscopic images, improving judgment efficiency while ensuring accuracy. It also reduces the cost of existing LSCI methods. Compared with existing technologies, this method solves the shortcomings of existing LSCI methods, such as high cost, long evaluation time, and inability to achieve remote evaluation.

[0098] In one alternative implementation, to better understand the process of step 300 above, the following will be combined with... Figure 3A and Figure 3B Detailed explanation:

[0099] 301. For each paired sample: the true gross image is input into the first generator G-I2P, and the first blood perfusion map is output;

[0100] The true blood flow perfusion map is input into the second generator G-P2I, which outputs the first gross image;

[0101] The true macro image is input into the third generator G-I2S, which outputs the first segmented image;

[0102] The true segmented image is input into the fourth generator G-S2I, which outputs the second macro image;

[0103] The first blood flow perfusion image is input into the first generator G-I2P, and the third gross image is output.

[0104] The first gross image is input into the second generator G-P2I, which outputs the second blood perfusion map.

[0105] The first segmented image is input into the third generator G-I2S, and the output is the fourth large image;

[0106] The second large image is input into the fourth generator G-S2I, which outputs the second segmented image.

[0107] In this embodiment, Figure 3A This diagram only illustrates, for illustrative purposes, a generator's convolutional layer, BatchNorm layer, and ReLU activation function module, as well as the flow of skip connections, upsampling layers, and downsampling layers. Meanwhile, in Figure 3A The diagram shows information and processing flow for the convolutional layers, normalization layers, and ReLU activation functions in the discriminator; this is only an illustrative illustration.

[0108] exist Figure 3B In this diagram, generators of the same color are considered as one. To better illustrate the input and output, they are represented by two identical colors. The inputs are a general image (Image, I), a segmentation image (S), and a perfusion image (P). "real" indicates true, representing the image from the sample. "fake" indicates false, representing the image output by a generator. "rec" indicates a loop (i.e., the image has been generated twice).

[0109] 302. After each sample is input into the generator, the discriminator loss, consistency loss, and cycle loss are obtained respectively;

[0110] The total loss information L includes: the discriminator loss L of the four generators. D The cyclic loss L of the four generators cyc and two consistency losses L cons ;

[0111] ;

[0112] Among them, G I2P Represents the first generator, G P2I Indicates the second generator, G I2S Represents the third generator, G S2I Indicates the fourth generator;

[0113] D S Indicates the first discriminator, D I Indicates the second discriminator, D P This represents the third discriminator;

[0114] I, S, and P correspond to the gross image, segmentation image, and blood perfusion image, respectively.

[0115] G P2I (P) indicates that P is input into G. P2IThe output result; the base e of the logarithm.

[0116] Throughout the training process, the total loss information is used to make the four generators converge to a deployment state.

[0117] In this embodiment, the above-mentioned triple loss is dynamically weighted and integrated to drive the four generators to converge to a usable state that can be directly deployed.

[0118] Example 2

[0119] The method of this embodiment can be implemented on any server or a central server. After training the prediction model, the prediction model can be sent to any medical client, such as a PAD or mobile terminal, or other self-service detection terminal, to predict the blood perfusion map of gross images with keloids. The prediction model training method of this embodiment may include the following steps:

[0120] Step 1: Data Acquisition

[0121] Sample pairs of gross images are obtained. All gross images can be captured by a mobile terminal such as a smartphone. In this embodiment, any lesion that can be clearly seen when captured by the mobile terminal is sufficient. This embodiment does not limit the magnification.

[0122] For each gross image, LSCI and the system's accompanying software were used to measure the blood perfusion map and blood perfusion value of the pathological scar (measured in perfusion units, PUs, mL / 100g / min), thereby obtaining a sample pair of gross images.

[0123] Step 2: Data Preprocessing

[0124] In this embodiment, historical medical information of the patient ID to which the gross image belongs can be obtained in advance, and then the data can be filtered based on the historical medical information. For example, gross images and blood perfusion maps of the gross images containing keloid treatment records or epidermal incompleteness information can be removed from the sample dataset, as well as gross images and blood perfusion maps of the gross images containing specified defect information in the historical medical information of the sample dataset.

[0125] In this embodiment, the data standards for obtaining gross images are: 1) meeting the clinical diagnosis of pathological scars, containing one or more lesions; 2) the patient's pathological scars have not previously received treatment, including surgery, radiotherapy, laser therapy, corticosteroid injections, pressure therapy, cryotherapy, etc.; 3) the lesion epidermis is intact, with no obvious signs of infection such as ulceration or purulent discharge; 4) the patient has no systemic diseases, such as a history of diseases of vital organs or active autoimmune diseases. This indicates that the above methods are used to select samples in the training dataset, and it is not necessary to select the gross images to be analyzed during the testing phase.

[0126] Next, the images in the sample are cropped, for example, the general image is cropped to a square with a side length of 3 times the width of the scar, and the keloid is placed as centrally as possible in the image.

[0127] Furthermore, the scar boundaries in the gross image and blood perfusion map are manually delineated. A boundary-based registration method is then used, which involves scaling, translating, and rotating the delineated boundaries in the gross image to align them with the boundaries in the blood perfusion map. This completes the registration between the gross image and the blood perfusion map, thereby constructing pix2pix (pixel-to-pixel) training samples. In this embodiment, pix2pixel-based paired samples are used to supervise the value of each pixel during model training, ensuring the accuracy and precision of the model output.

[0128] To obtain a better dataset for training, the applicant retrospectively collected 948 lesion photographs of pathological scars and 758 blood perfusion maps from January 2018 to December 2021, and constructed 156 paired samples (i.e., registration sample pairs where the lesion photographs, after segmentation, scaling, translation, and rotation, coincide with the boundaries of the blood perfusion maps). Of the 156 paired samples, 116 were used as the training set for the blood perfusion prediction model, and the other 40 were used as the test set.

[0129] Each of the 156 paired samples included: a gross photograph and a blood perfusion map.

[0130] In this embodiment, paired samples need to be constructed during the data preprocessing stage.

[0131] In this embodiment, the segmented image in the paired sample can be a binarized image, which displays pixels of type 1 and 1, where 1 represents a keloid pixel and 0 represents a non-keloid pixel.

[0132] Step 3: Construct a TripleGAN neural network and train it.

[0133] The TripleGAN neural network consists of four generators and three discriminators. The generators are tasked with generating realistic images to confuse the discriminators, while the discriminators are tasked with distinguishing the generated images from the original images. Through the adversarial interaction between the generators, the images become more realistic.

[0134] The four generators are G_I2P, which generates a macro image to a blood perfusion map; G_P2I, which generates a blood perfusion map to a macro image; G_I2S, which generates a macro image to a segmented image; and G_S2I, which generates a segmented image to a macro image.

[0135] In this process, a real image (real) is processed by a generator to produce a fake image, and the fake image is then processed by a reverse generator to obtain a recurrent image (rec). In Figure 3, real includes real_I, Real_S, and Real_P. I, S, and P correspond to the gross image, segmentation image, and blood flow perfusion image, respectively. fake includes fake_I, fake_S, and fake_P, and rec includes rec_I, rec_S, and rec_P. Figure 3 shows the network architecture required for the entire training process.

[0136] After model training, during the usage phase, only the generator G_I2P can be used, with a gross image as input, to generate a perfusion map. The other generators and discriminators can be omitted.

[0137] In this embodiment, there are three types of loss functions in the training process: discriminator loss function, consistency loss function, and recurrence loss function.

[0138] For each fake image generated by the generator (e.g., fake_SI) and the original image of that category (real_I), the images are fed into the discriminator for judgment, and its loss function LD is calculated.

[0139] For a fake image (such as fake_SI) that is then processed by a generator to obtain a loop image (such as rec_S), the difference between the loop image and the original image is calculated to obtain the loss function LC.

[0140] Since the input includes paired samples, meaning that some samples can achieve pixel-level correspondence between the image and blood perfusion, a consistency loss function is calculated for the generated fake image (e.g., fake_P) and the corresponding original image (real_P).

[0141] Additionally, for images with paired input samples, a consistency loss function between fake_SI and fake_PI is calculated. It's important to note that the consistency loss function is not calculated when unpaired images are used as input. The specific loss function is as follows:

[0142] For the cyclic loss L cyc ;

[0143] ;

[0144] ;

[0145] For the discriminant loss L D :

[0146] ;

[0147] For consistency loss L cons ;

[0148] ;

[0149] ;

[0150] For the overall loss L

[0151] ;

[0152] In this context, G1 and G2 represent different generators. G1(x) represents the output of generator G1 after inputting x, and G2(G1(x)) represents the output of generator G2 after inputting G1(x). E represents the expectation, and norm represents normalization.

[0153] D y (y) indicates that y is input into the discriminator D. y The subsequent result, D y (G(x)) represents inputting G(x) into D y The result afterwards.

[0154] The bases e, I, S, and P of log correspond to the gross image, segmentation image, and blood perfusion image, respectively.

[0155] It should be noted that in this embodiment, the sample contains both unpaired and paired gross images and blood perfusion maps, and the sample also includes segmented images (or scar segmentation maps), i.e., the aforementioned binarized images.

[0156] This embodiment employs a phased training method. At the beginning of each epoch, unpaired samples are input for training first, followed by all paired samples for training, alternating between the two phases. The main parameter is setting the epoch to 200 rounds.

[0157] Step 4: Model Evaluation

[0158] Blood perfusion values ​​reflect the proliferation of blood vessels and the degree of scar growth activity. In this embodiment, blood perfusion values ​​can be used to evaluate the prediction model. Figure 4 The first column is the overall image, the second column is the output of LSCI, and the third column is the output of the prediction model. By comparison, the results are found to be in line with expectations. To better evaluate the model, relative blood perfusion values ​​are used for evaluation.

[0159] Relative blood perfusion value: During the research, it was found that the image generation model was affected by color and brightness even when processing photographs of the same scar. Previous literature studies have shown that using relative blood perfusion values ​​to evaluate scars is more meaningful than absolute blood perfusion evaluation; that is, calculating the difference in blood perfusion between the scar and the surrounding skin is more meaningful than calculating the blood perfusion value within the scar itself. Therefore, a relative blood perfusion index was calculated: first, the blood perfusion value within the scar was calculated, and then, based on the size of the scar, the blood perfusion value of the normal skin within a radius from the scar was calculated and taken as the normal skin blood perfusion value of the scar. The relative blood perfusion value was obtained by calculating the difference between the two. All blood perfusion values ​​in the aforementioned embodiments are relative blood perfusion values.

[0160] In addition, in practical applications, shooting conditions may significantly affect model performance; therefore, the model needs to have high robustness.

[0161] In this embodiment, the following three tasks are designed to evaluate the stability of the prediction model:

[0162] 1): Consistency of blood perfusion values ​​under different perspectives;

[0163] 2): Consistency of blood perfusion values ​​under different lighting conditions (bright, dim);

[0164] 3): The effect of image blur on blood perfusion values;

[0165] The robustness test data consisted of 400 images, each extracted from four video segments of 100 frames. The videos were recorded continuously from an angle of 45° to 135° in front of the patient's scar. For each extracted image, the following processing was performed: lighting adjustment: brightness was adjusted to 0.8 to 1.2 times the original brightness; blurring: Gaussian blurring was applied to the image to simulate potential image quality degradation in real-world scenarios.

[0166] Pearson correlation coefficients and absolute errors between the predicted and actual relative blood perfusion values ​​of pathological scars were calculated for 40 pairs of test set samples. The results showed a correlation coefficient of 0.761 between the predicted and actual relative blood perfusion values, with a blood perfusion error of 9.5 PU. This indicates that the model can predict blood perfusion in scar areas relatively accurately, demonstrating higher robustness. Furthermore, it shows a strong correlation between the model's prediction of relative blood perfusion values ​​for keloids and the actual relative blood perfusion values ​​measured using LSCI, demonstrating that the model can predict scar blood perfusion values ​​in photographs relatively accurately.

[0167] in addition, Figure 5The results of the predicted blood perfusion value, the blood perfusion value measured by LSCI before treatment, and the blood perfusion value measured by LSCI after treatment were compared. It was found that the output of the prediction model was relatively stable and had high accuracy. Figure 5 The results show that the predicted values ​​of the model achieved correlations of 0.64 and 0.76 with the measured values ​​before and after treatment, respectively.

[0168] Stability assessments are performed for different perspectives. Stability is defined as the amount of fluctuation in the results caused by changes in perspective, calculated by subtracting the mean of all results from the predictions for each perspective. The smaller the fluctuation, the more stable the model.

[0169] Anti-interference capability is assessed under conditions of lighting and blurring. Anti-interference capability is defined as the average of the results under the interference condition and the results under the no-interference condition. If the absolute value of the difference is small, the model exhibits strong anti-interference capability; conversely, if the difference is large and fluctuates significantly, the model has weak anti-interference capability.

[0170] Example 3

[0171] This embodiment provides a method for predicting blood perfusion results in keloids. This prediction method can be implemented in any lightweight client, such as a mobile terminal APP or mini-program, or a doctor's client or self-service device client, etc. The prediction model described above can be deployed to implement the prediction method of this application. The prediction method of this embodiment may include:

[0172] A1. Obtain a gross image of the keloid to be analyzed at a specified size;

[0173] A2. Input the general image into the trained generator and obtain the blood perfusion results output by the generator;

[0174] The trained generator is a generator obtained based on the training method of the prediction model for obtaining blood perfusion maps of keloids as described in any of the foregoing embodiments.

[0175] Step A2 above may further include:

[0176] A21. If the prediction interface receives a user-triggered instruction to predict the blood perfusion map, the general image is input into the trained generator to obtain the blood perfusion map output by the generator, and the general image and the predicted blood perfusion map are compared and displayed on the prediction interface.

[0177] A22. If the prediction interface receives a user-triggered instruction to predict blood perfusion values, the general image is input into the trained generator to obtain the blood perfusion map output by the generator, and blood perfusion values ​​are automatically generated based on the blood perfusion map. The blood perfusion values ​​are the average values ​​of blood flow pixels within the boundary range of the keloid.

[0178] The prediction interface displays the general image, predicted blood perfusion map, and blood perfusion values; the blood perfusion map shows the boundary of the keloid. In this embodiment, the boundary of the keloid can be identified by an automatic recognition model or manually drawn, depending on the actual situation; this embodiment does not limit it.

[0179] The prediction method in this embodiment is simple and portable, and can effectively reduce doctors' costs and save time. At the same time, it reduces the time cost for patients and is easy to promote and use.

[0180] In addition, this embodiment also provides an electronic device, which includes: a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program stored in the memory and executes the steps of the method for predicting the blood perfusion results of keloids as described in any of Embodiment 3; or, executes the steps of the training method for the prediction model of obtaining blood perfusion maps of keloids as described in any of Embodiment 1 or Embodiment 2.

[0181] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0182] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions.

[0183] It should be noted that any reference numerals placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In claims that enumerate several means, several of these means may be embodied by the same hardware. The use of the terms first, second, third, etc., is merely for convenience of expression and does not indicate any order. These terms can be understood as part of the component names.

[0184] Furthermore, it should be noted that in the description of this specification, the terms "one embodiment," "some embodiments," "embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0185] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the claims should be interpreted to include both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0186] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, then this invention should also include these modifications and variations.

Claims

1. A method for training a predictive model to obtain blood flow perfusion maps of keloids, characterized in that, include: Obtain a sample dataset, wherein each sample in the sample dataset includes: a gross image of at least one keloid, a blood perfusion map corresponding to the gross image; and a gross image outlining the boundary of the keloid. The sample dataset is preprocessed to obtain a training dataset including a training set and a test set. The training set includes paired samples, each paired sample being a pix2pix sample, including: a true gross image, the true blood flow perfusion map to which the true gross image belongs, and the true segmentation image of the gross image. The three images of each sample are input into the four generators in the pre-built TripleGAN neural network, and the three discriminators of the TripleGAN neural network obtain the total loss information based on the output of the four generators and the corresponding images in the sample. The step of inputting the three images of each sample into the four corresponding generators in the pre-constructed TripleGAN neural network includes: For each paired sample: The true macro image is input into the first generator G. I2P Output the first blood flow perfusion map; The true blood flow perfusion map is input into the second generator G. P2I Output the first general image; The true macro image is input into the third generator G. I2S Output the first segmented image; The true segmented image is input into the fourth generator G. S2I Output the second large image; The first blood flow perfusion map is input into the first generator G. I2P Output the third large image; The first general image is input into the second generator G. P2I Output the second blood flow perfusion map; The first segmented image is input into the third generator G. I2S Output the fourth large image; The second large image is input into the fourth generator G. S2I Output the second segmented image; Based on the training set, test set, and pre-set target value of total loss information, the TripleGAN neural network is trained and tested to obtain the trained TripleGAN neural network. The generator in the TripleGAN neural network that receives the gross image and outputs the blood flow perfusion map is used as the prediction model after training.

2. The training method according to claim 1, characterized in that, The TripleGAN neural network's three discriminators obtain total loss information based on the outputs of the four generators and the corresponding images in the samples, including: After each sample is input into the generator, the discriminator loss, consistency loss, and cycle loss are obtained respectively. The total loss information L includes: the discriminator loss L of the four generators. D The cyclic loss L of the four generators cyc and two consistency losses L cons .

3. The training method according to claim 1, characterized in that, The sample dataset is preprocessed to obtain a training dataset including a training set and a test set, including: Retrieve the historical medical information of the patient ID to which the gross image belongs; Based on historical medical information, gross images containing keloid treatment records, incomplete epidermal information, and blood perfusion maps of such gross images were removed from the sample dataset. Remove gross images and blood perfusion maps of the specified defects from the historical medical information in the sample dataset; All remaining gross images and blood perfusion maps are cropped and registered using a boundary registration method to obtain pix2pix true gross images and the true blood perfusion maps to which the true gross images belong; The system automatically identifies and processes the general image with the keloid boundary to obtain a binarized true segmentation image. The training set includes paired samples, and the test set includes paired samples and unpaired samples; The true gross images and true blood flow perfusion maps in the unpaired samples are not paired.

4. The training method according to claim 1, characterized in that, The pre-built TripleGAN neural network includes four generators and three discriminators. Each generator is an adversarial neural network. The TripleGAN neural network is trained and tested based on the training set, the test set, and a pre-defined target value for total loss information to obtain the trained TripleGAN neural network. A phased training method is adopted. First, unpaired samples are input for training as the first phase, and then paired samples are input for training as the second phase. The first and second phases are alternated, with each cycle consisting of more than 200 rounds. There is no consistency loss in the loss for training unpaired samples.

5. The training method according to claim 1, characterized in that, The perfusion map of each paired sample is obtained by performing LSCI on the gross image of the sample. And obtain the blood perfusion value corresponding to the blood perfusion map through LSCI; Training the prediction model also includes: For paired samples, obtain the blood perfusion value calculated by the prediction model based on the output blood perfusion map, and compare the blood perfusion value with the blood perfusion value in the paired samples; The trained prediction model can output blood perfusion maps and blood perfusion values.

6. A method for predicting blood perfusion results in keloids, characterized in that, include: Obtain a gross image of the keloid to be analyzed at a specified size; The general image is input into the trained generator to obtain the blood perfusion results output by the generator; The trained generator is a generator obtained based on the training method of the prediction model for obtaining blood perfusion maps of keloids as described in any one of claims 1 to 5.

7. The prediction method according to claim 6, characterized in that, The general image is input into the trained generator to obtain the blood perfusion results output by the generator, including: If the prediction interface receives a user-triggered instruction to predict the blood perfusion map, the gross image is input into the trained generator to obtain the blood perfusion map output by the generator, and the gross image and the predicted blood perfusion map are compared and displayed on the prediction interface. If the prediction interface receives a user-triggered instruction to predict blood perfusion values, the general image is input into the trained generator to obtain the blood perfusion map output by the generator, and the blood perfusion value is automatically generated based on the blood perfusion map. The blood perfusion value is the average value of blood flow pixels within the boundary range of the keloid. The prediction interface displays the general image, the predicted blood perfusion map, and the blood perfusion value; the blood perfusion map shows the boundary of the keloid.

8. An electronic device, characterized in that, include: The device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program stored in the memory and performs the steps of the method for predicting the blood perfusion results of keloids according to any one of claims 6 and 7; or, performs the steps of the training method for a prediction model for obtaining blood perfusion maps of keloids according to any one of claims 1 to 5.

Images