A training method and device of an image inpainting model and a computer device
By acquiring the edge map of the original CT image and training it with the image infilling model, combined with multi-data center data, and using local optimization networks and adversarial networks, the problem of blood vessel blurring in the image infilling model was solved, improving the robustness of the model and image quality, and achieving efficient image infilling without annotation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUANGZHOU NAT LAB
- Filing Date
- 2024-11-25
- Publication Date
- 2026-06-16
AI Technical Summary
In existing technologies, when image inpainting models trained using adversarial networks are used for image inpainting, the blood vessels in the inpainted images are blurred, which cannot meet the actual needs.
By acquiring training sample images and corresponding real label images, edge detection is performed to obtain edge maps. CT incomplete images are generated using an image incomplete model, and blood vessel images are extracted separately. The loss function value is determined based on the blood vessel images, and the model parameters are adjusted. Multiple CT original images from multiple data centers are used for training, and local optimization networks and adversarial networks are used for encoding and binarization.
It solves the problem of blurred blood vessels in image filling, improves the robustness and performance of the image filling model, eliminates the need for annotation of the original CT image, and generates an image with quality close to the original image.
Smart Images
Figure CN120876265B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical image processing technology, and specifically to a training method, apparatus, and computer equipment for an image filling model. Background Technology
[0002] With the rapid development of deep learning technology, people's demand for CT images is increasing. When processing CT images, image filling is involved. In related technologies, when image filling models directly generated by adversarial networks are used for image filling, the blood vessels in the image are blurred, which cannot meet the actual needs. Summary of the Invention
[0003] In view of this, the present invention provides an image inpainting model, an image inpainting model training method and apparatus to solve the problem that when image inpainting is performed using an image inpainting model trained by an adversarial network, the blood vessels in the image are blurred, which cannot meet the actual needs.
[0004] In a first aspect, the present invention provides a training method for an image inpainting model, comprising the following steps: acquiring training sample images and corresponding ground truth labels, wherein the training sample images are edge maps of a CT image to be inpainted; inputting the training sample images into the image inpainting model to generate a CT inpainted image; extracting blood vessel images from the CT inpainted image and the ground truth labels respectively to obtain a first blood vessel image and a second blood vessel image; determining a first loss function value based on the first blood vessel image and the second blood vessel image; and adjusting the parameters of the image inpainting model based on the first loss function value.
[0005] The image inpainting model training method provided by this invention involves: acquiring training sample images and corresponding ground truth label images; inputting the training sample images into the image inpainting model to generate CT inpainted images; extracting blood vessel images from the CT inpainted images and ground truth label images respectively to obtain a first blood vessel image and a second blood vessel image; determining a first loss function value based on the first blood vessel image and the second blood vessel image; and adjusting the parameters of the image inpainting model based on the first loss function value. Because blood vessel images are considered during the training of the image inpainting model, the problem of blurred blood vessels in the inpainted image can be solved when using the trained image inpainting model for image inpainting; and no annotation of the original CT image is required.
[0006] In one alternative implementation, the training method for the image incomplete model further includes the following steps: acquiring multiple original CT images; performing edge detection on the original CT images to obtain an edge map; using the edge map as a training sample image and the original CT images as the ground truth label images.
[0007] This allows the edge map to be input into the image inpainting model for training, without the need to annotate the original CT image.
[0008] In one optional implementation, the training method for the image inpainting model further includes the following steps: obtaining a second loss function value based on the original CT image and the inpainted CT image; adjusting the parameters of the image inpainting model based on the first loss function value, including: adjusting the parameters of the image inpainting model based on the first loss function value and the second loss function value.
[0009] Therefore, the parameters of the image infilling model can be adjusted using the first and second loss function values, so that the image filled by the trained image infilling model is close to the original image.
[0010] In one optional implementation, extracting vascular images from the CT-filled image and the real-labeled image to obtain a first vascular image and a second vascular image includes: encoding the CT-filled image and the original CT image using a local optimization network to obtain a first encoded image and a second encoded image; and binarizing the first encoded image and the second encoded image to obtain the first vascular image and the second vascular image.
[0011] Therefore, blood vessel images can be added during the training of the image filling model, so that the problem of blurred blood vessels in the image can be solved when using the trained image filling model to fill the image.
[0012] In one optional implementation, edge detection of the original CT image to obtain an edge map includes: filtering the original CT image; and performing edge detection on the filtered original CT image using a preset operator to obtain an edge map, wherein the edge map includes a first edge map excluding the lung window and / or a second edge map excluding the mediastinal window.
[0013] Therefore, the image filling model can be trained using the first edge map and the original CT image to obtain the lung window image filling model; the image filling model can be trained using the second edge map and the original CT image to obtain the mediastinal window image filling model.
[0014] In one alternative implementation, the local optimization network includes a backbone network and a local attention module, wherein the backbone network is a ResNet50.
[0015] This allows for accurate encoding of CT-filled images and original CT images.
[0016] In one alternative implementation, multiple original CT images belong to multiple data centers.
[0017] This can increase the robustness of the image inpainting model and improve its performance.
[0018] Secondly, the present invention also provides a training apparatus for an image inpainting model, comprising an acquisition module, a CT inpainting image generation module, a blood vessel image determination module, a first loss function value determination module, and a parameter adjustment module; wherein, the acquisition module is used to acquire training sample images and corresponding ground truth label images, the training sample images being edge maps of the CT image to be inpainted; the CT inpainting image generation module is used to input the training sample images into the image inpainting model to generate CT inpainting images; the blood vessel image determination module is used to extract blood vessel images from the CT inpainting images and the ground truth label images respectively to obtain a first blood vessel image and a second blood vessel image; the first loss function value determination module is used to determine a first loss function value based on the first blood vessel image and the second blood vessel image; and the parameter adjustment module is used to adjust the parameters of the image inpainting model based on the first loss function value.
[0019] Fourthly, the present invention also provides a computer device, including a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the training method of the image filling model described in the first aspect or any corresponding embodiment.
[0020] Fifthly, the present invention also provides a computer-readable storage medium storing computer instructions for causing a computer to execute the training method of the image infilling model described in the first aspect or any corresponding embodiment thereof.
[0021] In a sixth aspect, the present invention also provides a computer program product, including computer instructions for causing a computer to execute a training method for an image infilling model according to the first aspect or any corresponding embodiment thereof. Attached Figure Description
[0022] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0023] Figure 1 This is a flowchart of an image infilling model training method according to an embodiment of the present invention;
[0024] Figure 2 This is a schematic diagram of the input image for the lung window image filling model according to an embodiment of the present invention;
[0025] Figure 3 This is a schematic diagram of the output image of the lung window image filling model according to an embodiment of the present invention;
[0026] Figure 4 This is a schematic diagram of the input image for the diaphragm window image filling model according to an embodiment of the present invention;
[0027] Figure 5 This is a schematic diagram of the output image of the septal window image filling model according to an embodiment of the present invention;
[0028] Figure 6 This is a flowchart of another image filling model training method according to an embodiment of the present invention;
[0029] Figure 7 This is a schematic diagram of an example of a local optimization network according to an embodiment of the present invention;
[0030] Figure 8 This is a flowchart of another image filling model training method according to an embodiment of the present invention;
[0031] Figure 9 This is a schematic diagram of an image filling model training method according to an embodiment of the invention;
[0032] Figure 10 This is a structural block diagram of an image filling model training device according to an embodiment of the present invention;
[0033] Figure 11 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation
[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0035] According to an embodiment of the present invention, a training method for an image filling model is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.
[0036] This embodiment provides a training method for an image filling model, which can be used with computer devices. Figure 1 This is a flowchart of an image inpainting model training method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:
[0037] Step S101: Obtain training sample images and corresponding real label images. The training sample images are the edge maps of the CT images to be filled.
[0038] In one alternative implementation, the training sample images and the ground truth label images can be obtained by: acquiring multiple original CT images; performing edge detection on the original CT images to obtain an edge map; using the edge map as the training sample images and the original CT images as the ground truth label images.
[0039] Specifically, edge detection of the original CT image to obtain an edge map includes: filtering the original CT image; and performing edge detection on the filtered original CT image using a preset operator to obtain an edge map, wherein the edge map includes a first edge map excluding the lung window and / or a second edge map excluding the mediastinal window. In other words, the first edge map and the original CT image can be used to train the image filling model to obtain a lung window image filling model; and the second edge map and the original CT image can be used to train the image filling model to obtain a mediastinal window image filling model.
[0040] Example, Figure 2 This is a schematic diagram of the input image for the lung window image filling model according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the output image of the lung window image filling model according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the input image for the diaphragm window image filling model according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the output image of the septal window image filling model according to an embodiment of the present invention.
[0041] The original CT images can be collected in a data center; the Canny operator can be used to perform edge detection on the original CT images to obtain an edge map.
[0042] It should be noted that with the rapid development of deep learning technology, the demand for CT images is increasing, but the difficulty in obtaining data annotations has restricted the development of image inpainting technology. The image inpainting model training method provided in this embodiment performs edge detection on the original CT image before training the image inpainting model, obtaining an edge map. This edge map is then input into the image inpainting model for training. Therefore, it is unnecessary to annotate the original CT image when training the image inpainting model.
[0043] Step S102: Input the training sample images into the image inpainting model to generate CT inpainted images.
[0044] Step S103: Extract blood vessel images from the CT filled image and the real label image respectively to obtain the first blood vessel image and the second blood vessel image.
[0045] Step S104: Determine the value of the first loss function based on the first blood vessel image and the second blood vessel image.
[0046] Step S105: Adjust the parameters of the image inpainting model based on the first loss function value.
[0047] The image inpainting model training method provided in this embodiment involves acquiring training sample images and corresponding ground truth label images; inputting the training sample images into the image inpainting model to generate CT inpainted images; extracting blood vessel images from the CT inpainted images and ground truth label images respectively to obtain a first blood vessel image and a second blood vessel image; determining a first loss function value based on the first blood vessel image and the second blood vessel image; and adjusting the parameters of the image inpainting model based on the first loss function value. Since blood vessel images are considered during the training of the image inpainting model, the problem of blurred blood vessels in the inpainted image can be solved when using the trained image inpainting model for image inpainting; and no annotation of the original CT image is required.
[0048] This embodiment provides a training method for an image filling model, which can be used with computer devices. Figure 6 This is a flowchart of another image filling model training method according to an embodiment of the present invention, such as... Figure 6 As shown, the process includes the following steps:
[0049] Step S601: Obtain training sample images and corresponding real label images. The training sample images are the edge maps of the CT images to be filled.
[0050] As shown above, training sample images and ground truth label images can be obtained by: acquiring multiple original CT images; performing edge detection on the original CT images to obtain edge maps; using the edge maps as training sample images and the original CT images as ground truth label images.
[0051] In related technologies, multiple original CT images are sourced from the same data center. However, due to the varying data distribution across different data centers, the trained image inpainting model often only predicts data distributions from the same data center. Therefore, in this embodiment, multiple original CT images are sourced from multiple data centers, thereby increasing the robustness and performance of the image inpainting model.
[0052] Step S602: Input the training sample images into the image inpainting model to generate CT inpainted images.
[0053] Step S603: Extract blood vessel images from the CT-filled image and the real label image respectively to obtain the first blood vessel image and the second blood vessel image.
[0054] In one optional implementation, extracting vascular images from the CT-filled image and the ground truth labeled image to obtain a first vascular image and a second vascular image includes the following steps:
[0055] Step S6031: Encode the CT infill image and the original CT image using a local optimization network to obtain the first encoded image and the second encoded image.
[0056] In one alternative implementation, such as Figure 7 As shown, the local optimization network uses ResNet50 pre-trained on ImageNet as the backbone network, and expands the receptive field using a local attention module, thereby learning more vascular details. Finally, it is processed by 1×1 convolution to obtain the first and second encoded images.
[0057] Step S6032: Binarize the first coded image and the second coded image respectively to obtain the first blood vessel image and the second blood vessel image.
[0058] Step S604: Determine the value of the first loss function based on the first blood vessel image and the second blood vessel image.
[0059] In one alternative implementation, the first loss function is the cross-entropy loss function (also known as the BCE loss function).
[0060] Step S605: Adjust the parameters of the image inpainting model based on the first loss function value.
[0061] The image inpainting model training method provided in this embodiment can not only solve the problem that the image inpainting model in related technologies cannot meet the actual needs when the blood vessels in the image are blurred; it also eliminates the need to annotate the original CT images; furthermore, since multiple original CT images come from multiple data centers, it can also increase the robustness of the image inpainting model and improve its performance.
[0062] This embodiment provides a training method for an image filling model, which can be used with computer devices. Figure 8 This is a flowchart of another image filling model training method according to an embodiment of the present invention. Figure 9 This is a schematic diagram of the image filling model training method according to an embodiment of the invention, such as... Figure 8 As shown, the training process for the image inpainting model includes the following steps:
[0063] Step S801: Obtain training sample images and corresponding real label images. The training sample images are the edge maps of the CT images to be filled.
[0064] Step S802: Input the training sample images into the image inpainting model to generate CT inpainted images.
[0065] like Figure 9 As shown, training the image inpainting model requires the use of adversarial networks and local optimization networks. The adversarial network includes a generator and a discriminator. The generator generates a CT-inpainted image from the input edge map; the discriminator distinguishes between the original CT image and the CT-inpainted image, determining a second loss function value based on them. The local optimization network extracts blood vessel images from the CT-inpainted image and the ground truth label image, respectively, to obtain a first blood vessel image and a second blood vessel image; a first loss function value is then determined based on the first and second blood vessel images.
[0066] The generator is a DeepLab decoder, whose purpose is to generate a CT-filled image from the input edge map. To reduce the impact of excessive density differences between images during training, the images are reconstructed by the infilling network, and a second loss function value is calculated with the original CT image. The parameters of the image infilling model are then updated through backpropagation, thereby supervising the network to learn how to fill the boundary map into a complete CT-filled image.
[0067] Specifically, before training the image infilling model by inputting the training sample images into the model, the training sample images are preprocessed, including normalization, standardization, and data augmentation.
[0068] For example, the normalized result of the training sample image = original CT image / 255; the standardized result of the training sample image = (normalized result of the training sample image – 0.373) / 0.641. Where 0.373 is the pixel mean of all original CT images, and 0.641 is the pixel standard deviation of all original CT images.
[0069] Step S803: Obtain the second loss function value based on the original CT image and the CT-filled image.
[0070] like Figure 9 As shown, the discriminator consists of a Mobile Net encoder, which aims to learn and distinguish between the original CT image and the CT-filled image after network filling at the semantic level, and update the parameters of the image filling model and the image discrimination network through a second loss function.
[0071] In one alternative implementation, the second loss function is an absolute loss function (also known as the L1 loss function).
[0072] Specifically, the formula for calculating L1 loss is as follows:
[0073]
[0074] The purpose of the L1 loss function is to calculate the model's predicted values. True value X iThe average of the absolute differences between them.
[0075] Step S804: Extract blood vessel images from the CT filled image and the real label image respectively to obtain the first blood vessel image and the second blood vessel image.
[0076] In one optional implementation, extracting vascular images from the CT-filled image and the ground truth labeled image to obtain a first vascular image and a second vascular image includes the following steps:
[0077] Step S8041: Encode the CT infill image and the original CT image using a local optimization network to obtain the first encoded image and the second encoded image.
[0078] In one alternative implementation, the local optimization network is a ResNet50 pre-trained from imageNet.
[0079] Step S8042: Binarize the first coded image and the second coded image respectively to obtain the first blood vessel image and the second blood vessel image.
[0080] Step S805: Determine the value of the first loss function based on the first blood vessel image and the second blood vessel image.
[0081] In one alternative implementation, the first loss function is the cross-entropy loss function (also known as the BCE loss function).
[0082] Specifically, the formula for calculating BCE loss is as follows:
[0083]
[0084] For example, such as Figure 9 As shown, the local optimization network is used to encode and binarize the CT-filled image and the original CT image, obtaining a first probability distribution P(X') and a second probability distribution P(X). Finally, the cross-entropy loss function between the first probability distribution P(X') and the second probability distribution P(X) is calculated. The cross-entropy loss function can be used to supervise the image filling model, and the parameters of the image filling model can be updated through backpropagation. The closer the image filled by the network is to the original CT image, the lower the value of the first loss function generated by the local optimization network will be.
[0085] Step S806: Adjust the parameters of the image inpainting model based on the first loss function value and the second loss function value.
[0086] For example, the initial learning rate during training is set to 0.001 for the image generator and 0.0005 for the discriminator. The training duration is 200 epochs. The learning rate update method is WarmUp + Cosine Annealing, meaning that in the first epoch, the learning rate is warmed up with 0.0001, and subsequent epochs use the formula T = 50 and eta = 0.0005. The optimizer used during training is the SGD optimizer, with momentum set to 0.9.
[0087] This embodiment uses Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) to compare models trained on single-data-center data to evaluate the overall performance of the model. Peak Signal-to-Noise Ratio (PSNR) is the ratio of the energy of the peak signal to the average energy of the noise, usually expressed as logarithmically in decibels (dB). Since MSE is the average energy difference between the real image and the noisy image, and the difference is the noise, PSNR is the ratio of peak signal energy to MSE. A higher PSNR indicates that the quality of the generated image is closer to the original image, meaning less noise. The formula is as follows:
[0088]
[0089] SSIM evaluates the similarity between two images based on three aspects: brightness, contrast, and structure. A higher SSIM value indicates greater similarity. A comparison of metrics between multi-datacenter and single-datacenter images is shown below:
[0090] Single data center Multi-datacenter PSNR 11.11 11.36 SSIM 0.76 0.78 L1 loss 0.0223 0.0209
[0091] As can be seen from the above comparison, this embodiment uses multi-data center data for training, making the model more robust to different data. It can effectively fill in medical data from different data centers, thereby increasing CT data and reducing the cost of acquiring CT images.
[0092] As can be seen from the above comparison, this embodiment uses multi-data center data for training, making the model more robust to different data. It can effectively fill in medical data from different data centers, thereby increasing CT data and reducing the cost of acquiring CT images.
[0093] The image inpainting model training method provided in this embodiment can not only improve the quality of blood vessels in the images generated by the image inpainting model, but also eliminates the need to annotate the original CT images. Furthermore, since multiple original CT images come from multiple data centers, it can also increase the robustness of the image inpainting model and improve its performance.
[0094] In summary, this invention provides a multi-center unlabeled lung CT image filling model that combines data from multiple medical image data centers, making the model more robust. Furthermore, a local optimization network is designed to further refine the details (mainly blood vessels) in the CT image, making the image quality closer to the original image.
[0095] This embodiment also provides a training apparatus for an image inpainting model, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.
[0096] This embodiment provides a training device for an image filling model, such as... Figure 10 As shown, it includes:
[0097] The acquisition module 1001 is used to acquire training sample images and corresponding real label images. The training sample images are the edge maps of the CT images to be filled.
[0098] The CT infill image generation module 1002 is used to input training sample images into the image infill model to generate CT infill images;
[0099] The blood vessel image determination module 1003 is used to extract blood vessel images from CT filled images and real label images respectively to obtain a first blood vessel image and a second blood vessel image;
[0100] The first loss function value determination module 1004 is used to determine the first loss function value based on the first blood vessel image and the second blood vessel image;
[0101] The parameter adjustment module 1005 is used to adjust the parameters of the image inpainting model based on the first loss function value.
[0102] In one optional implementation, the training apparatus for the image inpainting model further includes a sample processing module. The sample processing module is used to acquire multiple original CT images; perform edge detection on the original CT images to obtain an edge map; use the edge map as training sample images, and use the original CT images as ground truth label images.
[0103] In one optional implementation, the training apparatus for the image inpainting model further includes a second loss function value determination module. The second loss function value determination module is used to obtain a second loss function value based on the original CT image and the inpainted CT image; the parameter adjustment module 1005 is used to adjust the parameters of the image inpainting model based on the first and second loss function values.
[0104] In one optional implementation, the vascular image determination module 1003 is specifically used to: encode the CT filled image and the original CT image using a local optimization network to obtain a first encoded image and a second encoded image; and to perform binarization processing on the first encoded image and the second encoded image to obtain a first vascular image and a second vascular image.
[0105] In one optional implementation, the sample processing module includes an acquisition unit and an edge detection unit, wherein the edge detection unit is used to filter the original CT image; and to perform edge detection on the filtered original CT image using a preset operator to obtain an edge map, wherein the edge map includes a first edge map other than the lung window and / or a second edge map other than the mediastinal window.
[0106] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.
[0107] In this embodiment, the training device for the image filling model is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.
[0108] This invention also provides a computer device having the above-described features. Figure 10 The training of the image filling model is shown.
[0109] Please see Figure 11 , Figure 11 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 11 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 11 Take a processor 10 as an example.
[0110] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.
[0111] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.
[0112] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
[0113] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.
[0114] The computer device also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 40 can be connected via a bus or other means. Figure 11 Taking the example of a connection between China and Israel via a bus.
[0115] Input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the computer device, such as a touchscreen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touchscreen.
[0116] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.
[0117] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.
[0118] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and all such modifications and variations fall within the scope defined by the appended claims.
Claims
1. A training method for an image filling model, characterized in that, include: Obtain training sample images and corresponding ground truth label images, wherein the training sample images are edge maps of the CT images to be filled; The training sample images are input into the image inpainting model to generate CT inpainted images; The blood vessel images are extracted from the CT-filled image and the real label image, respectively, to obtain the first blood vessel image and the second blood vessel image; Based on the first blood vessel image and the second blood vessel image, determine the value of the first loss function; The parameters of the image infilling model are adjusted based on the first loss function value; Also includes: Obtain multiple original CT images; Edge detection is performed on the original CT image to obtain an edge map; The edge map is used as the training sample image, and the original CT image is used as the real label image; The step of extracting blood vessel images from the CT-filled image and the ground-labeled image respectively to obtain a first blood vessel image and a second blood vessel image includes: The CT infill image and the original CT image are encoded using a local optimization network to obtain a first encoded image and a second encoded image; wherein the local optimization network includes a backbone network and a local attention module, and wherein the backbone network is ResNet50. The first encoded image and the second encoded image are binarized respectively to obtain the first blood vessel image and the second blood vessel image.
2. The method of claim 1, wherein, Also includes: The second loss function value is obtained based on the original CT image and the incomplete CT image; The adjustment of the parameters of the image infilling model based on the first loss function value includes: The parameters of the image infilling model are adjusted based on the first loss function value and the second loss function value.
3. The method of claim 1, wherein, The step of performing edge detection on the original CT image to obtain an edge map includes: The original CT image is filtered; The filtered original CT image is used to perform edge detection using a preset operator to obtain the edge map, wherein the edge map includes a first edge map other than the lung window and / or a second edge map other than the mediastinal window.
4. The method of claim 1, wherein, The multiple original CT images belong to multiple data centers.
5. A training device for an image filling model, characterized in that, include: The acquisition module is used to acquire training sample images and corresponding real label images, wherein the training sample images are edge maps of the CT images to be filled; The CT infill image generation module is used to input the training sample images into the image infill model to generate CT infill images; A blood vessel image determination module is used to extract blood vessel images from the CT filled image and the real label image respectively to obtain a first blood vessel image and a second blood vessel image; The first loss function value determination module is used to determine the first loss function value based on the first blood vessel image and the second blood vessel image; The parameter adjustment module is used to adjust the parameters of the image infilling model based on the first loss function value; The training device for the image filling model also includes a sample processing module, which is used to acquire multiple original CT images; perform edge detection on the original CT images to obtain an edge map; use the edge map as the training sample image and the original CT images as the ground truth label image; The blood vessel image determination module is specifically used to: encode the CT filled image and the original CT image using a local optimization network to obtain a first encoded image and a second encoded image; wherein the local optimization network includes a backbone network and a local attention module, wherein the backbone network is ResNet50; and to perform binarization processing on the first encoded image and the second encoded image to obtain the first blood vessel image and the second blood vessel image.
6. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the training method of the image filling model according to any one of claims 1 to 4.
7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the training method of the image filling model according to any one of claims 1 to 4.
8. A computer program product, characterized in that, Includes computer instructions for causing a computer to perform the training method of the image filling model according to any one of claims 1 to 4.