A method for generating a denoised vascular network image and a related device
By inputting the vascular network image to the multimodal convolutional neural network denoising model, extracting and fusing features, the problem of long scanning time in the prior art is solved, and high signal-to-noise ratio and convenient denoising vascular network image generation is achieved.
Patent Information
- Application Number
- CN202410955608.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-17
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-07-17
AI Technical Summary
The prior art requires increasing the number of repeated acquisitions to improve the signal-to-noise ratio when acquiring denoising blood vessel network images, but this leads to an increase in scanning time, increasing cost and time.
The original vascular network image and the whole vascular network segmented image are input to the multimodal convolutional neural network denoising model, and the vascular network features are extracted through the downsampling and upsampling modules, and the denoising vascular network image is generated after fusion.
It improves the convenience and signal-to-noise ratio of obtaining denoised blood vessel network images, reduces material costs, and increases image details.
Smart Images

Figure CN118674811B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a method for generating a denoised vascular network image and a related device. Background Art
[0002] Optical coherence tomography (OCT) is a non-invasive, non-contact, rapid imaging method that is widely used in the diagnosis of clinical ophthalmic diseases such as retinal perfusion and macular degeneration. Due to its fast imaging and high resolution, it is also widely used in the detection and imaging of capillary networks in the cerebral cortex of rodents.
[0003] At present, the steps of vascular network imaging generally include multiple OCT image acquisitions of the same part, OCT image alignment, vascular network image calculation, vascular network image enhancement and denoising, etc., and currently when obtaining denoised vascular network images, the signal-to-noise ratio of the blood flow signal is improved by increasing the number of repeated acquisitions. This method of improving the signal-to-noise ratio will lead to an increase in scanning time, and this method of increasing scanning time requires a lot of money and time.
[0004] Therefore, how to obtain denoised vascular network images is a problem that needs to be solved urgently. Summary of the invention
[0005] The embodiments of the present invention provide a method for generating a denoised vascular network image and a related device, which are used to input the original vascular network image and the whole vascular network segmentation image into a multimodal convolutional neural network denoising model respectively, so as to obtain a denoised vascular network image. This method of obtaining a denoised vascular network image can directly obtain the denoised vascular network image through the model, thereby improving the convenience of obtaining the denoised vascular network image and reducing the material cost of obtaining the denoised vascular network image.
[0006] A first aspect of an embodiment of the present application provides a method for generating a denoised vascular network image, comprising:
[0007] Obtaining the original vascular network image and the whole vascular network segmentation image;
[0008] Inputting the original vascular network image into the first downsampling module in the pre-trained multimodal convolutional neural network denoising model, the multimodal convolutional neural network denoising model includes a first downsampling module, an image segmentation feature downsampling module and a first upsampling module, wherein the first downsampling module is used to extract context information and vascular network features in the original vascular network image to obtain a first vascular network feature map output by the first downsampling module;
[0009] The whole vascular network segmentation image is input into the image segmentation feature downsampling module to extract the vascular network structure features in the whole vascular network segmentation image, and obtain a second vascular network feature map output by the image segmentation feature downsampling module;
[0010] fusing the first vascular network characteristic map and the second vascular network characteristic map to obtain a third vascular network characteristic map;
[0011] The third vascular network feature map is input into the first upsampling module, wherein the first upsampling module is used to restore the image size of the third vascular network feature map to the size of the original vascular network image to obtain a denoised vascular network image output by the first upsampling module.
[0012] A second aspect of an embodiment of the present application provides a device for generating a denoised vascular network image, comprising:
[0013] An acquisition unit, used for acquiring an original vascular network image and a whole vascular network segmentation image;
[0014] An input-output unit, used to input the original vascular network image into a first down-sampling module in a pre-trained multimodal convolutional neural network denoising model, wherein the multimodal convolutional neural network denoising model includes a first down-sampling module, an image segmentation feature down-sampling module and a first up-sampling module, wherein the first down-sampling module is used to extract context information and vascular network features from the original vascular network image to obtain a first vascular network feature map output by the first down-sampling module;
[0015] The input-output unit is further used to input the whole vascular network segmentation image into the image segmentation feature downsampling module to extract the vascular network structure features in the whole vascular network segmentation image, and obtain a second vascular network feature map output by the image segmentation feature downsampling module;
[0016] a processing unit, configured to fuse the first vascular network characteristic map and the second vascular network characteristic map to obtain a third vascular network characteristic map;
[0017] The input-output unit is also used to input the third vascular network feature map into the first upsampling module, wherein the first upsampling module is used to restore the image size of the third vascular network feature map to the size of the original vascular network image to obtain a denoised vascular network image output by the first upsampling module.
[0018] A third aspect of an embodiment of the present application provides a computer device, including a processor, which, when executing a computer program stored in a memory, is used to implement the method for generating a denoised vascular network image provided in the first aspect of an embodiment of the present application.
[0019] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for generating a denoised vascular network image provided in the first aspect of an embodiment of the present application is implemented.
[0020] A fifth aspect of an embodiment of the present application provides a computer program product having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, it is used to implement the method for generating a denoised vascular network image provided in the first aspect of an embodiment of the present application.
[0021] It can be seen from the above technical solutions that the embodiments of the present invention have the following advantages:
[0022] In an embodiment of the present application, the original vascular network image is input into the first down-sampling module in the multimodal convolutional neural network denoising model to extract the contextual information and vascular network features in the original vascular network image, and the whole vascular network segmentation image is input into the image segmentation feature down-sampling module in the multimodal convolutional neural network denoising model to extract the vascular network structural features in the whole vascular network segmentation image, and then the first vascular network feature map output by the first down-sampling module and the second vascular network feature map output by the image segmentation feature down-sampling module are fused to obtain a third vascular network feature map with a high signal-to-noise ratio including vascular network structural features, and finally the third vascular network feature map is input into the first up-sampling module to restore the image size of the third vascular network feature map to the size of the original vascular network image to obtain a denoised vascular network image.
[0023] Because the denoised vascular network image output by the multimodal convolutional neural network denoising model is obtained through model training, compared with the existing technology, it improves the convenience of obtaining the denoised vascular network image and reduces the material cost of obtaining the denoised vascular network image.
[0024] Furthermore, the embodiment of the present application also inputs the whole vascular network segmentation image into the image feature downsampling module to extract the vascular network structure features in the whole vascular network segmentation image. Therefore, the third vascular network feature map obtained in the embodiment of the present application contains more features about the vascular network structure. Therefore, the denoised vascular network image output by the embodiment of the present application contains more detailed information and a higher signal-to-noise ratio. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 This is a schematic diagram of the architecture of a system for generating a denoised vascular network image in an embodiment of the present application;
[0026] Figure 2 A schematic diagram of an embodiment of generating a denoised vascular network image in an embodiment of the present application;
[0027] Figure 3Schematic diagram of the structure of a multimodal convolutional neural network denoising model in an embodiment of the present application;
[0028] Figure 4 Schematic diagram of comparison between the original vascular network image and the denoised vascular network image in the embodiment of the present application;
[0029] Figure 5 A schematic diagram of another embodiment of the method for generating a denoised vascular network image in an embodiment of the present application;
[0030] Figure 6 Schematic diagram of the structure of the optimized multimodal convolutional neural network denoising model in the embodiment of the present application;
[0031] Figure 7 A schematic diagram of an embodiment of the training process of the multimodal convolutional neural network denoising model in the embodiment of the present application;
[0032] Figure 8 A schematic diagram showing the comparison between the original vascular network image, the preliminary denoised vascular network image and the high signal-to-noise ratio vascular network image in the embodiment of the present application;
[0033] Fig. 9 This is a schematic diagram of an embodiment of training a vascular network denoising model in an embodiment of the present application;
[0034] Fig.10 A schematic diagram of an embodiment of generating a preliminary denoised vascular network image in an embodiment of the present application;
[0035] Fig.11 Schematic diagram of an embodiment of a device for generating a denoised vascular network image in an embodiment of the present application. DETAILED DESCRIPTION
[0036] The embodiments of the present invention provide a method for generating a denoised vascular network image and a related device, which are used to input the original vascular network image and the whole vascular network segmentation image into a multimodal convolutional neural network denoising model respectively, so as to obtain a denoised vascular network image. This method of obtaining a denoised vascular network image can directly obtain the denoised vascular network image through the model, thereby improving the convenience of obtaining the denoised vascular network image and reducing the material cost of obtaining the denoised vascular network image.
[0037] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0038] The terms "first", "second", "third", "fourth", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0039] The present application embodiment provides a method for generating a denoised vascular network image. The general principle of the method is as follows:
[0040] Acquire an original vascular network image and a full vascular network segmentation image; input the original vascular network image into a first down-sampling module in a pre-trained multimodal convolutional neural network denoising model, the multimodal convolutional neural network denoising model comprising a first down-sampling module, an image segmentation feature down-sampling module and a first up-sampling module, wherein the first down-sampling module is used to extract context information and vascular network features in the original vascular network image to obtain a first vascular network feature map output by the first down-sampling module; input the full vascular network segmentation image into the image segmentation feature down-sampling module to extract vascular network structural features in the full vascular network segmentation image to obtain a second vascular network feature map output by the image segmentation feature down-sampling module; fuse the first vascular network feature map and the second vascular network feature map to obtain a third vascular network feature map; input the third vascular network feature map into the first up-sampling module, wherein the first up-sampling module is used to restore the image size of the third vascular network feature map to the size of the original vascular network image to obtain a denoised vascular network image output by the first up-sampling module. In the embodiment of the present application, the original vascular network image and the whole vascular network segmentation image are respectively input into the multimodal convolutional neural network denoising model to obtain a denoised vascular network image. This method of generating a denoised vascular network image through a model not only improves the convenience of obtaining the denoised vascular network image, but also reduces the material cost of obtaining the denoised vascular network image.
[0041] In order to better implement the above method for generating a denoised vascular network image, the present application embodiment provides a system for generating a denoised vascular network image. Figure 1 , Figure 1A schematic diagram of the architecture of a system for generating a denoised vascular network image provided in an embodiment of the present application. The system for generating a denoised vascular network image may include at least one terminal device 102 and a server 101; different types of applications may be installed on the terminal device 102, for example, a Meitu Xiuxiu application, an instant communication application, a conference communication application, etc. may be installed on the terminal device 102; the terminal device 102 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart car, etc. The server 101 may be used to store application data and image data generated by different types of applications of the terminal device 102. The server 101 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms, etc.
[0042] Among them, the above-mentioned method for generating a denoised vascular network image can be executed by the terminal device 102 or the server 101. When the method for generating a denoised vascular network image is executed by the terminal device 102, the original vascular network image and the whole vascular network segmentation image required by the terminal device 102 can be included in the server. Then, when the terminal device 102 needs to process the original vascular network image and the whole vascular network segmentation image, the terminal device 102 can obtain the original vascular network image and the whole vascular network segmentation image from the server 101. Then, after the terminal device 102 obtains the original vascular network image and the whole vascular network segmentation image from the server 101, it can input them into the multimodal convolutional neural network denoising model stored in the terminal device 102, thereby obtaining the denoised vascular network image output by the multimodal convolutional neural network denoising model, thereby improving the convenience of obtaining the denoised vascular network image and reducing the material cost of obtaining the denoised vascular network image.
[0043] For ease of understanding, the method for generating a denoised vascular network image in an embodiment of the present application is described below. Figure 2 , an embodiment of generating a denoised vascular network image in an embodiment of the present application includes:
[0044] 201. Obtaining an original vascular network image and a whole vascular network segmentation image;
[0045] Specifically, in the embodiment of the present application, when acquiring the original vascular network image, the original vascular network image may be acquired from the public data set OCT500, or the original vascular network image may be acquired by collecting data.
[0046] Specifically, when acquiring the original vascular network image by collecting data, it is necessary to obtain multiple B-Scan images at the same horizontal position, scan the entire area from top to bottom, and thus form the entire blood flow map. In order to ensure that accurate blood flow signals are obtained at specific positions, at least two B-Scan images need to be collected, so two repeated scans are required. This is to ensure that there is sufficient data for blood flow calculation, and increasing the number of repeated acquisitions will increase the signal-to-noise ratio of the blood flow signal. Although increasing the number of repeated acquisitions can improve the signal-to-noise ratio, it will also increase the scanning time. Increasing the scanning time will increase the risk of eye movement noise, thereby affecting the clarity and accuracy of the image.
[0047] As an optional embodiment, the embodiment of the present application comprehensively considers the number of repeated scans, the scanning time and the image characteristics of the acquisition instrument, and sets the number of repeated scans to 4.
[0048] Furthermore, when acquiring a whole vascular network segmentation image, the embodiments of the present application may utilize an existing whole vascular network segmentation image corresponding to the original vascular network image, or utilize the original vascular network image and the whole vascular network segmentation model to generate a whole vascular network segmentation image. The process of generating a whole vascular network image using the original vascular network image and the whole vascular network segmentation model will be described in the following embodiments and will not be repeated here.
[0049] 202. Inputting the original vascular network image into a first downsampling module in a pre-trained multimodal convolutional neural network denoising model, the multimodal convolutional neural network denoising model comprising a first downsampling module, an image segmentation feature downsampling module and a first upsampling module, wherein the first downsampling module is used to extract context information and vascular network features from the original vascular network image to obtain a first vascular network feature map output by the first downsampling module;
[0050] Specifically, the multimodal convolutional neural network denoising model in the embodiment of the present application includes a first down-sampling module, an image segmentation feature down-sampling module and a first up-sampling module, wherein each module includes a convolution layer, a batch normalization layer and an activation function layer, but the role of the convolution layer in each module is different. For example, the role of the convolution layer in the first down-sampling module is to extract contextual information and vascular network features in the original vascular network image to obtain the first vascular network feature map output by the first down-sampling module. The roles of the convolution layers in the image segmentation feature down-sampling module and the first up-sampling module will be described in the following embodiments and will not be repeated here.
[0051] For ease of understanding, Figure 3 A structural schematic diagram of the multimodal convolutional neural network denoising model is given.
[0052] Furthermore, the multimodal convolutional neural network denoising model in the embodiment of the present application can be a multimodal convolutional neural network model based on Unet, or it can be a multimodal convolutional neural network model based on CNN, and no specific limitation is made here.
[0053] When the multimodal convolutional neural network denoising model is a Unet-based multimodal convolutional neural network model, the model adopts a U-shaped structure, extracts features by performing a convolution operation on the input image, and then uses a deconvolution operation to restore the feature map to its original size, thereby obtaining a denoised vascular network image.
[0054] When the multimodal convolutional neural network denoising model is a convolutional neural network model based on CNN, wherein CNN is mainly composed of a convolutional layer, a pooling layer and a fully connected layer, the convolutional layer extracts features by performing a convolution operation on the input image, the pooling layer reduces the size of the feature map by downsampling, and the fully connected layer maps the feature map to the output structure space.
[0055] 203. Input the whole vascular network segmentation image into the image segmentation feature downsampling module to extract the vascular network structure features in the whole vascular network segmentation image, and obtain a second vascular network feature map output by the image segmentation feature downsampling module;
[0056] In order to obtain the features of the vascular structure in the vascular network image, in the embodiment of the present application, the whole vascular network segmentation image is also input into the image segmentation feature downsampling module in the multimodal convolutional neural network denoising model to extract the vascular network structure features in the whole vascular network segmentation image, thereby obtaining the second vascular network feature map output by the image segmentation feature downsampling module.
[0057] Specifically, in the image segmentation feature downsampling module, in order to obtain the structural features of the vascular network, a convolution layer composed of dilated convolution blocks is used in the image segmentation feature downsampling module in the embodiment of the present application, because dilated convolution blocks of different sizes can learn features of different scales, and the dilated convolution blocks can also capture a wider range of contextual information by expanding the receptive field, thereby retaining more detailed information of the vascular network structure.
[0058] As an optional embodiment, the convolution kernel size of the dilated convolution block in the embodiment of the present application is 3*3, and the dilation ratio is 2, so that for the whole vascular network segmentation image, more detailed vascular network structural features can be captured on the basis of expanding the receptive field.
[0059] As another optional embodiment, the image segmentation feature downsampling module in the embodiment of the present application can also adopt a standard convolution block and an attention module, wherein the standard convolution block is used to extract the vascular network structural features in the whole vascular network segmentation image, and the attention module is used to enhance the vascular network structural features extracted from the whole vascular network segmentation image to obtain a vascular network image with more vascular network structural features.
[0060] 204. Merge the first vascular network characteristic map and the second vascular network characteristic map to obtain a third vascular network characteristic map;
[0061] In order to obtain a vascular network structure characteristic map with a higher signal-to-noise ratio, the embodiment of the present application fuses the first vascular network characteristic map and the second vascular network characteristic map after obtaining the first vascular network characteristic map and the second vascular network characteristic map, thereby obtaining a third vascular network characteristic map.
[0062] Specifically, in the process of obtaining the third vascular network characteristic map, the first vascular network characteristic map and the second vascular network characteristic map may be fused by a connection or superposition method, wherein if the fusion is performed by superposition, the pixel values at the same position of the first vascular network structure characteristic map and the second vascular network structure characteristic map are superimposed to obtain the fused third vascular network characteristic map;
[0063] If the first vascular network feature map and the second vascular network feature map are fused by connection, the number of channels of the first vascular network feature map and the number of channels of the second vascular network feature map are combined together. For example, when the first vascular network feature map is a feature map of 2 channels and the second vascular network feature map is also a feature map of 2 channels, the third vascular network feature map after connection is a feature map of 4 channels.
[0064] 205. Input the third vascular network feature map to the first upsampling module, wherein the first upsampling module is used to restore the image size of the third vascular network feature map to the size of the original vascular network image to obtain a denoised vascular network image output by the first upsampling module.
[0065] After obtaining the third vascular network feature map, the third vascular network feature map is further input into the first upsampling module in the multimodal convolutional neural network denoising model, wherein the first upsampling module includes a convolution layer, a batch normalization layer and an activation function layer, wherein the function of the first upsampling module is to restore the image size of the third vascular network feature map to the size of the original vascular network image to obtain the denoised vascular network image output by the first upsampling module.
[0066] For ease of understanding, Figure 4 A schematic diagram comparing the original vascular network image and the denoised vascular network image is given.
[0067] In an embodiment of the present application, the original vascular network image is input into the first down-sampling module in the multimodal convolutional neural network denoising model to extract the contextual information and vascular network features in the original vascular network image, and the whole vascular network segmentation image is input into the image segmentation feature down-sampling module in the multimodal convolutional neural network denoising model to extract the vascular network structural features in the whole vascular network segmentation image, and then the first vascular network feature map output by the first down-sampling module and the second vascular network feature map output by the image segmentation feature down-sampling module are fused to obtain a third vascular network feature map with a high signal-to-noise ratio including vascular network structural features, and finally the third vascular network feature map is input into the first up-sampling module to restore the image size of the third vascular network feature map to the size of the original vascular network image to obtain a denoised vascular network image.
[0068] Because the denoised vascular network image output by the multimodal convolutional neural network denoising model is obtained through model training, compared with the existing technology, it improves the convenience of obtaining the denoised vascular network image and reduces the material cost of obtaining the denoised vascular network image.
[0069] Furthermore, the embodiment of the present application also inputs the whole vascular network segmentation image into the image feature downsampling module to extract the vascular network structure features in the whole vascular network segmentation image. Therefore, the third vascular network feature map obtained in the embodiment of the present application contains more features about the vascular network structure. Therefore, the denoised vascular network image output by the embodiment of the present application contains more detailed information and a higher signal-to-noise ratio.
[0070] based on Figure 2 In the embodiment, after obtaining the second vascular network feature map, in order to enhance the vascular network structure feature in the second vascular network feature, the following embodiments may also be performed. Figure 5 Another embodiment of the method for generating a denoised vascular network image in the embodiment of the present application includes:
[0071] 501. Acquire an original vascular network image and a segmented image of the entire vascular network;
[0072] 502. Input the original vascular network image into a first down-sampling module in a pre-trained multimodal convolutional neural network denoising model, wherein the multimodal convolutional neural network denoising model includes a first down-sampling module, an image segmentation feature down-sampling module, a self-attention module and a first up-sampling module, wherein the first down-sampling module is used to extract context information and vascular network features in the original vascular network image to obtain a first vascular network feature map output by the first down-sampling module;
[0073] 503. Input the whole vascular network segmentation image into the image segmentation feature downsampling module to extract the vascular network structure features in the whole vascular network segmentation image, and obtain a second vascular network feature map output by the image segmentation feature downsampling module;
[0074] It should be noted that the description of step 501 to step 503 in the embodiment of the present application is the same as Figure 2 The descriptions of steps 201 to 203 in the embodiment are similar and will not be repeated here.
[0075] 504. Input the second vascular network feature map into a self-attention module in a multimodal convolutional neural network denoising model, wherein the self-attention module is used to enhance the vascular network structural features in the second vascular network feature map to obtain an enhanced second vascular network feature map;
[0076] Furthermore, in order to strengthen the vascular network structural features in the second vascular network feature map, the second vascular network feature map can also be input into the self-attention module in the multimodal convolutional neural network denoising model to strengthen the vascular network structural features in the second vascular network feature map to obtain an enhanced second vascular network structural feature map.
[0077] Specifically, the self-attention module in the embodiment of the present application is a module for processing a vascular network structure feature map, which can automatically learn the relationship between each position in the vascular network structure feature map, thereby extracting the vascular network structure features in the vascular network structure feature map. The self-attention module obtains the weights of each position in the vascular network structure feature map by performing attention calculation on each position in the vascular network structure feature map, and then performs weighted summation on the vascular network structure feature map according to these weights to obtain a representation of the vascular network structure feature map.
[0078] 505. Fusing the first vascular network characteristic map and the enhanced second vascular network characteristic map to obtain a third vascular network characteristic map;
[0079] After obtaining the enhanced second vascular network structure characteristic map, the first vascular network structure characteristic map and the enhanced vascular network structure map are fused to obtain a third vascular network characteristic map.
[0080] 506. Input the third vascular network feature map into the first upsampling module in the multimodal convolutional neural network denoising model, wherein the first upsampling module is used to restore the image size of the third vascular network feature map to the size of the original vascular network image to obtain a denoised vascular network image output by the first upsampling module.
[0081] The description of step 506 is the same as Figure 2 The description of step 205 in the embodiment is similar and will not be repeated here.
[0082] For ease of understanding, Figure 6 The structural schematic diagram of the optimized multimodal convolutional neural network denoising module is given.
[0083] In the embodiment of the present application, relative to Figure 2 In the embodiment, the second vascular network feature map is input into the self-attention module to obtain an enhanced second vascular network feature map output by the self-attention module. The enhanced second vascular network feature map increases the weight of the vascular network structure part compared to the second vascular network feature map. Therefore, the denoised vascular network image output in the embodiment of the present application has a higher signal-to-noise ratio in terms of the vascular network structure characteristics.
[0084] based on Figure 2 and Figure 5 In the embodiment, before the original vascular network image and the whole vascular network segmentation image are input into the multimodal convolutional neural network denoising model, the multimodal convolutional neural network denoising model needs to be trained. The training process of the multimodal convolutional neural network denoising model is described below. Please refer to Figure 7 , an embodiment of the training process of the multimodal convolutional neural network denoising model in the embodiment of the present application includes:
[0085] 701. Obtain a first training sample, wherein the first training sample includes an original vascular network image;
[0086] Specifically, before training the multimodal convolutional neural network denoising model, it is necessary to first obtain a first training sample and a first training label, and use the first training sample and the first training label to train the multimodal convolutional neural network denoising model.
[0087] Specifically, the first training sample in the embodiment of the present application includes an original vascular network image, wherein the acquisition process of the original vascular network image is the same as Figure 2 The description in the embodiment is consistent and will not be repeated here.
[0088] 702. Acquire a first training label, wherein the first training label includes a high signal-to-noise ratio vascular network image;
[0089] Specifically, in the embodiment of the present application, supervised learning is used to train the multimodal convolutional neural network denoising model. Therefore, when training the multimodal convolutional neural network denoising model, it is also necessary to obtain a first training label, wherein the first training label includes a high signal-to-noise ratio vascular network image, wherein the high signal-to-noise ratio vascular network image is relative to the original vascular network image.
[0090] 703. Using the original vascular network image and the high signal-to-noise ratio vascular network image, the multimodal convolutional neural network denoising model is trained, wherein the generation process of the high signal-to-noise ratio vascular network image includes: obtaining a preliminary denoised vascular network image, inputting the preliminary denoised vascular network image into a second down-sampling module in the vascular network denoising model, wherein the vascular network denoising model includes a second down-sampling module and a second up-sampling module, and the second down-sampling module is used to extract the vascular network features in the preliminary denoised vascular network image to obtain a fourth vascular network feature map output by the second down-sampling module; inputting the fourth vascular network feature map into the second up-sampling module in the vascular network denoising model, wherein the second up-sampling module is used to restore the size of the fourth vascular network feature to the size of the preliminary denoised vascular network image to obtain a high signal-to-noise ratio vascular network image output by the up-sampling module.
[0091] After obtaining the first training sample (original vascular network image) and the first training label (high signal-to-noise ratio vascular network image), the original vascular network image and the high signal-to-noise ratio vascular network image are used to train the multimodal convolutional neural network denoising model.
[0092] The specific training process is: after the original vascular network image is input into the multimodal convolutional neural network denoising model, the predicted denoised vascular network image output by the multimodal convolutional neural network denoising model is obtained, and then the preset loss function (such as mean square error loss function or mean absolute error loss function) is used to calculate the loss between the predicted noisy vascular network image and the high signal-to-noise ratio vascular network image, and then the loss and back propagation algorithm are used to continuously update and iterate the parameters of the multimodal convolutional neural network denoising model until the multimodal convolutional neural network denoising model converges.
[0093] The following describes the process of generating a high signal-to-noise ratio vascular network image:
[0094] Specifically, when generating a high signal-to-noise ratio vascular network image, the embodiment of the present application needs to first obtain a preliminary denoised vascular network image, and then input the preliminary denoised vascular network image into the vascular network denoising model to obtain a high signal-to-noise ratio vascular network image output by the vascular network denoising model, wherein the vascular network denoising model in the embodiment of the present application includes a second down-sampling module and a second up-sampling module, and the second down-sampling module and the second up-sampling module respectively include a convolution layer, a batch normalization layer and an activation function layer, wherein the second down-sampling module is used to extract the vascular network features in the preliminary denoised vascular network image to obtain a fourth vascular network feature map output by the second down-sampling module, and the second up-sampling module is used to restore the size of the fourth vascular network feature map to the size of the preliminary denoised vascular network image to obtain a high signal-to-noise ratio vascular network image output by the second up-sampling module.
[0095] For ease of understanding, Figure 8A schematic diagram of the comparison between the original vascular network image, the preliminary denoised vascular network image and the high signal-to-noise ratio vascular network image is given.
[0096] The training process of the multimodal convolutional neural network denoising model is described in detail in the embodiment of the present application, and in the process of training the multimodal convolutional neural network denoising model, a high signal-to-noise ratio vascular network image is used to perform supervised training on the multimodal convolutional neural network denoising model, thereby improving the signal-to-noise ratio of the denoised vascular network image output by the multimodal convolutional neural network denoising model.
[0097] Furthermore, when generating a high signal-to-noise ratio vascular network image, the embodiment of the present application inputs the preliminary denoised vascular network image into the vascular network denoising model, thereby further improving the convenience of obtaining a high signal-to-noise ratio vascular network image.
[0098] based on Figure 7 In the above-mentioned embodiment, before the preliminary denoised vascular network image is input into the vascular network denoising model, the vascular network denoising model needs to be trained. The training process of the vascular network denoising model is described below. Fig. 9 In the embodiment of the present application, one embodiment of training a vascular network denoising model includes:
[0099] 901. Obtain a first training sample, wherein the first training sample includes an original vascular network image;
[0100] It is easy to understand that before training the vascular network denoising model, it is necessary to obtain a first training sample and a second training label, wherein one training sample includes an original vascular network image, wherein the acquisition process of the original vascular network image is the same as Figure 2 The description of step 201 in the embodiment is similar and will not be repeated here.
[0101] 902. Acquire a second training label, wherein the second training label includes a preliminary denoised vascular network image;
[0102] The embodiment of the present application uses a supervised learning algorithm when training the vascular network denoising model, so step 902 also needs to obtain the training label of the vascular network denoising model. As an optional embodiment, the embodiment of the present application uses a preliminary denoised vascular network image as the training label of the vascular network denoising model.
[0103] 903. The vascular network denoising model is trained using the original vascular network image and the preliminary denoised vascular network image, wherein the generation process of the preliminary denoised vascular network image includes: inputting the original vascular network image into the coarse vascular network segmentation model to obtain the coarse vascular network segmentation image output by the coarse vascular network segmentation model, wherein the coarse vascular network segmentation model includes a third down-sampling module and a third up-sampling module; inputting the original vascular network image into the full vascular network segmentation model to obtain the full vascular network segmentation image output by the full vascular network segmentation model, wherein the full vascular network segmentation model includes a fourth down-sampling module and a fourth up-sampling module; and performing weighted averaging on the coarse vascular network segmentation image and the full vascular network segmentation image to obtain the preliminary denoised vascular network image.
[0104] After obtaining the first training sample (original vascular network image) and the second training label (preliminary denoised vascular network image), the back propagation algorithm is used to train the vascular network denoising model.
[0105] Specifically, the vascular network denoising model in the embodiment of the present application includes a second down-sampling module and a second up-sampling module, and the second down-sampling module and the second up-sampling module respectively include a convolution layer, a batch normalization layer and an activation function layer, and the training process of the vascular network denoising model is as follows: after the original vascular network image is input into the vascular network denoising model, the predicted preliminary signal-to-noise ratio vascular network image output by the vascular network denoising model is obtained, and then the loss value between the predicted preliminary signal-to-noise ratio vascular network image and the preliminary signal-to-noise ratio vascular network image is calculated using a preset loss function, and finally the model parameters of the vascular network denoising model are updated and iterated using the loss value and the back propagation algorithm until the vascular network denoising model converges.
[0106] Next, we will describe the generation process of the second training label, that is, the initial denoised vascular network image. Fig.10 :
[0107] The original vascular network image is input into the coarse vascular network segmentation model to obtain the coarse vascular network segmentation image output by the coarse vascular network segmentation model, wherein the coarse vascular network segmentation model includes a third down-sampling module and a third up-sampling module; the original vascular network is input into the full vascular network segmentation model to obtain the full vascular network segmentation image output by the full vascular network segmentation model, wherein the full vascular network segmentation model includes a fourth down-sampling module and a fourth up-sampling module; the coarse vascular network segmentation image and the full vascular network segmentation image are weighted averaged to obtain a preliminary denoised vascular network image.
[0108] It should be noted that the model structures of the coarse vascular network segmentation model and the whole vascular network segmentation model in the embodiment of the present application are the same, and both include a downsampling module and an upsampling module, respectively. However, when the original vascular network image is input into the coarse vascular network segmentation model and the whole vascular network segmentation model, the coarse vascular network segmentation image output by the coarse vascular network segmentation model and the whole vascular network segmentation image output by the whole vascular network segmentation model can be obtained. This is because different training labels are used when training the coarse vascular network segmentation model and the whole vascular network segmentation model. For example, when training the coarse vascular network segmentation model, the coarse vascular network segmentation model is trained using the coarse vascular network segmentation label and the original vascular network image in the OCT500 data set, while when training the whole vascular network segmentation model, the whole vascular network segmentation model is trained using the whole vascular network segmentation label and the original vascular network image. Therefore, when the original vascular network image is input into the coarse vascular network segmentation model and the whole vascular network segmentation model, the coarse vascular network segmentation image output by the coarse vascular network segmentation model and the whole vascular network segmentation image output by the whole vascular network segmentation model can be obtained, respectively.
[0109] Furthermore, in the process of training the coarse vascular network segmentation model and the full vascular network segmentation model, in order to capture the detailed features of the vascular network, the coarse vascular network segmentation label and the full vascular network segmentation label both use binary images of the vascular network image.
[0110] In the embodiment of the present application, the vascular network denoising model is trained using the first training sample and the second training label, and the preliminary denoised vascular network image is used as the training label of the vascular network denoising model, thereby improving the convenience of training the vascular network denoising model.
[0111] Furthermore, in the process of training the vascular network denoising model and the multimodal convolutional neural network denoising model, in order to improve the generalization performance of the vascular network denoising model and the multimodal convolutional neural network denoising model, as well as the diversity of training data, the embodiment of the present application may also process the original vascular network image by at least one of rotation, translation or brightness change before inputting the original vascular network image into the vascular network denoising model and into the multimodal convolutional neural network denoising model, so as to improve the diversity of training data and the generalization performance of the vascular network denoising model and the multimodal convolutional neural network denoising model.
[0112] It is understandable that, in various embodiments of the present invention, the sequence numbers of the above steps do not mean the order of execution. The order of execution of the steps should be determined by their functions and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0113] The above is a detailed description of the method for generating a denoised vascular network image in the embodiment of the present application. Next, the device for generating a denoised vascular network image in the embodiment of the present application is described in detail. Fig.11, an embodiment of the device for generating a denoised vascular network image in the embodiment of the present application includes:
[0114] An acquisition unit 1101 is used to acquire an original vascular network image and a whole vascular network segmentation image;
[0115] The input-output unit 1102 is used to input the original vascular network image into the first down-sampling module in the pre-trained multimodal convolutional neural network denoising model, and the multimodal convolutional neural network denoising model includes a first down-sampling module, an image segmentation feature down-sampling module and a first up-sampling module, wherein the first down-sampling module is used to extract context information and vascular network features in the original vascular network image to obtain a first vascular network feature map output by the first down-sampling module;
[0116] The input-output unit 1102 is further used to input the whole vascular network segmentation image into the image segmentation feature downsampling module to extract the vascular network structure features in the whole vascular network segmentation image, and obtain a second vascular network feature map output by the image segmentation feature downsampling module;
[0117] A processing unit 1103 is used to merge the first vascular network characteristic map and the second vascular network characteristic map to obtain a third vascular network characteristic map;
[0118] The input-output unit 1102 is also used to input the third vascular network feature map into the first upsampling module, wherein the first upsampling module is used to restore the image size of the third vascular network feature map to the size of the original vascular network image to obtain a denoised vascular network image output by the first upsampling module.
[0119] Preferably, the multimodal convolutional neural network denoising model further includes a self-attention module;
[0120] The input-output unit 1102 is further used for:
[0121] Inputting the second vascular network feature map into the self-attention module, wherein the self-attention module is used to enhance the vascular network structural features in the second vascular network feature map to obtain an enhanced second vascular network feature map;
[0122] The processing unit 1103 is specifically configured to:
[0123] The first vascular network characteristic map and the enhanced second vascular network characteristic map are fused to obtain a third vascular network characteristic map.
[0124] Preferably, the image segmentation feature downsampling module includes a plurality of atrous convolution blocks, or includes a plurality of standard convolution blocks and a self-attention module;
[0125] The multimodal convolutional neural network denoising model includes a Unet-based multimodal convolutional neural network or a CNN-based multimodal convolutional neural network.
[0126] Preferably, the acquisition unit 1101 is further used for:
[0127] Acquire a first training sample, wherein the first training sample includes an original vascular network image;
[0128] Acquire a first training label, wherein the first training label includes a high signal-to-noise ratio vascular network image;
[0129] The device for generating a denoised vascular network image also includes:
[0130] The training unit 1104 is used to train the multimodal convolutional neural network denoising model using the original vascular network image and the high signal-to-noise ratio vascular network image, wherein the generation process of the high signal-to-noise ratio vascular network image includes:
[0131] Obtaining a preliminary denoised vascular network image;
[0132] Inputting the preliminary denoised vascular network image into the second down-sampling module in the vascular network denoising model, wherein the vascular network denoising model includes the second down-sampling module and the second up-sampling module, and the second down-sampling module is used to extract the vascular network features in the preliminary denoised vascular network image to obtain a fourth vascular network feature map output by the second down-sampling module;
[0133] The fourth vascular network feature map is input into the second upsampling module, wherein the second upsampling module is used to restore the size of the fourth vascular network feature to the size of the preliminary denoised vascular network image to obtain a high signal-to-noise ratio vascular network image output by the second upsampling module.
[0134] Preferably, the acquiring unit 1101 is further used for:
[0135] Inputting the original vascular network image into the coarse vascular network segmentation model to obtain a coarse vascular network segmentation image output by the coarse vascular network segmentation model, wherein the coarse vascular network segmentation model includes a third down-sampling module and a third up-sampling module;
[0136] Inputting the original vascular network image into the whole vascular network segmentation model to obtain the whole vascular network segmentation image output by the whole vascular network segmentation model, wherein the whole vascular network segmentation model includes a fourth down-sampling module and a fourth up-sampling module;
[0137] The coarse vascular network segmentation image and the whole vascular network segmentation image are weighted averaged to obtain a preliminary denoised vascular network image.
[0138] Preferably, the training unit 1104 is further used for:
[0139] The vascular network denoising model is trained, wherein the training process of the vascular network denoising model includes:
[0140] Acquire a first training sample, wherein the first training sample includes an original vascular network image;
[0141] Acquire a second training label, wherein the second training label includes a preliminary denoised vascular network image;
[0142] The vascular network denoising model is trained using the original vascular network image and the preliminary denoised vascular network image.
[0143] Preferably, the processing unit 1103 is further configured to:
[0144] The original vascular network image is processed by a preset method to obtain a processed original vascular network image, wherein the preset method includes at least one of rotation, translation and brightness change.
[0145] It should be noted that the functions of each unit in the embodiment of the present application are similar to those described in the above method embodiment and will not be repeated here.
[0146] In an embodiment of the present application, the original vascular network image is input into the first down-sampling module in the multimodal convolutional neural network denoising model through the input-output unit 1102 to extract the context information and vascular network features in the original vascular network image, and the whole vascular network segmentation image is input into the image segmentation feature down-sampling module in the multimodal convolutional neural network denoising model to extract the vascular network structural features in the whole vascular network segmentation image, and then the first vascular network feature map output by the first down-sampling module and the second vascular network feature map output by the image segmentation feature down-sampling module are fused through the processing unit 1103 to obtain a third vascular network feature map with a high signal-to-noise ratio including vascular network structural features, and finally the third vascular network feature map is input into the first up-sampling module to restore the image size of the third vascular network feature map to the size of the original vascular network image to obtain a denoised vascular network image.
[0147] Because the denoised vascular network image output by the multimodal convolutional neural network denoising model is obtained through model training, compared with the existing technology, it improves the convenience of obtaining the denoised vascular network image and reduces the material cost of obtaining the denoised vascular network image.
[0148] Furthermore, the embodiment of the present application also inputs the whole vascular network segmentation image into the image feature downsampling module through the input-output unit 1102 to extract the vascular network structure features in the whole vascular network segmentation image. Therefore, the third vascular network feature map obtained in the embodiment of the present application contains more features about the vascular network structure. Therefore, the denoised vascular network image output by the embodiment of the present application contains more detailed information and a higher signal-to-noise ratio.
[0149] The embodiment of the present application also provides a computer program product having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, it is used to implement the method for generating a denoised vascular network image described in the above method embodiment.
[0150] The above describes the apparatus for generating a denoised vascular network image in the embodiment of the present invention from the perspective of modular functional entities. The following describes the computer apparatus in the embodiment of the present invention from the perspective of hardware processing:
[0151] The computer device is used to realize the function of a device for generating a denoised vascular network image. In an embodiment of the present invention, an embodiment of the computer device includes:
[0152] Processor and memory;
[0153] The memory is used to store computer programs, and when the processor is used to execute the computer programs stored in the memory, the method for generating a denoised vascular network image described in the above method embodiment can be implemented.
[0154] It can be understood that when the processor in the computer device described above executes the computer program, it can also implement the functions of the various units in the corresponding device embodiments described above, which will not be repeated here. Exemplarily, the computer program can be divided into one or more modules / units, and the one or more modules / units are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program in the device for generating a denoised vascular network image. For example, the computer program can be divided into the various units in the above-mentioned device for generating a denoised vascular network image, and each unit can implement the specific functions described in the above-mentioned device for generating a denoised vascular network image.
[0155] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The computer device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that a processor and a memory are merely examples of computer devices and do not constitute a limitation of the computer device. The computer device may include more or fewer components, or a combination of certain components, or different components. For example, the computer device may also include input and output devices, network access devices, buses, and the like.
[0156] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the computer device, and various interfaces and lines are used to connect various parts of the entire computer device.
[0157] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the computer device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0158] The present invention also provides a computer-readable storage medium, which is used to implement the function of a device for generating a denoised vascular network image, and stores a computer program thereon. When the computer program is executed by a processor, it is used to implement the method for generating a denoised vascular network image described in the above method embodiment.
[0159] It is understandable that if the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a corresponding computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned corresponding embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.
[0160] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0161] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0162] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0163] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for generating a denoised vascular network image, characterized in that: include: Obtaining the original vascular network image and the whole vascular network segmentation image; Inputting the original vascular network image into a first downsampling module in a pre-trained multimodal convolutional neural network denoising model, the multimodal convolutional neural network denoising model comprising the first downsampling module, an image segmentation feature downsampling module and a first upsampling module, wherein the first downsampling module is used to extract context information and vascular network features from the original vascular network image to obtain a first vascular network feature map output by the first downsampling module; Inputting the whole vascular network segmentation image into the image segmentation feature downsampling module to extract the vascular network structural features in the whole vascular network segmentation image, and obtaining a second vascular network feature map output by the image segmentation feature downsampling module, wherein the image segmentation feature downsampling module includes a plurality of atrous convolution blocks, or a plurality of standard convolution blocks and a self-attention module; fusing the first vascular network characteristic map and the second vascular network characteristic map to obtain a third vascular network characteristic map; The third vascular network feature map is input into the first upsampling module, wherein the first upsampling module is used to restore the image size of the third vascular network feature map to the size of the original vascular network image to obtain a denoised vascular network image output by the first upsampling module.
2. The method according to claim 1, characterized in that The multimodal convolutional neural network denoising model also includes a self-attention module; After obtaining the second vascular network characteristic map, the method further includes: inputting the second vascular network feature map into the self-attention module, wherein the self-attention module is used to enhance the vascular network structural features in the second vascular network feature map to obtain an enhanced second vascular network feature map; The first vascular network characteristic map and the second vascular network characteristic map are merged to obtain a third vascular network characteristic map, comprising: The first vascular network characteristic map and the enhanced second vascular network characteristic map are fused to obtain the third vascular network characteristic map.
3. The method according to claim 1, characterized in that The multimodal convolutional neural network denoising model includes a multimodal convolutional neural network based on Unet or a multimodal convolutional neural network based on CNN.
4. The method according to claim 1, characterized in that: The fusing the first vascular network characteristic map and the second vascular network characteristic map comprises: The first vascular network characteristic map and the second vascular network characteristic map are fused by connection or superposition.
5. The method according to claim 1, characterized in that Before inputting the original vascular network image into the first down-sampling module, the method further includes: Acquire a first training sample, wherein the first training sample includes the original vascular network image; Acquire a first training label, wherein the first training label includes a high signal-to-noise ratio vascular network image; The multimodal convolutional neural network denoising model is trained using the original vascular network image and the high signal-to-noise ratio vascular network image to obtain the pre-trained multimodal convolutional neural network denoising model, wherein the generation process of the high signal-to-noise ratio vascular network image includes: Obtaining a preliminary denoised vascular network image; Inputting the preliminary denoised vascular network image into a second down-sampling module in a vascular network denoising model, wherein the vascular network denoising model includes the second down-sampling module and a second up-sampling module, and the second down-sampling module is used to extract vascular network features in the preliminary denoised vascular network image to obtain a fourth vascular network feature map output by the second down-sampling module; The fourth vascular network feature map is input into the second upsampling module, wherein the second upsampling module is used to restore the size of the fourth vascular network feature map to the size of the preliminary denoised vascular network image to obtain the high signal-to-noise ratio vascular network image output by the second upsampling module.
6. The method according to claim 5, characterized in that The obtaining of a preliminary denoised vascular network image comprises: Inputting the original vascular network image into a coarse vascular network segmentation model to obtain a coarse vascular network segmentation image output by the coarse vascular network segmentation model, wherein the coarse vascular network segmentation model includes a third down-sampling module and a third up-sampling module; Inputting the original vascular network image into the whole vascular network segmentation model to obtain the whole vascular network segmentation image output by the whole vascular network segmentation model, wherein the whole vascular network segmentation model includes a fourth down-sampling module and a fourth up-sampling module; The coarse vascular network segmentation image and the whole vascular network segmentation image are weighted averaged to obtain the preliminary denoised vascular network image.
7. The method according to claim 5, characterized in that Before inputting the preliminary denoised vascular network image into a second downsampling module in the vascular network denoising model, the method further includes: The vascular network denoising model is trained, wherein the training process of the vascular network denoising model includes: Acquire the first training sample, wherein the first training sample includes the original vascular network image; Acquire a second training label, wherein the second training label includes the preliminary denoised vascular network image; The vascular network denoising model is trained using the original vascular network image and the preliminary denoised vascular network image.
8. The method according to claim 7, characterized in that Before training the vascular network denoising model and the multimodal convolutional neural network denoising model, the method further includes: The original vascular network image is processed by a preset method to obtain a processed original vascular network image, wherein the preset method includes at least one of rotation, translation and brightness change.
9. A device for generating a denoised vascular network image, characterized in that: include: An acquisition unit, used for acquiring an original vascular network image and a whole vascular network segmentation image; An input-output unit, used to input the original vascular network image into a first down-sampling module in a pre-trained multimodal convolutional neural network denoising model, wherein the multimodal convolutional neural network denoising model includes the first down-sampling module, an image segmentation feature down-sampling module and a first up-sampling module, wherein the first down-sampling module is used to extract context information and vascular network features from the original vascular network image to obtain a first vascular network feature map output by the first down-sampling module; The input-output unit is further used to input the whole vascular network segmentation image into the image segmentation feature downsampling module to extract the vascular network structural features in the whole vascular network segmentation image, and obtain a second vascular network feature map output by the image segmentation feature downsampling module, wherein the image segmentation feature downsampling module includes a plurality of atrous convolution blocks, or includes a plurality of standard convolution blocks and a self-attention module; a processing unit, configured to fuse the first vascular network characteristic map and the second vascular network characteristic map to obtain a third vascular network characteristic map; The input-output unit is also used to input the third vascular network feature map into the first upsampling module, wherein the first upsampling module is used to restore the image size of the third vascular network feature map to the size of the original vascular network image to obtain a denoised vascular network image output by the first upsampling module.
10. A computer device comprising a processor, characterized in that: When executing the computer program stored in the memory, the processor is used to implement the method for generating a denoised vascular network image according to any one of claims 1 to 8.
11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it is used to implement the method for generating a denoised vascular network image according to any one of claims 1 to 8.
12. A computer program product having a computer program / instructions stored thereon, characterized in that: The computer program / instructions, when executed by a processor, are used to implement the method for generating a denoised vascular network image according to any one of claims 1 to 8.
