Continuous super-resolution reconstruction method and device for medical image, equipment and medium
By using the combination of feature extraction network, Fourier feature mapping network and implicit function network in medical image processing, the problem of insufficient resolution of medical images is solved, high-resolution image reconstruction is achieved, and diagnostic accuracy is improved .
Patent Information
- Application Number
- CN202411278504.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-12
- Publication Date
- 2025-01-14
AI Technical Summary
The resolution of medical images is physically limited by the imaging equipment, resulting in insufficient resolution and affecting the accuracy of clinical diagnosis.
Using a continuous super-resolution reconstruction method of medical images, the preset super-segment reconstruction model that acquires low-resolution medical images and inputs them including feature extraction networks, Fourier feature mapping networks and implicit function networks is used to utilize Fourier The Leye feature mapping network improves high-frequency information learning capabilities, and improves model robustness through the activation function based on Gabor wavelets, generating high-resolution images.
It effectively improves the resolution of medical images, improves image detail recovery ability and noise robustness, and provides higher accuracy for clinical diagnosis.
Smart Images
Figure CN119313560A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a method, device, equipment and medium for continuous super-resolution reconstruction of medical images. Background Art
[0002] Medical images can reflect the internal structure of the human body and play an important role in clinical diagnosis. The resolution of medical images affects the accuracy of clinical diagnosis, but the resolution of medical images is often limited by the physical limitations of imaging equipment, resulting in insufficient resolution of medical images.
[0003] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention
[0004] The main purpose of the present application is to provide a method, device, equipment and medium for continuous super-resolution reconstruction of medical images, aiming to solve the technical problem that the resolution of medical images is insufficient due to the physical limitations of imaging equipment.
[0005] To achieve the above object, the present application proposes a continuous super-resolution reconstruction method for medical images, the continuous super-resolution reconstruction method for medical images comprising:
[0006] Acquisition of low-resolution medical images;
[0007] The low-resolution medical image is input into a preset super-resolution reconstruction model to obtain a high-resolution medical image, wherein the preset super-resolution reconstruction model includes a feature extraction network, a Fourier feature mapping network and an implicit function network, and the implicit function network includes a fully connected layer, an activation function based on Gabor wavelets and a jump connection layer.
[0008] In one embodiment, before the step of inputting the low-resolution medical image into a preset super-resolution reconstruction model to obtain a high-resolution medical image, the step further includes:
[0009] obtaining a reference medical image of the initial medical image;
[0010] Inputting the initial medical image into an initial super-resolution reconstruction model to obtain a predicted medical image;
[0011] Based on the reference medical image and the predicted medical image, a total loss value is determined, and the initial super-resolution reconstruction model is updated according to the total loss value to obtain a preset super-resolution reconstruction model.
[0012] In one embodiment, the step of determining the total loss value based on the reference medical image and the predicted medical image comprises:
[0013] determining a first loss value between the reference medical image and the predicted medical image;
[0014] Performing Fourier transformation on pixel values of the reference medical image to obtain a first frequency domain representation;
[0015] Performing Fourier transform on the pixel values of the predicted medical image to obtain a second frequency domain representation;
[0016] extracting first high frequency region information represented by the first frequency domain;
[0017] extracting second high frequency region information represented by the second frequency domain;
[0018] determining a second loss value based on the first high-frequency region information and the second high-frequency region information;
[0019] A total loss value is determined based on the first loss value, the second loss value, and a weight coefficient.
[0020] In one embodiment, the step of inputting the low-resolution medical image into a preset super-resolution reconstruction model to obtain a high-resolution medical image comprises:
[0021] Inputting the low-resolution medical image into a preset super-resolution reconstruction model, and extracting latent space features from the low-resolution medical image through the feature extraction network;
[0022] Converting the low-dimensional spatial coordinates of the low-resolution medical image into high-dimensional features through the Fourier feature mapping network;
[0023] A high-resolution medical image is predicted based on the latent space features and the high-dimensional features through the implicit function network.
[0024] In one embodiment, the feature extraction network is a residual dense network, and the residual dense network includes densely connected convolutional layers; wherein the step of extracting latent space features from the low-resolution medical image through the feature extraction network includes:
[0025] Latent space features are extracted from the low-resolution medical image through the residual dense network.
[0026] In one embodiment, the step of converting the low-dimensional spatial coordinates of the low-resolution medical image into high-dimensional features through the Fourier feature mapping network comprises:
[0027] Determining Fourier transform coefficients of the Fourier feature mapping network;
[0028] Based on the Fourier transform coefficients, the low-dimensional spatial coordinates of the low-resolution medical image are mapped into high-dimensional features.
[0029] In one embodiment, the step of predicting a high-resolution medical image according to the latent space features and the high-dimensional features through the implicit function network includes:
[0030] Determine the sinusoidal curve control frequency and the Gaussian curve control width of the Gabor wavelet-based activation function;
[0031] Concatenate the latent space features and the high-dimensional features into a feature vector;
[0032] Based on the sinusoidal curve controlling the frequency, the Gaussian curve controlling the width and the eigenvector, a high-resolution medical image of the low-resolution medical image is predicted through the implicit function network.
[0033] In addition, to achieve the above-mentioned purpose, the present application also proposes a continuous super-resolution reconstruction device for medical images, the continuous super-resolution reconstruction device for medical images comprising:
[0034] An acquisition module, used for acquiring low-resolution medical images;
[0035] An input module is used to input the low-resolution medical image into a preset super-resolution reconstruction model to obtain a high-resolution medical image, wherein the preset super-resolution reconstruction model includes a feature extraction network, a Fourier feature mapping network and an implicit function network, and the implicit function network includes a fully connected layer, an activation function based on Gabor wavelets and a jump connection layer.
[0036] In addition, to achieve the above-mentioned purpose, the present application also proposes a continuous super-resolution reconstruction device for medical images, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the continuous super-resolution reconstruction method for medical images as described above.
[0037] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the continuous super-resolution reconstruction method of medical images as described above are implemented.
[0038] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the continuous super-resolution reconstruction method of medical images as described above.
[0039] One or more technical solutions proposed in this application have at least the following technical effects:
[0040] The continuous super-resolution reconstruction method, device, equipment and medium of medical images proposed in the present application obtain a low-resolution medical image; input the low-resolution medical image into a preset super-resolution reconstruction model to obtain a high-resolution medical image, wherein the preset super-resolution reconstruction model includes a feature extraction network, a Fourier feature mapping network and an implicit function network, and the implicit function network includes a fully connected layer, an activation function based on Gabor wavelets and a jump connection layer, which solves the technical problem that the resolution of medical images is insufficient due to the physical limitations of the imaging device. Compared with the prior art, the present application applies implicit neural representation learning to medical images to achieve continuous super-resolution of medical images, maps low-dimensional coordinates to high-dimensional codes through the Fourier feature mapping network to enhance the learning ability of high-frequency information, thereby improving the ability to restore image details, and replaces the activation function of the implicit function network with an activation function based on Gabor wavelets to improve the robustness of the model to noise, thereby achieving continuous super-resolution of medical images while making the model more robust, providing more convenience for doctors in clinical diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0043] Figure 1 A schematic diagram of a process flow provided in Embodiment 1 of the method for continuous super-resolution reconstruction of medical images of the present application;
[0044] Figure 2 A schematic diagram of the network architecture of a dense residual network provided in Example 1 of the continuous super-resolution reconstruction method for medical images of the present application;
[0045] Figure 3 A schematic diagram of the network architecture of an implicit function network provided in Example 1 of the method for continuous super-resolution reconstruction of medical images of the present application;
[0046] Figure 4 A schematic diagram of a flow chart provided for the second embodiment of the method for continuous super-resolution reconstruction of medical images of the present application;
[0047] Figure 5 A schematic diagram of the module structure of a continuous super-resolution reconstruction device for medical images according to an embodiment of the present application;
[0048] Figure 6 Schematic diagram of the device structure of the hardware operating environment involved in the continuous super-resolution reconstruction method of medical images in the embodiment of the present application.
[0049] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0050] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.
[0051] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.
[0052] The main solution of the embodiment of the present application is: to obtain a low-resolution medical image; to input the low-resolution medical image into a preset super-resolution reconstruction model to obtain a high-resolution medical image, wherein the preset super-resolution reconstruction model includes a feature extraction network, a Fourier feature mapping network and an implicit function network, and the implicit function network includes a fully connected layer, an activation function based on Gabor wavelets and a jump connection layer.
[0053] In this embodiment, for ease of description, the following description is made by taking a continuous super-resolution reconstruction device for identifying medical images as an execution subject.
[0054] Since the resolution of existing medical images is limited by the physical limitations of imaging equipment, medical images have a technical problem of insufficient resolution.
[0055] It can be seen from the above embodiments that the continuous super-resolution reconstruction method, device, equipment and medium of medical images proposed in the present application obtain a low-resolution medical image; input the low-resolution medical image into a preset super-resolution reconstruction model to obtain a high-resolution medical image, wherein the preset super-resolution reconstruction model includes a feature extraction network, a Fourier feature mapping network and an implicit function network, and the implicit function network includes a fully connected layer, an activation function based on Gabor wavelets and a jump connection layer, which solves the technical problem that the resolution of medical images is insufficient due to the physical limitations of the imaging device. Compared with the prior art, the present application applies implicit neural representation learning to medical images to achieve continuous super-resolution of medical images, and maps low-dimensional coordinates to high-dimensional codes through the Fourier feature mapping network to improve the learning ability of high-frequency information, thereby improving the ability to restore image details, and by replacing the activation function of the implicit function network with an activation function based on Gabor wavelets to improve the robustness of the model to noise, thereby achieving continuous super-resolution of medical images while making the model more robust, providing more convenience for doctors in clinical diagnosis.
[0056] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a continuous super-resolution reconstruction device for medical images, etc. The following takes the continuous super-resolution reconstruction device for medical images as an example to illustrate this embodiment and the following embodiments.
[0057] Based on this, the present application embodiment provides a continuous super-resolution reconstruction method for medical images, referring to Figure 1 , Figure 1 Schematic diagram of the flow chart of the first embodiment of the continuous super-resolution reconstruction method for medical images of the present application.
[0058] In this embodiment, the continuous super-resolution reconstruction method of medical images includes steps S10 to S20:
[0059] Step S10, acquiring a low-resolution medical image;
[0060] It should be noted that super-resolution reconstruction is used to convert low-resolution medical images into high-resolution images; low-resolution medical images refer to medical images with insufficient resolution.
[0061] Step S20, inputting the low-resolution medical image into a preset super-resolution reconstruction model to obtain a high-resolution medical image, wherein the preset super-resolution reconstruction model includes a feature extraction network, a Fourier feature mapping network and an implicit function network, and the implicit function network includes a fully connected layer, an activation function based on Gabor wavelet and a jump connection layer.
[0062] It should be noted that the feature extraction network is mainly used to extract latent space features from low-resolution medical images. The feature extraction network can be a residual dense network, which includes densely connected convolutional layers and mainly extracts rich local features through densely connected convolutional layers.
[0063] In a feasible implementation, the step of inputting the low-resolution medical image into a preset super-resolution reconstruction model to obtain a high-resolution medical image includes: inputting the low-resolution medical image into a preset super-resolution reconstruction model, extracting latent space features from the low-resolution medical image through the feature extraction network; converting the low-dimensional space coordinates of the low-resolution medical image into high-dimensional features through the Fourier feature mapping network; and predicting a high-resolution medical image based on the latent space features and the high-dimensional features through the implicit function network.
[0064] It should be noted that the process from low-resolution medical images to high-resolution medical images mainly includes two stages: feature extraction and image reconstruction. In the feature extraction stage, the first is to use the residual dense network to extract latent space features from low-resolution images, and the second is to convert the low-dimensional space coordinates into high-dimensional features using Fourier feature mapping to enhance the learning ability of high-frequency information; in the image reconstruction stage, the latent space features and high-dimensional features extracted in the previous stage are mainly input into the implicit function network to predict the pixel value of the corresponding spatial position (that is, the high-resolution medical image).
[0065] In a feasible implementation, the feature extraction network is a residual dense network, which includes densely connected convolutional layers; wherein the step of extracting latent space features from the low-resolution medical image through the feature extraction network includes: extracting latent space features from the low-resolution medical image through the residual dense network.
[0066] In the specific implementation, the network architecture of the residual dense network is as follows Figure 2 As shown in the figure, rich local features are extracted through densely connected convolutional layers, and the final upsampling operation is removed. The corresponding feature extraction process can be expressed as:
[0067]
[0068] In the formula, Indicates being Parameterized Neural Networks, I lr represents a low-resolution image, v f Represents the latent space features extracted from low-resolution images.
[0069] In a feasible implementation, the step of converting the low-dimensional space coordinates of the low-resolution medical image into high-dimensional features through the Fourier feature mapping network includes: determining the Fourier transform coefficients of the Fourier feature mapping network; and mapping the low-dimensional space coordinates of the low-resolution medical image into high-dimensional features based on the Fourier transform coefficients.
[0070] It should be noted that in order to effectively learn high-frequency functions, we introduced the Fourier feature mapping operation, which improves the network's ability to restore image details by encoding spatial coordinates using Fourier features. The mapping process of high-dimensional features can be expressed as:
[0071] ζ(c)=[cos(2πBc),sin(2πBc)] T
[0072] In the formula, the matrix B represents the Fourier transform coefficients, which are generally sampled from a standard normal distribution, and c represents the corresponding low-dimensional space coordinates.
[0073] In a feasible implementation, the step of predicting a high-resolution medical image based on the latent space features and the high-dimensional features through the implicit function network includes: determining the sinusoidal curve control frequency and the Gaussian curve control width of the Gabor wavelet-based activation function; splicing the latent space features and the high-dimensional features into a feature vector; and predicting a high-resolution medical image of the low-resolution medical image through the implicit function network based on the sinusoidal curve control frequency, the Gaussian curve control width and the feature vector.
[0074] It should be noted that the implicit function network is mainly composed of a fully connected layer, an activation function based on Gabor wavelet, and a skip connection layer, such as Figure 3 As shown in the figure, the high-dimensional features and latent space features obtained in the feature extraction stage are concatenated together to obtain a feature vector, and then the feature vector is input into the implicit function network to predict the pixel value of the corresponding spatial position.
[0075] It should be noted that four cascaded fully connected layers and the Gabor wavelet-based activation function constitute a basic block. The first and second basic blocks are followed by a skip connection operation. The Gabor wavelet-based activation function is mainly used to improve details and enhance the model's robustness to noise, which can be expressed as follows:
[0076] ψ(x;ω,d)=sin(ωx)e -(dx)2
[0077] Wherein, ω represents the frequency of the controlled sine curve (ie, the sine curve control frequency), and d represents the width of the controlled Gaussian curve (ie, the Gaussian curve control width).
[0078] The continuous super-resolution reconstruction method, device, equipment and medium of medical images proposed in the present application obtain a low-resolution medical image; input the low-resolution medical image into a preset super-resolution reconstruction model to obtain a high-resolution medical image, wherein the preset super-resolution reconstruction model includes a feature extraction network, a Fourier feature mapping network and an implicit function network, and the implicit function network includes a fully connected layer, an activation function based on Gabor wavelets and a jump connection layer, which solves the technical problem that the resolution of medical images is insufficient due to the physical limitations of the imaging device. Compared with the prior art, the present application applies implicit neural representation learning to medical images to achieve continuous super-resolution of medical images, maps low-dimensional coordinates to high-dimensional codes through the Fourier feature mapping network to enhance the learning ability of high-frequency information, thereby improving the ability to restore image details, and replaces the activation function of the implicit function network with an activation function based on Gabor wavelets to improve the robustness of the model to noise, thereby achieving continuous super-resolution of medical images while making the model more robust, providing more convenience for doctors in clinical diagnosis.
[0079] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 4 Before step S20, the method for continuous super-resolution reconstruction of medical images further includes steps S01 to S03:
[0080] Step S01, obtaining a reference medical image of an initial medical image;
[0081] It should be noted that the initial medical image is a low-resolution medical image used to train the initial super-resolution reconstruction model, and the reference medical image refers to a high-resolution image obtained after super-resolution reconstruction of the initial medical image.
[0082] Step S02, inputting the initial medical image into an initial super-resolution reconstruction model to obtain a predicted medical image;
[0083] It should be noted that the initial super-resolution reconstruction model refers to an untrained super-resolution reconstruction model.
[0084] Step S03, determining a total loss value based on the reference medical image and the predicted medical image, and obtaining a preset super-resolution reconstruction model after updating the initial super-resolution reconstruction model according to the total loss value.
[0085] It should be noted that the preset super-resolution reconstruction model refers to a trained super-resolution reconstruction model, which can be used to perform super-resolution reconstruction on low-resolution images.
[0086] In a feasible implementation, the step of determining the total loss value based on the reference medical image and the predicted medical image includes: determining a first loss value of the reference medical image and the predicted medical image; performing Fourier transform on the pixel values of the reference medical image to obtain a first frequency domain representation; performing Fourier transform on the pixel values of the predicted medical image to obtain a second frequency domain representation; extracting first high-frequency area information represented by the first frequency domain; extracting second high-frequency area information represented by the second frequency domain; determining a second loss value based on the first high-frequency area information and the second high-frequency area information; and determining a total loss value based on the first loss value, the second loss value, and a weight coefficient.
[0087] It should be noted that in order to improve the model's ability to restore detail information, a loss function constraint based on the high-frequency area is designed. The calculation process is as follows:
[0088]
[0089] In the formula, f Ω Indicates the extraction of high-frequency area information, fft means Fourier transform, Represents the pixel value predicted by the network, I hr Represents the reference medical image, the total loss value is: l = l s +λl k , l s represents the l1 loss (i.e., the first loss value) between the predicted medical image and the reference medical image, and λ represents the weight coefficient, which is used to control l k The proportion of k Represents the second loss value.
[0090] It should be noted that the first loss value represents the difference in pixel values between the predicted medical image and the reference medical image; k By strengthening the constraints on high-frequency areas, the texture information in the image domain is improved and the image details are improved.
[0091] In this embodiment, a loss function constraint based on high-frequency regions is introduced, and the medical image is converted into the Fourier domain using Fourier transform. By strengthening the learning of the high-frequency regions of the image, the detail information of the image is improved.
[0092] This embodiment obtains a reference medical image of the initial medical image; inputs the initial medical image into the initial super-resolution reconstruction model to obtain a predicted medical image; determines a total loss value based on the reference medical image and the predicted medical image, and updates the initial super-resolution reconstruction model according to the total loss value to obtain a preset super-resolution reconstruction model. This can effectively improve the training effect of the super-resolution reconstruction model.
[0093] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the continuous super-resolution reconstruction method of medical images of the present application. More forms of simple transformations based on this technical concept are all within the scope of protection of the present application.
[0094] This application also provides a continuous super-resolution reconstruction device for medical images, please refer to Figure 5 , the continuous super-resolution reconstruction device of the medical image comprises:
[0095] An acquisition module 10, used for acquiring low-resolution medical images;
[0096] The input module 20 is used to input the low-resolution medical image into a preset super-resolution reconstruction model to obtain a high-resolution medical image, wherein the preset super-resolution reconstruction model includes a feature extraction network, a Fourier feature mapping network and an implicit function network, and the implicit function network includes a fully connected layer, an activation function based on Gabor wavelet and a jump connection layer.
[0097] The continuous super-resolution reconstruction device for medical images provided by the present application adopts the continuous super-resolution reconstruction method for medical images in the above-mentioned embodiment, and can solve the technical problem that the resolution of medical images is insufficient due to the physical limitation of the imaging device. Compared with the prior art, the beneficial effects of the continuous super-resolution reconstruction device for medical images provided by the present application are the same as the beneficial effects of the continuous super-resolution reconstruction method for medical images provided by the above-mentioned embodiment, and other technical features in the continuous super-resolution reconstruction device for medical images are the same as the features disclosed in the above-mentioned embodiment method, which will not be described in detail here.
[0098] The present application provides a continuous super-resolution reconstruction device for medical images, and the continuous super-resolution reconstruction device for medical images includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the continuous super-resolution reconstruction method for medical images in the above-mentioned embodiment one.
[0099] Reference below Figure 6, which shows a schematic diagram of the structure of a continuous super-resolution reconstruction device for medical images suitable for implementing the embodiment of the present application. The continuous super-resolution reconstruction device for medical images in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The continuous super-resolution reconstruction device for medical images shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0100] like Figure 6 As shown, the continuous super-resolution reconstruction device for medical images may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. Various programs and data required for the operation of the continuous super-resolution reconstruction device for medical images are also stored in the RAM 1004. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems may be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the continuous super-resolution reconstruction device for medical images to communicate wirelessly or wired with other devices to exchange data. Although the continuous super-resolution reconstruction device for medical images with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have instead.
[0101] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.
[0102] The continuous super-resolution reconstruction device for medical images provided by the present application adopts the continuous super-resolution reconstruction method for medical images in the above-mentioned embodiment, which can solve the technical problem that the resolution of medical images is insufficient due to the physical limitation of the imaging device. Compared with the prior art, the beneficial effects of the continuous super-resolution reconstruction device for medical images provided by the present application are the same as the beneficial effects of the continuous super-resolution reconstruction method for medical images provided by the above-mentioned embodiment, and other technical features in the continuous super-resolution reconstruction device for medical images are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.
[0103] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.
[0104] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0105] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the continuous super-resolution reconstruction method of medical images in the above-mentioned embodiment.
[0106] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.
[0107] The computer-readable storage medium may be included in the continuous super-resolution reconstruction device for medical images; or may exist independently without being assembled into the continuous super-resolution reconstruction device for medical images.
[0108] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by a continuous super-resolution reconstruction device for medical images, the continuous super-resolution reconstruction device for medical images: obtains a low-resolution medical image; inputs the low-resolution medical image into a preset super-resolution reconstruction model to obtain a high-resolution medical image, wherein the preset super-resolution reconstruction model includes a feature extraction network, a Fourier feature mapping network, and an implicit function network, and the implicit function network includes a fully connected layer, an activation function based on Gabor wavelets, and a jump connection layer.
[0109] Computer program code for performing the operations of the present application may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0110] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0111] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.
[0112] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned continuous super-resolution reconstruction method for medical images, and can solve the technical problem that the resolution of medical images is insufficient due to the physical limitations of the imaging device. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the continuous super-resolution reconstruction method for medical images provided by the above-mentioned embodiments, and will not be elaborated here.
[0113] The present application also provides a computer program product, comprising a computer program, which implements the steps of the continuous super-resolution reconstruction method of medical images as described above when the computer program is executed by a processor.
[0114] The computer program product provided by the present application can solve the technical problem that the resolution of medical images is insufficient due to the physical limitation of the imaging device. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the continuous super-resolution reconstruction method of medical images provided by the above-mentioned embodiment, which will not be described in detail here.
[0115] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.
Claims
1. A method for continuous super-resolution reconstruction of medical images, characterized in that: The method comprises: Acquisition of low-resolution medical images; The low-resolution medical image is input into a preset super-resolution reconstruction model to obtain a high-resolution medical image, wherein the preset super-resolution reconstruction model includes a feature extraction network, a Fourier feature mapping network and an implicit function network, and the implicit function network includes a fully connected layer, an activation function based on Gabor wavelets and a jump connection layer.
2. The method according to claim 1, characterized in that Before the step of inputting the low-resolution medical image into a preset super-resolution reconstruction model to obtain a high-resolution medical image, the step further includes: obtaining a reference medical image of the initial medical image; Inputting the initial medical image into an initial super-resolution reconstruction model to obtain a predicted medical image; Based on the reference medical image and the predicted medical image, a total loss value is determined, and the initial super-resolution reconstruction model is updated according to the total loss value to obtain a preset super-resolution reconstruction model.
3. The method according to claim 2, characterized in that The step of determining the total loss value based on the reference medical image and the predicted medical image comprises: determining a first loss value between the reference medical image and the predicted medical image; Performing Fourier transformation on pixel values of the reference medical image to obtain a first frequency domain representation; Performing Fourier transform on the pixel values of the predicted medical image to obtain a second frequency domain representation; extracting first high frequency region information represented by the first frequency domain; extracting second high frequency region information represented by the second frequency domain; determining a second loss value based on the first high-frequency region information and the second high-frequency region information; A total loss value is determined based on the first loss value, the second loss value, and a weight coefficient.
4. The method according to claim 1, characterized in that The step of inputting the low-resolution medical image into a preset super-resolution reconstruction model to obtain a high-resolution medical image comprises: Inputting the low-resolution medical image into a preset super-resolution reconstruction model, and extracting latent space features from the low-resolution medical image through the feature extraction network; Converting the low-dimensional spatial coordinates of the low-resolution medical image into high-dimensional features through the Fourier feature mapping network; A high-resolution medical image is predicted based on the latent space features and the high-dimensional features through the implicit function network.
5. The method according to claim 4, characterized in that The feature extraction network is a residual dense network, and the residual dense network includes densely connected convolutional layers; wherein the step of extracting latent space features from the low-resolution medical image through the feature extraction network includes: Latent space features are extracted from the low-resolution medical image through the residual dense network.
6. The method according to claim 4, characterized in that The step of converting the low-dimensional spatial coordinates of the low-resolution medical image into high-dimensional features through the Fourier feature mapping network comprises: Determining Fourier transform coefficients of the Fourier feature mapping network; Based on the Fourier transform coefficients, the low-dimensional spatial coordinates of the low-resolution medical image are mapped into high-dimensional features.
7. The method according to claim 4, characterized in that The step of predicting a high-resolution medical image according to the latent space features and the high-dimensional features through the implicit function network includes: Determine the sinusoidal curve control frequency and the Gaussian curve control width of the Gabor wavelet-based activation function; Concatenate the latent space features and the high-dimensional features into a feature vector; Based on the sinusoidal curve controlling the frequency, the Gaussian curve controlling the width and the eigenvector, a high-resolution medical image of the low-resolution medical image is predicted through the implicit function network.
8. A continuous super-resolution reconstruction device for medical images, characterized in that: The device comprises: An acquisition module, used for acquiring low-resolution medical images; An input module is used to input the low-resolution medical image into a preset super-resolution reconstruction model to obtain a high-resolution medical image, wherein the preset super-resolution reconstruction model includes a feature extraction network, a Fourier feature mapping network and an implicit function network, and the implicit function network includes a fully connected layer, an activation function based on Gabor wavelets and a jump connection layer.
9. A continuous super-resolution reconstruction device for medical images, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the method for continuous super-resolution reconstruction of medical images according to any one of claims 1 to 7.
10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the continuous super-resolution reconstruction method for medical images according to any one of claims 1 to 7 are implemented.