Image processing method and device, storage medium and electronic equipment

Abstract

The present disclosure relates to an image processing method, device, storage medium and electronic equipment, which comprises the following steps: obtaining an initial medical image, wherein the initial medical image comprises a blood vessel image; determining a blood vessel image region in the initial medical image; generating a to-be-processed blood vessel image with a mask according to the blood vessel image region; inputting the to-be-processed blood vessel image into a preset blood vessel processing model to obtain a target blood vessel image output by the preset blood vessel processing model. In this way, the blood vessel image region in the initial medical image can be strengthened by the preset blood vessel processing model, so as to obtain a target blood vessel image with higher resolution, which is beneficial to improving the clarity of the blood vessel in the image, thereby providing more accurate diagnostic basis for clinical diagnosis.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and more specifically, to an image processing method, apparatus, storage medium, and electronic device. Background Technology

[0002] Medical images (such as X-ray images, CT (Computed Tomography) images, PET (Positron Emission Computed Tomography) images, SPECT (Positron Emission Tomography PET) images, and MRI (Magnetic Resonance Imaging) images) reflect the internal structure of the human body and are one of the main bases for modern medical diagnosis. Utilizing image processing technology to process and analyze medical images enables the location detection, segmentation, extraction, three-dimensional reconstruction, and three-dimensional display of human organs, soft tissues, and lesions, which can greatly improve the efficiency, accuracy, and reliability of clinical diagnosis. However, current image processing methods still cannot effectively guarantee the clarity of vascular images, failing to provide doctors with more reliable data for diagnosis and thus failing to play a crucial role in diagnostic evidence. Summary of the Invention

[0003] The purpose of this disclosure is to provide an image processing method, apparatus, storage medium, and electronic device.

[0004] To achieve the above objectives, the first aspect of this disclosure provides an image processing method, the method comprising:

[0005] Acquire initial medical images, including vascular images;

[0006] Identify the vascular image region in the initial medical image;

[0007] Generate a masked vascular image to be processed based on the vascular image region;

[0008] The blood vessel image to be processed is input into a preset blood vessel processing model to obtain a target blood vessel image output by the preset blood vessel processing model. The preset blood vessel processing model is used to perform resolution enhancement processing on the blood vessel image region in the blood vessel image to be processed, and the resolution of the target blood vessel image is higher than that of the blood vessel image to be processed.

[0009] Optionally, the preset blood vessel processing model includes a convolutional residual network and a tail interpolation network, wherein the output of the convolutional residual network is coupled to the input of the tail interpolation network, and the output of the tail interpolation network serves as the output of the preset blood vessel processing model.

[0010] The step of inputting the blood vessel image to be processed into a preset blood vessel processing model to obtain the target blood vessel image output by the preset blood vessel processing model includes:

[0011] The convolutional residual network is used to extract deep features from the blood vessel image to be processed, so as to obtain a first number of unused feature maps.

[0012] The tail interpolation network is used to convolve the first number of available feature maps to obtain a second number of specified feature maps, and image interpolation is performed on the second number of specified feature maps to obtain the target blood vessel image.

[0013] Optionally, the resolution of the blood vessel image to be processed is M×N, the number of first channels is X, and the resolution of the target blood vessel image is aM×aN, where M and N are positive integers, and a is a positive number;

[0014] The step of convolving the first number of candidate feature maps using the tail interpolation network to obtain a second number of specified feature maps, and then performing image interpolation on the second number of specified feature maps to obtain the target blood vessel image, includes:

[0015] The tail interpolation network convolves the first number of unused feature maps into a second number of specified feature maps, where the second number of channels is a. 2 X; and the a 2 X specified feature maps are interpolated into a target feature map of aM×aN using a preset interpolation method.

[0016] Optionally, the convolutional residual network includes a head convolutional network module and a backbone network module. The output of the head convolutional network module is coupled to the input of the backbone network module, and the output of the backbone network module is coupled to the input of the tail interpolation network. The backbone network module includes multiple convolutional sub-modules, each of which includes a first convolutional layer, an activation function layer, and a second convolutional layer. The input of the first convolutional layer is residually connected to the output of the second convolutional layer. The step of inputting the blood vessel image to be processed into a preset blood vessel processing model to obtain the target blood vessel image output by the preset blood vessel processing model includes:

[0017] The shallow features of the blood vessel image to be processed are extracted by the head convolutional network module.

[0018] The backbone network module extracts deep features from the shallow features output by the head convolutional network module to obtain the first number of available feature maps.

[0019] Optionally, generating a masked vascular image based on the vascular image region includes:

[0020] Generate a specified mask corresponding to the blood vessel image region;

[0021] The specified mask is added to the initial medical image to obtain the mask image to be used;

[0022] The mask image to be used is normalized to obtain the blood vessel image to be processed.

[0023] Optionally, the preset blood vessel processing model is trained through the following steps:

[0024] Acquire multiple blood vessel sample images and corresponding label images for the blood vessel sample images, wherein the resolution of the blood vessel sample images is lower than the resolution of the label images;

[0025] Using the multiple blood vessel sample images and the corresponding label images as training data, the preset initial model is iteratively trained to obtain the preset blood vessel processing model.

[0026] Optionally, acquiring multiple vascular sample images and corresponding label images of the vascular sample images includes:

[0027] Acquire a first blood vessel image of a first resolution within each of a plurality of medical sample images;

[0028] The first blood vessel image is downsampled to obtain a second blood vessel image with a second resolution;

[0029] The second blood vessel image corresponding to each of the plurality of medical sample images is used as the blood vessel sample image, and the first blood vessel image corresponding to the blood vessel sample image is used as the label image of the blood vessel sample image.

[0030] A second aspect of this disclosure provides an image processing apparatus, the apparatus comprising:

[0031] The first acquisition module is configured to acquire an initial medical image, the initial medical image including a vascular image;

[0032] The determination module is configured to determine the vascular image region in the initial medical image;

[0033] The generation module is configured to generate a masked vascular image to be processed based on the vascular image region;

[0034] The second acquisition module is configured to input the blood vessel image to be processed into a preset blood vessel processing model to obtain a target blood vessel image output by the preset blood vessel processing model, wherein the preset blood vessel processing model is used to perform resolution enhancement processing on the blood vessel image region in the blood vessel image to be processed, and the resolution of the target blood vessel image is higher than the resolution of the blood vessel image to be processed.

[0035] Optionally, the preset blood vessel processing model includes a convolutional residual network and a tail interpolation network, wherein the output of the convolutional residual network is coupled to the input of the tail interpolation network, and the output of the tail interpolation network serves as the output of the preset blood vessel processing model.

[0036] The second acquisition module is configured as follows:

[0037] The convolutional residual network is used to extract deep features from the blood vessel image to be processed, so as to obtain a first number of unused feature maps.

[0038] The tail interpolation network is used to convolve the first number of available feature maps to obtain a second number of specified feature maps, and image interpolation is performed on the second number of specified feature maps to obtain the target blood vessel image.

[0039] Optionally, the resolution of the blood vessel image to be processed is M×N, the number of first channels is X, and the resolution of the target blood vessel image is aM×aN, where M and N are positive integers, and a is a positive number;

[0040] The second acquisition module is configured as follows:

[0041] The tail interpolation network convolves the first number of unused feature maps into a second number of specified feature maps, where the second number of channels is a. 2 X; and the a 2 X specified feature maps are interpolated into a target feature map of aM×aN using a preset interpolation method.

[0042] Optionally, the convolutional residual network includes a head convolutional network module and a backbone network module. The output of the head convolutional network module is coupled to the input of the backbone network module, and the output of the backbone network module is coupled to the input of the tail interpolation network. The backbone network module includes multiple convolutional sub-modules, each of which includes a first convolutional layer, an activation function layer, and a second convolutional layer. The input of the first convolutional layer is residually connected to the output of the second convolutional layer. The second acquisition module is configured to:

[0043] The shallow features of the blood vessel image to be processed are extracted by the head convolutional network module.

[0044] The backbone network module extracts deep features from the shallow features output by the head convolutional network module to obtain the first number of available feature maps.

[0045] Optionally, the generation module is configured as follows:

[0046] Generate a specified mask corresponding to the blood vessel image region;

[0047] The specified mask is added to the initial medical image to obtain the mask image to be used;

[0048] The mask image to be used is normalized to obtain the blood vessel image to be processed.

[0049] Optionally, the device further includes a model training module, which is configured to:

[0050] Acquire multiple blood vessel sample images and corresponding label images for the blood vessel sample images, wherein the resolution of the blood vessel sample images is lower than the resolution of the label images;

[0051] Using the multiple blood vessel sample images and the corresponding label images as training data, the preset initial model is iteratively trained to obtain the preset blood vessel processing model.

[0052] Optionally, the model training module is configured as follows:

[0053] Acquire a first blood vessel image of a first resolution within each of a plurality of medical sample images;

[0054] The first blood vessel image is downsampled to obtain a second blood vessel image with a second resolution;

[0055] The second blood vessel image corresponding to each of the plurality of medical sample images is used as the blood vessel sample image, and the first blood vessel image corresponding to the blood vessel sample image is used as the label image of the blood vessel sample image.

[0056] A third aspect of this disclosure provides a non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the steps of the method described in the first aspect above.

[0057] A fourth aspect of this disclosure provides an electronic device comprising:

[0058] A memory on which computer programs are stored;

[0059] A processor for executing the computer program in the memory to implement the steps of the method described in the first aspect above.

[0060] The above technical solution involves acquiring an initial medical image, including a vascular image; determining the vascular image region within the initial medical image; generating a masked vascular image to be processed based on the vascular image region; and inputting the vascular image to be processed into a preset vascular processing model to obtain a target vascular image output by the preset vascular processing model. The preset vascular processing model is used to perform resolution enhancement processing on the vascular image region in the vascular image to be processed, and the resolution of the target vascular image is higher than that of the vascular image to be processed. In this way, the preset vascular processing model can enhance the vascular image region in the initial medical image to obtain a higher-resolution target vascular image, which helps improve the clarity of blood vessels in the image and provides a more accurate and reliable basis for the diagnosis of vascular-related diseases.

[0061] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0062] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings:

[0063] Figure 1 This is a flowchart illustrating an exemplary embodiment of the present disclosure of an image processing method;

[0064] Figure 2 This is a schematic diagram of a model structure shown in an exemplary embodiment of the present disclosure;

[0065] Figure 3 It is based on Figure 2The illustrated embodiment presents a schematic diagram of a model structure;

[0066] Figure 4 This is a schematic diagram illustrating the working principle of a tail interpolation network according to an exemplary embodiment of this disclosure;

[0067] Figure 5 It is based on Figure 1 The illustrated embodiment shows a flowchart of an image processing method;

[0068] Figure 6 This is a flowchart illustrating a model training method according to an exemplary embodiment of the present disclosure;

[0069] Figure 7 It is based on Figure 6 The illustrated embodiment shows a flowchart of a model training method;

[0070] Figure 8 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment of the present disclosure;

[0071] Figure 9 It is based on Figure 8 The illustrated embodiment shows a block diagram of an image processing apparatus;

[0072] Figure 10 This is a block diagram illustrating an electronic device according to an exemplary embodiment;

[0073] Figure 11 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0074] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0075] It should be noted that all actions taken to obtain information or data in this disclosure are carried out in accordance with the relevant data protection laws and policies of the country where the information is located, and with the authorization of the owner of the relevant device.

[0076] Before detailing the technical solution of this disclosure, the application scenarios of this disclosure are explained as follows: This disclosure can be used for the processing of vascular images in medical images (such as CT images, magnetic resonance images, or PET images). Since many blood vessels are relatively narrow and small, the effective pixels on medical images are relatively few, so the clarity is mostly poor and cannot provide reliable image basis for the diagnosis of vascular-related diseases.

[0077] To address the aforementioned technical problems, this disclosure provides an image processing method, apparatus, storage medium, and electronic device. The image processing method involves acquiring an initial medical image, including a vascular image; determining a vascular image region within the initial medical image; generating a masked vascular image to be processed based on the vascular image region; and inputting the vascular image to be processed into a preset vascular processing model to obtain a target vascular image output by the preset vascular processing model. This allows for enhancement processing of the vascular image region in the initial medical image using the preset vascular processing model, resulting in a higher-resolution target vascular image, which improves the clarity of blood vessels in the image and provides more accurate diagnostic evidence for clinical diagnosis.

[0078] The implementation methods of this disclosure will be described in detail below with reference to specific embodiments.

[0079] Figure 1 This is a flowchart illustrating an exemplary embodiment of the present disclosure of an image processing method; as shown below. Figure 1 As shown, the image processing method includes:

[0080] Step 101: Obtain an initial medical image, which includes a vascular image.

[0081] The initial medical image can be a CT image, an MRI image, or a PET image, etc.

[0082] Step 102: Identify the vascular image region in the initial medical image.

[0083] In this step, a pre-trained blood vessel segmentation model can be used to identify blood vessel regions in the initial medical image. This model can be a deep learning-based neural network model or another machine learning model. It should be noted that the training data for this blood vessel segmentation model can include training image samples and blood vessel location information for each training image sample, where the blood vessel location information is the label data corresponding to the training image sample. Alternatively, this step can directly obtain a manually specified blood vessel region, for example, by manually inputting the pixel coordinates of the blood vessel region in a preset interface, or by manually selecting a region in the image and using the selected region as the blood vessel region.

[0084] Step 103: Generate a masked vascular image to be processed based on the vascular image region.

[0085] In this step, a specified mask corresponding to the blood vessel image region can be generated; the specified mask is added to the initial medical image to obtain a mask image to be used; in one possible implementation, the mask image to be used can be used as the image to be processed; in another possible implementation, the mask image to be used can be normalized to obtain the blood vessel image to be processed.

[0086] For example, when generating the specified mask, the pixel matrix corresponding to the initial medical image is obtained. The values ​​at the coordinate positions corresponding to the blood vessel image region in the pixel matrix are set to 1, and the values ​​at the coordinate positions other than the blood vessel image region in the pixel matrix are set to 0, thus obtaining the specified mask. The product of the specified mask and the elements at the same positions in the pixel matrix corresponding to the initial medical image is obtained to obtain the mask image to be used. Furthermore, when normalizing the mask image to be used, the minimum value HU in the blood vessel image region can be obtained. min And the maximum value HU max Subtract HU from the value at each coordinate location within the blood vessel image region. min To obtain the first difference corresponding to each coordinate position within each blood vessel image region, and to obtain the first difference and ( max -U min The quotient of ) is used to obtain the normalized blood vessel image to be processed.

[0087] Step 104: Input the blood vessel image to be processed into a preset blood vessel processing model to obtain the target blood vessel image output by the preset blood vessel processing model. The preset blood vessel processing model is used to perform resolution enhancement processing on the blood vessel image region in the blood vessel image to be processed. The resolution of the target blood vessel image is higher than that of the blood vessel image to be processed.

[0088] The preset blood vessel processing model may include a convolutional residual network and a tail interpolation network. The output of the convolutional residual network is coupled to the input of the tail interpolation network, and the output of the tail interpolation network serves as the output of the preset blood vessel processing model. The convolutional residual network is used to extract deep features from the blood vessel image to be processed to obtain a first number of available feature maps. The tail interpolation network is used to perform convolution processing on the first number of available feature maps to obtain a second number of specified feature maps, and to perform image interpolation processing on the second number of specified feature maps to obtain the target blood vessel image.

[0089] For example, Figure 2 This is a schematic diagram of a model structure shown in an exemplary embodiment of this disclosure; as follows: Figure 2As shown, the preset blood vessel processing model may include a convolutional residual network and a tail interpolation network. The convolutional residual network includes a head convolutional network module and a backbone network module. The output of the head convolutional network module is coupled to the input of the backbone network module, and the output of the backbone network module is coupled to the input of the tail interpolation network. The head convolutional network module is used to extract shallow features of the blood vessel image to be processed. The backbone network module includes multiple convolutional sub-modules, and the structure of each convolutional sub-module can be as follows: Figure 3 As shown ( Figure 3 It is based on Figure 2 The illustrated embodiment presents a schematic diagram of a model structure. The convolutional submodule may include a first convolutional layer, an activation function layer, and a second convolutional layer. The input of the first convolutional layer is residually connected to the output of the second convolutional layer, used to extract deep features from the shallow features output by the head convolutional network module, thereby obtaining the first number of channel-level feature maps. The residual connection between the input of the first convolutional layer and the output of the second convolutional layer effectively alleviates the gradient vanishing problem caused by increasing depth in deep neural networks, thus significantly improving model optimization efficiency.

[0090] The above-described implementation of convolving the first number of unused feature maps to obtain a second number of specified feature maps, and then performing image interpolation on the second number of specified feature maps to obtain the target blood vessel image, may include:

[0091] If the resolution of the blood vessel image to be processed is M×N, the number of the first channel is X, and the resolution of the target blood vessel image is aM×aN, where M and N are positive integers, and a is a positive number; the tail interpolation network can convolve the first number of feature maps to be used into a specified number of feature maps with a second number of channels. 2 X; and will the a 2 X specified feature maps are interpolated into a target feature map of aM×aN using a preset interpolation method. This preset interpolation method may include a specified order of channel values.

[0092] For example, Figure 4 This is a schematic diagram illustrating the working principle of a tail interpolation network according to an exemplary embodiment of this disclosure, such as... Figure 4 As shown, the resolution of the blood vessel image to be processed is 3×3. After passing through the convolutional residual network, a feature map with one channel (i.e., the first number of channels) is obtained (resolution 3×3). After image interpolation processing by the tail interpolation network, 2 2 The specified feature is 1 × 1 channel (i.e., the number of second channels), which will 2 2Upsampling is performed on specified features from 1 × 1 channel, with channel 1 positioned at the top right of each pixel, channel 2 at the top left of each pixel, channel 3 at the bottom left of each pixel, and channel 4 at the bottom right of each pixel, resulting in a (2×3)×(2×3) target feature map. Interpolation can also be performed using other channel value arrangements, each with a different preset interpolation method. For example, channel 1 could be positioned at the top left of each pixel, channel 2 at the top right of each pixel, channel 3 at the bottom left of each pixel, and channel 4 at the bottom right of each pixel.

[0093] The above technical solution can enhance the vascular image region in the initial medical image through the preset vascular processing model to obtain a target vascular image with higher resolution, which is beneficial to improve the clarity of the vascular in the image, thereby providing a more accurate diagnostic basis for clinical diagnosis.

[0094] Figure 5 It is based on Figure 1 The illustrated embodiment presents a flowchart of an image processing method; based on the above... Figure 2 and Figure 3 The model structure shown is as follows: Figure 5 As shown, step 104 above, which involves inputting the blood vessel image to be processed into a preset blood vessel processing model to obtain the target blood vessel image output by the preset blood vessel processing model, may include:

[0095] Step 1041: Extract the shallow features of the blood vessel image to be processed through the head convolutional network module.

[0096] The head convolutional network module can be a shallow feature extractor formed by a convolutional network. The output of the head convolutional network module is coupled to the input of the backbone network module.

[0097] Step 1042: The backbone network module performs deep feature extraction on the shallow features output by the head convolutional network module to obtain the feature map to be used for the first number of channels.

[0098] The output of the backbone network module is coupled to the input of the tail interpolation network. The backbone network module includes multiple convolutional sub-modules. Each convolutional sub-module includes a first convolutional layer, an activation function layer, and a second convolutional layer. The input of the first convolutional layer is residually connected to the output of the second convolutional layer.

[0099] Step 1043: The tail interpolation network is used to convolve the first number of available feature maps to obtain a second number of specified feature maps, and image interpolation is performed on the second number of specified feature maps to obtain the target blood vessel image.

[0100] In this step, the tail interpolation network can be used to convolve the feature map of the first number of channels (X) into the feature map of the second number of channels (a). 2 The specified feature map of X); and the a 2 X specified feature maps are interpolated into a target feature map of aM×aN according to a preset interpolation method. The resolution of the blood vessel image to be processed is M×N, the number of first channels is X, and the resolution of the target blood vessel image is aM×aN, where M and N are positive integers, and a is a positive number.

[0101] The above technical solution can enhance the vascular image region in the initial medical image through the preset vascular processing model to obtain a target vascular image with higher resolution, which is beneficial to improve the clarity of the vascular in the image, thereby providing a more accurate diagnostic basis for clinical diagnosis.

[0102] Figure 6 This is a flowchart illustrating a model training method according to an exemplary embodiment of this disclosure; as shown below. Figure 6 As shown, the preset blood vessel processing model can be trained through the following steps S1 and S2:

[0103] S1, acquire multiple blood vessel sample images and corresponding label images for each blood vessel sample image, wherein the resolution of the blood vessel sample image is lower than that of the label image.

[0104] This step can be done Figure 7 The steps shown in S11 to S13 are implemented. Figure 7 It is based on Figure 6 The illustrated embodiment shows a flowchart of a model training method, as follows: Figure 7 As shown, step S1 above may include:

[0105] S11, acquire a first blood vessel image with a first resolution in each of the multiple medical sample images.

[0106] The medical sample image can be a CT image, an MRI image, or a PET image, etc.

[0107] S12, the first blood vessel image is downsampled to obtain a second blood vessel image with a second resolution.

[0108] S13, take the second blood vessel image corresponding to each of the plurality of medical sample images as the blood vessel sample image, and take the first blood vessel image corresponding to the blood vessel sample image as the label image of the blood vessel sample image.

[0109] For example, the vascular sample image can be obtained from a vascular image of a specified blood vessel (such as the aorta or pulmonary artery). Taking CT images as an example, the specified vascular image can be a vascular image of the aorta or pulmonary artery from CT image data with a small spacing (pixel pitch) parameter in the X and Y axes. For example, a fixed-size image, such as a 40*40 resolution image, can be cropped centered on the location of the specified blood vessel and used as the training label image. The label image is then downsampled to obtain a reduced-resolution image; for example, halving the resolution could result in a 20*20 resolution image, which can then be used as the vascular sample image.

[0110] S2, using the multiple blood vessel sample images and the corresponding label images as training data, iteratively train the preset initial model to obtain the preset blood vessel processing model.

[0111] The preset initial model can be the same as the one mentioned above. Figure 2 and Figure 3 The initial network shown has the same model structure as the one described above. For a detailed description of the model structure, please refer to the above. Figure 2 and Figure 3 The contents shown herein will not be repeated here.

[0112] The above technical solution can effectively train the preset blood vessel processing model, which can enhance the blood vessel image region in the initial medical image to obtain a higher resolution target blood vessel image. This is beneficial to improve the clarity of blood vessels in the image, thereby providing a more accurate diagnostic basis for clinical diagnosis.

[0113] Figure 8 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment of the present disclosure; as shown below. Figure 8 As shown, the image processing apparatus may include:

[0114] The first acquisition module 801 is configured to acquire an initial medical image, which includes a vascular image;

[0115] The determination module 802 is configured to determine the vascular image region in the initial medical image;

[0116] The generation module 803 is configured to generate a masked vascular image to be processed based on the vascular image region;

[0117] The second acquisition module 804 is configured to input the blood vessel image to be processed into a preset blood vessel processing model to obtain a target blood vessel image output by the preset blood vessel processing model. The preset blood vessel processing model is used to perform resolution enhancement processing on the blood vessel image region in the blood vessel image to be processed, and the resolution of the target blood vessel image is higher than that of the blood vessel image to be processed.

[0118] The above technical solution can enhance the vascular image region in the initial medical image through the preset vascular processing model to obtain a target vascular image with higher resolution, which is beneficial to improve the clarity of the vascular in the image, thereby providing a more accurate diagnostic basis for clinical diagnosis.

[0119] Optionally, the preset blood vessel processing model includes a convolutional residual network and a tail interpolation network, the output of the convolutional residual network is coupled to the input of the tail interpolation network, and the output of the tail interpolation network serves as the output of the preset blood vessel processing model.

[0120] The second acquisition module 804 is configured as follows:

[0121] The convolutional residual network is used to extract deep features from the blood vessel image to be processed, so as to obtain a feature map of the first number of channels.

[0122] The tail interpolation network is used to convolve the first number of unused feature maps to obtain a second number of specified feature maps, and then image interpolation is performed on the second number of specified feature maps to obtain the target blood vessel image.

[0123] Optionally, the resolution of the blood vessel image to be processed is M×N, the number of first channels is X, and the resolution of the target blood vessel image is aM×aN, where M and N are positive integers and a is a positive number;

[0124] The second acquisition module 804 is configured as follows:

[0125] The tail interpolation network convolves the first number of unused feature maps into a second number of specified feature maps, where the second number of channels is a. 2 X; and will the a 2 X specified feature maps are interpolated into a target feature map of aM×aN using a preset interpolation method.

[0126] Optionally, the convolutional residual network includes a head convolutional network module and a backbone network module. The output of the head convolutional network module is coupled to the input of the backbone network module, and the output of the backbone network module is coupled to the input of the tail interpolation network. The backbone network module includes multiple convolutional sub-modules, each of which includes a first convolutional layer, an activation function layer, and a second convolutional layer. The input of the first convolutional layer is residually connected to the output of the second convolutional layer. The second acquisition module 804 is configured to:

[0127] The shallow features of the blood vessel image to be processed are extracted using the head convolutional network module.

[0128] The backbone network module extracts deep features from the shallow features output by the head convolutional network module to obtain the feature map to be used for the first number of channels.

[0129] Optionally, the generation module 803 is configured as follows:

[0130] Generate a specified mask corresponding to the blood vessel image region;

[0131] Add the specified mask to the initial medical image to obtain the mask image to be used;

[0132] The mask image to be used is normalized to obtain the blood vessel image to be processed.

[0133] Figure 9 It is based on Figure 8 The illustrated embodiment shows a block diagram of an image processing apparatus; as shown Figure 9 As shown, the device also includes a model training module 805, which is configured as follows:

[0134] Acquire multiple blood vessel sample images and corresponding label images for each blood vessel sample image, wherein the resolution of the blood vessel sample image is lower than that of the label image;

[0135] Using the multiple blood vessel sample images and the corresponding label images as training data, the preset initial model is iteratively trained to obtain the preset blood vessel processing model.

[0136] Optionally, the model training module 805 is configured as follows:

[0137] Acquire a first blood vessel image of a first resolution within each of multiple medical sample images;

[0138] The first blood vessel image is downsampled to obtain a second blood vessel image with a second resolution;

[0139] The second blood vessel image corresponding to each of the plurality of medical sample images is taken as the blood vessel sample image, and the first blood vessel image corresponding to the blood vessel sample image is taken as the label image of the blood vessel sample image.

[0140] The above technical solution can effectively train the preset blood vessel processing model, which can enhance the blood vessel image region in the initial medical image to obtain a higher resolution target blood vessel image. This is beneficial to improve the clarity of blood vessels in the image, thereby providing a more accurate diagnostic basis for clinical diagnosis.

[0141] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0142] Figure 10 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Figure 10 As shown, the electronic device 700 may include a processor 701 and a memory 702. The electronic device 700 may also include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705.

[0143] The processor 701 controls the overall operation of the electronic device 700 to complete all or part of the steps in the image processing method described above. The memory 702 stores various types of data to support the operation of the electronic device 700. This data may include, for example, instructions for any application or method operating on the electronic device 700, and application-related data such as contact data, sent and received messages, pictures, audio, video, etc. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 703 may include a screen and audio components. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 702 or transmitted via communication component 705. The audio component also includes at least one speaker for outputting audio signals. I / O interface 704 provides an interface between processor 701 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IoT, eMTC, or other 5G technologies, or combinations thereof, is not limited here. Therefore, the corresponding communication component 705 may include: a Wi-Fi module, a Bluetooth module, an NFC module, etc.

[0144] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the image processing method described above.

[0145] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the image processing method described above. For example, the computer-readable storage medium may be the memory 702 including program instructions described above, which may be executed by the processor 701 of the electronic device 700 to complete the image processing method described above.

[0146] Figure 11 This is a block diagram illustrating an electronic device according to an exemplary embodiment. For example, electronic device 1900 may be provided as a server. (Refer to...) Figure 11 The electronic device 1900 includes a processor 1922, which may be one or more, and a memory 1932 for storing computer programs executable by the processor 1922. The computer program stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processor 1922 may be configured to execute the computer program to perform the image processing method described above.

[0147] Additionally, the electronic device 1900 may also include a power supply component 1926 and a communication component 1950. The power supply component 1926 can be configured to perform power management of the electronic device 1900, and the communication component 1950 can be configured to enable communication of the electronic device 1900, such as wired or wireless communication. Furthermore, the electronic device 1900 may also include an input / output (I / O) interface 1958. The electronic device 1900 can operate on an operating system stored in memory 1932.

[0148] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the image processing method described above. For example, the non-transitory computer-readable storage medium may be the memory 1932 including the program instructions described above, which may be executed by the processor 1922 of the electronic device 1900 to complete the image processing method described above.

[0149] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a programmable device, the computer program having a code portion for performing the image processing method described above when executed by the programmable device.

[0150] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0151] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0152] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. An image processing method, characterized in that, The method includes: Acquire initial medical images, including vascular images; Identify the vascular image region in the initial medical image; Generate a masked vascular image to be processed based on the vascular image region; The blood vessel image to be processed is input into a preset blood vessel processing model to obtain a target blood vessel image output by the preset blood vessel processing model. The preset blood vessel processing model is used to perform resolution enhancement processing on the blood vessel image region in the blood vessel image to be processed, and the resolution of the target blood vessel image is higher than that of the blood vessel image to be processed. The initial medical image is a CT image, an MRI image, or a PET image; The preset blood vessel processing model includes a convolutional residual network and a tail interpolation network. The output of the convolutional residual network is coupled to the input of the tail interpolation network, and the output of the tail interpolation network serves as the output of the preset blood vessel processing model. The step of inputting the blood vessel image to be processed into a preset blood vessel processing model to obtain the target blood vessel image output by the preset blood vessel processing model includes: The convolutional residual network is used to extract deep features from the blood vessel image to be processed, so as to obtain a first number of unused feature maps. The tail interpolation network is used to convolve the first number of candidate feature maps to obtain a second number of specified feature maps, and image interpolation is performed on the second number of specified feature maps to obtain the target blood vessel image.

2. The method according to claim 1, characterized in that, The resolution of the blood vessel image to be processed is M×N, the number of first channels is X, and the resolution of the target blood vessel image is aM×aN, where M and N are positive integers, and a is a positive number; The step of convolving the first number of candidate feature maps using the tail interpolation network to obtain a second number of specified feature maps, and then performing image interpolation on the second number of specified feature maps to obtain the target blood vessel image, includes: The tail interpolation network convolves the first number of unused feature maps into a second number of specified feature maps, where the second number of channels is a. 2 X; and the a 2 X specified feature maps are interpolated into a target feature map of aM×aN using a preset interpolation method.

3. The method according to claim 1, characterized in that, The convolutional residual network includes a head convolutional network module and a backbone network module. The output of the head convolutional network module is coupled to the input of the backbone network module, and the output of the backbone network module is coupled to the input of the tail interpolation network. The backbone network module includes multiple convolutional sub-modules, each of which includes a first convolutional layer, an activation function layer, and a second convolutional layer. The input of the first convolutional layer is residually connected to the output of the second convolutional layer. The step of inputting the blood vessel image to be processed into a preset blood vessel processing model to obtain the target blood vessel image output by the preset blood vessel processing model includes: The shallow features of the blood vessel image to be processed are extracted by the head convolutional network module. The backbone network module extracts deep features from the shallow features output by the head convolutional network module to obtain the first number of available feature maps.

4. The method according to claim 1, characterized in that, The step of generating a masked vascular image to be processed based on the vascular image region includes: Generate a specified mask corresponding to the blood vessel image region; The specified mask is added to the initial medical image to obtain the mask image to be used; The mask image to be used is normalized to obtain the blood vessel image to be processed.

5. The method according to any one of claims 1-4, characterized in that, The preset blood vessel processing model is trained through the following steps: Acquire multiple blood vessel sample images and corresponding label images for the blood vessel sample images, wherein the resolution of the blood vessel sample images is lower than the resolution of the label images; Using the multiple blood vessel sample images and the corresponding label images as training data, the preset initial model is iteratively trained to obtain the preset blood vessel processing model.

6. The method according to claim 5, characterized in that, The acquisition of multiple vascular sample images and corresponding label images of the vascular sample images includes: Acquire a first blood vessel image of a first resolution within each of a plurality of medical sample images; The first blood vessel image is downsampled to obtain a second blood vessel image with a second resolution; The second blood vessel image corresponding to each of the plurality of medical sample images is used as the blood vessel sample image, and the first blood vessel image corresponding to the blood vessel sample image is used as the label image of the blood vessel sample image.

7. An image processing apparatus, characterized in that, The device includes: The first acquisition module is configured to acquire an initial medical image, the initial medical image including a vascular image; The determination module is configured to determine the vascular image region in the initial medical image; The generation module is configured to generate a masked vascular image to be processed based on the vascular image region; The second acquisition module is configured to input the blood vessel image to be processed into a preset blood vessel processing model to obtain a target blood vessel image output by the preset blood vessel processing model, wherein the preset blood vessel processing model is used to perform resolution enhancement processing on the blood vessel image region in the blood vessel image to be processed, and the resolution of the target blood vessel image is higher than the resolution of the blood vessel image to be processed. The initial medical image is a CT image, an MRI image, or a PET image; The preset blood vessel processing model includes a convolutional residual network and a tail interpolation network. The output of the convolutional residual network is coupled to the input of the tail interpolation network, and the output of the tail interpolation network serves as the output of the preset blood vessel processing model. The second acquisition module is configured as follows: The convolutional residual network is used to extract deep features from the blood vessel image to be processed, so as to obtain a first number of unused feature maps. The tail interpolation network is used to convolve the first number of candidate feature maps to obtain a second number of specified feature maps, and image interpolation is performed on the second number of specified feature maps to obtain the target blood vessel image.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-6.

9. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-6.

Images