Medical image processing method and device, electronic equipment and storage medium

Abstract

The invention provides a medical image processing method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence and the field of digital medical treatment. The method comprises the following steps: acquiring an initial medical image; performing noise addition processing on the initial medical image through a preset diffusion model to obtain a target noise medical image; performing multi-scale convolution on the initial medical image to obtain at least two initial feature maps; wherein any two initial feature maps are different in scale and equal in channel number; performing multi-scale fusion according to each initial feature map to obtain a target fusion channel feature map; carrying out attention processing on the target fusion channel feature map to obtain auxiliary de-noising channel features; and performing denoising processing on the auxiliary denoising channel features and the target noise medical image through the diffusion model to obtain a target denoised medical image. According to the invention, the denoising accuracy of the medical image can be ensured, and the time and resources required for denoising can be saved.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence technology and digital medicine, and in particular to a medical image processing method and device, electronic equipment, and storage medium. Background Art

[0002] There are often various noises in medical images, such as electronic noise of the camera, X-ray noise, scan line noise, motion blur, etc. These noises can cause medical images to appear blurry, blurred, distorted, and affect the judgment and diagnosis of medical images.

[0003] There are some medical image denoising technologies that have a certain degree of accuracy. However, these technologies require a lot of time and resources for actual denoising, or require setting reasonable hyperparameters, resulting in unstable denoising accuracy. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose a medical image processing method and device, an electronic device, and a storage medium, which can ensure the accuracy of medical image denoising and save the time and resources required for denoising.

[0005] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application provides a medical image processing method, the method comprising:

[0006] Acquiring an initial medical image;

[0007] The initial medical image is subjected to noise processing by a preset diffusion model to obtain a target noisy medical image;

[0008] Performing multi-scale convolution on the initial medical image to obtain at least two initial feature maps; wherein any two of the initial feature maps have different scales and the same number of channels;

[0009] Perform multi-scale fusion according to each of the initial feature maps to obtain a target fusion channel feature map;

[0010] Performing attention processing on the target fusion channel feature map to obtain auxiliary denoising channel features;

[0011] The auxiliary denoising channel features and the target noisy medical image are denoised using the diffusion model to obtain a target denoised medical image.

[0012] In some embodiments, the denoising process is performed on the auxiliary denoising channel features and the target noisy medical image by using the diffusion model to obtain the target denoised medical image, including:

[0013] Performing image encoding on the target noisy medical image by using the diffusion model to obtain target image noise characteristics;

[0014] The target image noise feature and the auxiliary denoising channel feature are fused through the diffusion model to obtain a channel noise image feature;

[0015] The channel noise image feature is denoised using the diffusion model to obtain the target denoised medical image.

[0016] In some embodiments, performing attention processing on the target fusion channel feature map to obtain auxiliary denoising channel features includes:

[0017] Performing adaptive weight assignment on each initial channel feature in the target fusion channel feature map to obtain an adaptive channel weight;

[0018] Multiplying the adaptive channel weight by the initial channel feature to obtain a target channel feature;

[0019] The auxiliary denoising channel features are obtained by fusing the features of each target channel.

[0020] In some embodiments, the step of adaptively assigning weights to each initial channel feature in the target fusion channel feature map to obtain an adaptive channel weight includes:

[0021] Adaptively process each initial channel feature in the target fusion channel feature map through a preset fully connected layer to obtain an adaptive channel feature;

[0022] The adaptive channel features are weight mapped using a preset activation function to obtain the adaptive channel weight.

[0023] In some embodiments, performing multi-scale fusion according to each of the initial feature maps to obtain a target fused channel feature map includes:

[0024] The initial medical image is subjected to a scale adjustment on each of the initial feature maps to obtain a target feature map; wherein any two of the target feature maps have the same scale and the same number of channels;

[0025] Channel splicing is performed on a plurality of the target feature maps to obtain the target fused channel feature map.

[0026] In some embodiments, the step of rescaling each of the initial feature maps of the initial medical image to obtain a target feature map includes:

[0027] Determining a target reference scale from the scales of each of the initial feature maps;

[0028] Enlarging the scale of the initial medical image that is smaller than the target reference scale to obtain the target feature map;

[0029] The initial medical image that is larger than the target reference scale is scaled down to obtain the target feature map.

[0030] In some embodiments, determining the target reference scale from the scales of each of the initial feature maps comprises:

[0031] Calculating the scale ratio between each of the initial feature maps and the initial medical image to obtain an image scale ratio;

[0032] Rounding the image scale ratio to determine a target rounding difference;

[0033] The image scale ratio corresponding to the smallest target rounding difference is determined as the target reference scale.

[0034] In one embodiment, the image scale ratio includes an image width ratio and an image height ratio, and rounding the image scale ratio to determine the target rounding difference includes:

[0035] Rounding the image width ratio to determine a width rounding difference;

[0036] Rounding the image height ratio to determine a height rounding difference;

[0037] The target rounding difference is determined according to the width rounding difference and the height rounding difference.

[0038] In some embodiments, the step of performing noise processing on the initial medical image using a preset diffusion model to obtain a target noisy medical image includes:

[0039] Determining a noise adding weight according to a noise coefficient of the diffusion model;

[0040] generating an initial noisy medical image according to the noise addition weight and the initial medical image;

[0041] generating random noise according to the noise coefficient and the noise variable of the diffusion model;

[0042] The target noisy medical image is generated according to the initial noisy medical image and the random noise.

[0043] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a medical image processing device, the device comprising:

[0044] An image acquisition module, used for acquiring an initial medical image;

[0045] A noise processing module, used for performing noise processing on the initial medical image through a preset diffusion model to obtain a target noisy medical image;

[0046] A multi-scale convolution module, used for performing multi-scale convolution on the initial medical image to obtain at least two initial feature maps; wherein any two of the initial feature maps have different scales and the same number of channels;

[0047] A multi-scale fusion module, used to perform multi-scale fusion according to each of the initial feature maps to obtain a target fusion channel feature map;

[0048] An attention processing module, used for performing attention processing on the target fusion channel feature map to obtain auxiliary denoising channel features;

[0049] The denoising processing module is used to perform denoising processing on the auxiliary denoising channel features and the target noisy medical image through the diffusion model to obtain a target denoised medical image.

[0050] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the medical image processing method described in the first aspect when executing the computer program.

[0051] To achieve the above objectives, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the medical image processing method described in the first aspect.

[0052] The medical image processing method, medical image processing device, electronic device and computer-readable storage medium proposed in the present application, in addition to introducing a diffusion model to perform noise processing on the initial medical image, also perform multi-scale convolution on the initial medical image to obtain at least two initial feature maps with different scales and equal number of channels. Then, multi-scale convolution is performed on each initial feature map to obtain a target fusion channel feature map. In this way, the obtained target fusion channel feature map has diversity and robustness. Then, attention processing is performed on the target fusion channel feature map to obtain auxiliary denoising channel features, which contribute to the denoising process to a large extent. Finally, the auxiliary denoising channel features and the target noisy medical image are denoised by the diffusion model, so as to achieve the use of the auxiliary denoising channel features to guide the denoising process of the diffusion model, while ensuring that the signal-to-noise ratio and structural similarity of the denoised medical image are similar to or higher than those of the initial medical image, thereby improving the denoising effect. In summary, this embodiment can not only ensure the accuracy of medical image denoising, but also save the time and resources required for denoising by introducing the auxiliary denoising channel features to guide the denoising process of the diffusion model.

[0053] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a flow chart of a medical image processing method provided by an embodiment of the present application;

[0055] Figure 2 yes Figure 1 Flowchart of step 102 in FIG.

[0056] Figure 3 yes Figure 1 Flowchart of step 104 in;

[0057] Figure 4 yes Figure 3 Flow chart of step 301 in;

[0058] Figure 5 yes Figure 1 Flowchart of step 105 in;

[0059] Figure 6 yes Figure 5 Flow chart of step 501 in;

[0060] Figure 7 is a block diagram of the module structure of the medical image processing device provided in an embodiment of the present application;

[0061] Figure 8 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0063] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0065] First, some nouns involved in this application are analyzed:

[0066] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robots, language recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0067] Natural language processing (NLP): NLP uses computers to process, understand and apply human languages ​​(such as Chinese and English). NLP is a branch of artificial intelligence and an interdisciplinary subject between computer science and linguistics. It is often referred to as computational linguistics. Natural language processing includes grammatical analysis, semantic analysis, and text understanding. Natural language processing is often used in technical fields such as machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, information image processing, information extraction and filtering, text classification and clustering, public opinion analysis and opinion mining. It involves data mining related to language processing, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research, and linguistic research related to language computing.

[0068] Information Extraction (NER): A text processing technology that extracts specified types of entity, relationship, event and other factual information from natural language text and forms structured data output. Information extraction is a technology that extracts specific information from text data. Text data is composed of some specific units, such as sentences, paragraphs, and chapters. Text information is composed of some small specific units, such as characters, words, phrases, sentences, paragraphs, or a combination of these specific units. Extracting noun phrases, names, place names, etc. from text data are all text information extraction. Of course, the information extracted by text information extraction technology can be various types of information.

[0069] Magnetic resonance imaging (MRI): The full name is nuclear magnetic resonance imaging technology, also known as spin imaging. It is a diagnostic technology that uses the nuclear magnetic resonance phenomenon of certain atomic nuclei in human tissues, processes the obtained radio frequency signals through electronic computers, and reconstructs images of a certain layer of the human body.

[0070] Computed Tomography (CT): It uses precisely collimated X-ray beams, gamma rays, ultrasound, etc., together with extremely sensitive detectors to perform cross-sectional scans around a certain part of the human body one by one. It has the characteristics of fast scanning time and clear images, and can be used to detect a variety of diseases. According to the different rays used, it can be divided into: X-ray CT (X-CT) and gamma-ray CT (γ-CT), etc.

[0071] Fully connected layer (FC): Each node of the fully connected layer is connected to all nodes of the previous layer, which is used to integrate the features extracted previously. Due to its fully connected characteristics, the fully connected layer generally has the most parameters. It plays the role of a "classifier" in the entire convolutional neural network. If the operations of the convolution layer, pooling layer, and activation function layer are to map the original data to the hidden layer feature space, the fully connected layer plays the role of mapping the learned "distributed feature representation" to the sample label space. In actual use, the fully connected layer can be implemented by convolution operations: the fully connected layer with a fully connected previous layer can be converted into a convolution with a convolution kernel of 1×1; and the fully connected layer with a convolution layer as the previous layer can be converted into a global convolution with a convolution kernel of H×W, where H and W are the height and width of the previous convolution result, respectively.

[0072] Various noises often exist in medical images. For example, medical images have electronic noise of cameras, X-ray noise, scan line noise, motion blur, etc. These noises can cause medical images to be blurred, blurred, distorted, and affect the judgment and diagnosis of medical images. Most people find it difficult to understand medical images. If users have questions about medical images, they can only seek help from online doctors, and real-time services are not available. In the actual application of digital medicine, an online viewing function can be developed to perform denoising on the medical images input by users to facilitate subsequent recognition processing. Specifically, denoising medical images can reduce the impact of noise, improve the quality and clarity of images, and make it easier to observe and diagnose medical images.

[0073] There are some medical image denoising technologies based on diffusion models. Although they have achieved relatively good performance, they still require a lot of time and resources for training and generation or require setting hyperparameters. If the setting is unreasonable, the generated medical images may be distorted or blurred.

[0074] The present application embodiment proposes a medical image processing method, a medical image processing device, an electronic device and a computer-readable storage medium, which ensure the accuracy of medical image denoising and can save the time and resources required for denoising. The present application embodiment can be applied to medical image analysis, diagnosis, reconstruction and other scenarios.

[0075] The medical image processing method provided in the embodiment of the present application can be applied to the terminal and the server side, and can also be software running on the server side. The server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the medical image processing method, etc., but is not limited to the above forms.

[0076] The present application can be used in many general or special computer system environments or configurations. For example: server computers, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0077] The embodiments of the present application provide a medical image processing method and a medical image processing device, an electronic device, and a computer-readable storage medium, which are specifically illustrated by the following embodiments. First, the medical image processing method in the embodiments of the present application is described.

[0078] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0079] Reference Figure 1 , Figure 1 It is an optional flowchart of the medical image processing method provided in an embodiment of the present application, which may include but is not limited to steps 101 to 106.

[0080] Step 101, obtaining an initial medical image;

[0081] Step 102, performing noise processing on the initial medical image using a preset diffusion model to obtain a target noisy medical image;

[0082] Step 103, performing multi-scale convolution on the initial medical image to obtain at least two initial feature maps; wherein any two initial feature maps have different scales and the same number of channels;

[0083] Step 104, performing multi-scale fusion according to each initial feature map to obtain a target fusion channel feature map;

[0084] Step 105, performing attention processing on the target fusion channel feature map to obtain auxiliary denoising channel features;

[0085] Step 106, denoising the auxiliary denoising channel features and the target noisy medical image using a diffusion model to obtain a target denoised medical image.

[0086] In steps 101 to 106 shown in the embodiment of the present application, in addition to introducing a diffusion model to perform noise processing on the initial medical image, multi-scale convolution is performed on the initial medical image to obtain at least two initial feature maps with different scales and equal number of channels. Then, multi-scale convolution is performed on each initial feature map to obtain a target fusion channel feature map. In this way, the obtained target fusion channel feature map has diversity and robustness. Then, the target fusion channel feature map is subjected to attention processing to obtain auxiliary denoising channel features, which contribute more to the denoising process. Finally, the auxiliary denoising channel features and the target noisy medical image are denoised by the diffusion model, so as to achieve the use of auxiliary denoising channel features to guide the denoising process of the diffusion model, while ensuring that the signal-to-noise ratio and structural similarity of the denoised medical image are similar to or higher than those of the initial medical image, thereby improving the denoising effect. In summary, this embodiment can not only ensure the accuracy of medical image denoising, but also save the time and resources required for denoising by introducing auxiliary denoising channel features to guide the denoising process of the diffusion model.

[0087] In step 101 of some embodiments, an initial medical image is obtained. The initial medical image refers to a medical image with different types and degrees of noise. The initial medical image may be a magnetic resonance imaging (MRI) image or a computed tomography (CT) image. Noise includes Gaussian noise, salt and pepper noise, speckle noise, occlusion, etc.

[0088] In one embodiment, the initial medical image may be obtained in the following manner:

[0089] (1) An online image denoising channel pre-deployed in the target online health assessment application receives the medical image uploaded by the user through the online image denoising channel to obtain an initial medical image.

[0090] (2) Read images from a medical image database to obtain an initial medical image.

[0091] (3) Start the camera to collect images of the medical report / medical record report to obtain the initial medical image.

[0092] In step 102 of some embodiments, the initial medical image is subjected to noise processing by a preset diffusion model to obtain a target noisy medical image. The diffusion model is a probabilistic model for generating data. The main idea of ​​the diffusion model is to generate new samples by gradually disrupting (also called noise addition or noise reduction) the data and then gradually restoring (also called denoising). The diffusion model includes a forward diffusion process and a reverse diffusion process. The forward diffusion process, also called the noise addition process, usually gradually adds random noise to the source data (such as an image) to form a sequence that gradually evolves from pure noise. This process is reversible and is used to construct each step of the time series. The reverse diffusion process, also called the denoising process, usually starts from the noise, and the model learns to gradually remove the noise and restore the original data. This process is usually achieved by training a neural network, and the network learns how to generate more realistic samples from less realistic samples. The reverse diffusion process can also be called the reverse process of a Markov chain with a learning length of T.

[0093] In one embodiment, referring to Figure 2 , step 102 may include:

[0094] Step 201, determining the noise adding weight according to the noise coefficient of the diffusion model;

[0095] Step 202, generating an initial noisy medical image according to the noise weight and the initial medical image;

[0096] Step 203, generating random noise according to the noise coefficient and the noise variable of the diffusion model;

[0097] Step 204: Generate a target noisy medical image based on the initial noisy medical image and the random noise.

[0098] In step 201, the noise coefficient is a pre-set coefficient used to add noise to the initial medical image. Step 201 may include: subtracting the noise coefficient from 1 to obtain an initial noise weight; calculating the arithmetic square root of the initial noise weight to obtain the noise weight.

[0099] In step 202, the noise addition weight may be multiplied by a feature matrix corresponding to the initial medical image to obtain an initial noisy medical image.

[0100] In step 203, the noise variable may be a noise variable conforming to a Gaussian distribution.

[0101] In one embodiment, step 203 may include: calculating the arithmetic square root of the noise coefficient to obtain a random weight; and multiplying the random weight and the noise variable to obtain random noise.

[0102] In step 204, the initial noise image and the random noise may be fused, such as directly added, to obtain a target noise medical image.

[0103] In one example, a target noisy medical image may be as follows:

[0104]

[0105] in, represents the target noisy medical image, β t represents the noise factor, Represents the initial noisy medical image. ε t Represents a noise variable that conforms to the Gaussian distribution, that is, ε t ∈(0,I). t∈[0,T], when t=T, the target noisy medical image is equivalent to a pure noise variable.

[0106] The benefit of the above-mentioned embodiment of step 201 to step 204 is that the target noisy medical image is generated by utilizing the joint effect of the noise coefficient and the noise variable, thereby improving the generation accuracy.

[0107] In step 103 of some embodiments, a multi-scale convolution is performed on the initial medical image to obtain at least two initial feature maps. Any two initial feature maps have different scales and the same number of channels. For example, there are 8 initial feature maps in total, and the number of channels of each initial feature map is N.

[0108] In one embodiment, step 103 may include: inputting the initial medical image into at least two convolution layers with different convolution kernel sizes for convolution operations to obtain at least two initial feature maps. For example, the initial medical image is convolved using convolution layers with different convolution kernel sizes to obtain initial feature maps of different scales. Assuming that the initial medical image is denoted as x, the initial feature map can be denoted as f(x)_i^s, where i=1,2,…,N represents the number of channels of the initial feature map, and s=1,2,…,S represents the number of scales of the feature map). Multi-scale feature extraction can enhance the detail information and contextual information of the medical image, thereby improving the auxiliary ability to the denoising process.

[0109] In step 104 of some embodiments, multi-scale fusion is performed according to each initial feature map to obtain a target fused channel feature map. For example, the target fused channel feature map has N×S channels. Multi-scale fusion refers to combining features from different levels to form a more comprehensive feature representation. Common fusion methods include: Concatenation: concatenating feature maps of different scales in the channel dimension. Weighted Sum: weighted sum of feature maps of different scales, and the weights can be obtained by learning. Multi-scale fusion can enhance the diversity and robustness of medical images, thereby improving the adaptability of the denoising function.

[0110] In one embodiment, referring to Figure 3 , step 104 may include:

[0111] Step 301, resizing each initial feature map of the initial medical image to obtain a target feature map; wherein any two target feature maps have the same scale and the same number of channels;

[0112] Step 302, channel splicing of multiple target feature maps to obtain a target fused channel feature map.

[0113] Specifically, the target fusion channel feature map has the same scale as any target feature map. The number of channels of the target fusion channel feature map is the sum of the number of channels of each target feature map. Step 302 specifically includes: splicing the feature maps of multiple target feature maps in a manner of keeping the scale unchanged and increasing the number of channels to obtain the target fusion channel feature map.

[0114] The benefit of the above-mentioned embodiment of step 301 to step 302 is that the information richness of the target fusion channel feature map can be improved.

[0115] In one embodiment, referring to Figure 4 , step 301 may include:

[0116] Step 401, determining a target reference scale from the scales of each initial feature map;

[0117] Step 402, enlarging the scale of the initial medical image that is smaller than the target reference scale to obtain a target feature map;

[0118] Step 403 , downscaling the initial medical image that is larger than the target reference scale to obtain a target feature map.

[0119] In step 401, one of the scales of each initial feature map may be randomly selected as a target reference scale.

[0120] Considering that the target feature map may affect the accuracy of generating auxiliary denoising channel features in the subsequent step and thus affect the accuracy of the denoising process, in one embodiment, step 401 may include:

[0121] Calculate the scale ratio between each initial feature map and the initial medical image to obtain the image scale ratio;

[0122] Rounding the image scale ratio to determine the target rounding difference;

[0123] The image scale ratio corresponding to the smallest target rounding difference is determined as the target reference scale.

[0124] Specifically, the image scale ratio corresponding to the target reference size is closest to an integer. The rounding may be rounding down or rounding up.

[0125] The benefit of this embodiment is that the target reference scale is determined by using the image scale ratio, so that the target feature map maintains an image structure similar to the initial medical image as much as possible, which helps to improve the denoising effect.

[0126] The image scale ratio includes an image width ratio and an image height ratio. In one embodiment, the image scale ratio is rounded to determine the target rounding difference, including:

[0127] Rounding the image width ratio to determine the width rounding difference;

[0128] Rounding the image height ratio to determine the height rounding difference;

[0129] The target rounding difference is determined according to the width rounding difference and the height rounding difference.

[0130] Specifically, the width rounding difference and the height rounding difference may be weighted and added to obtain the rounding difference. Other methods may also be used for determination. This embodiment of the present application does not specifically limit this.

[0131] The benefit of the above embodiment is that the rounding differences corresponding to the width and height are first determined respectively and then the target rounding difference is jointly determined, which is simple and highly flexible.

[0132] In step 402, the scale increase can be performed by upsampling using an upsampler. Upsampling refers to the operation of increasing / enlarging an image. Upsampling can be achieved by interpolation: bilinear interpolation is generally used, and there are also nearest neighbor interpolation, trilinear interpolation, etc. Upsampling can also be achieved by transpose convolution or deconvolution (Transpose Conv): by filling the pixel value intervals of the image with 0 and then performing a standard convolution calculation, the scale of the output image can be made larger than the input.

[0133] In step 403, the scale reduction can be performed by downsampling using a downsampler. Downsampling refers to the operation of compressing / reducing an image. Downsampling can be achieved using a convolution layer with a stride of 2: the image is reduced in size due to the convolution process in order to extract features. The downsampling process is a process of information loss, and the pooling layer is not learnable. Using a learnable convolution layer with a stride of 2 to replace pooling can achieve better results, but of course it also increases a certain amount of calculation. Downsampling can also be achieved using a pooling layer with a stride of 2: pooling downsampling is to reduce the dimension of the feature. Such as max-pooling and average-pooling, max-pooling is currently commonly used because it is simple to calculate and can better retain texture features.

[0134] The benefit of the embodiment of the above steps 401 to 403 is that by first determining the target reference scale from the scales of each initial feature map and then scaling up or down, the accuracy of the target feature map is improved on the basis of being able to complete the scale adjustment.

[0135] In step 105 of some embodiments, attention processing is performed on the target fusion channel feature map to obtain auxiliary denoising channel features. Attention processing is a feature processing method based on the attention mechanism. Attention mechanism: The contribution of different channel features to the denoising process is dynamically adjusted through the attention mechanism.

[0136] In one embodiment, referring to Figure 5 , step 105 may include:

[0137] Step 501, adaptively assigning weights to each initial channel feature in the target fusion channel feature map to obtain adaptive channel weights;

[0138] Step 502, multiplying the adaptive channel weight with the initial channel feature to obtain the target channel feature;

[0139] Step 503, fusing the features of each target channel to obtain auxiliary denoising channel features.

[0140] In step 501, for each initial channel feature i in the target fusion channel feature map, after adaptive weight allocation, an adaptive channel weight w_i can be obtained. w_i is a scalar that indicates the contribution of the initial channel feature i to the denoising process.

[0141] In one embodiment, referring to Figure 6 , step 501 may include:

[0142] Step 601, adaptively processing each initial channel feature in the target fusion channel feature map through a preset fully connected layer to obtain an adaptive channel feature;

[0143] Step 602: weight mapping is performed on the adaptive channel features through a preset activation function to obtain an adaptive channel weight.

[0144] Specifically, this embodiment uses a fully connected layer and an activation function (such as a softmax function) to perform weighted summation on the target fusion channel feature map to obtain the adaptive channel weight of each initial channel feature; the adaptive channel weight w_i calculation method can be as follows:

[0145]

[0146] Among them, f i Represents the adaptive channel feature. α is a hyperparameter that indicates the sensitivity of adaptive weight allocation.

[0147] The benefit of the above-mentioned embodiment of step 601 to step 602 is that the selectivity and flexibility of the medical image can be enhanced through adaptive weight allocation, thereby improving the robustness of the denoising process.

[0148] The auxiliary denoising channel features obtained through step 502 to step 503 can be shown as follows:

[0149]

[0150] Among them, f t represents the auxiliary denoising channel feature. i represents the adaptive channel feature. w_i represents the adaptive channel weight.

[0151] The benefit of the embodiment of the above steps 501 to 503 is that by performing weighted combination using the adaptive channel weights obtained by adaptive weight allocation, the obtained auxiliary denoising channel features can highlight the channels that contribute more to the denoising process and suppress the channels that do not contribute to or interfere with the denoising process.

[0152] In step 106 of some embodiments, the auxiliary denoising channel features and the target noisy medical image are denoised by the diffusion model to obtain a target denoised medical image. Using the auxiliary denoising channel features in the denoising process is equivalent to using the information of the initial medical image to guide the denoising process of the diffusion model, which can improve the denoising effect.

[0153] In one embodiment, step 106 may include:

[0154] Image encoding is performed on the target noisy medical image through a diffusion model to obtain the target image noise characteristics;

[0155] The target image noise features and the auxiliary denoising channel features are fused through the diffusion model to obtain the channel noise image features;

[0156] The channel noise image features are denoised using a diffusion model to obtain the target denoised medical image.

[0157] Specifically, the diffusion model can be Self-Attention combined with U-Net network structure, and the auxiliary denoising channel features are used to guide the denoising process, and finally the target denoised medical image is obtained. The loss function for training the diffusion model can adopt the mean square error function. During training, back propagation is generally used. The general principle of back propagation is: input the training set data into the input layer of the neural network, pass through the hidden layer of the neural network, and finally reach the output layer of the neural network and output the result; since there is an error between the output result of the neural network and the actual result, the error between the estimated value and the actual value is calculated, and the error is back propagated from the output layer to the hidden layer until it is propagated to the input layer; in the process of back propagation, the values ​​of various parameters are adjusted according to the error; the above process is continuously iterated until convergence.

[0158] The above-mentioned image encoding specifically refers to converting the target noisy medical image into a vector of a fixed length, namely, the target image noise feature.

[0159] The above fusion, also known as feature fusion, specifically refers to the process of extracting the beneficial information in each channel to the maximum extent through image processing and computer technology, and finally synthesizing it into high-quality image features to improve the utilization rate of image information, improve the accuracy and reliability of computer interpretation, and enhance the spatial resolution and spectral resolution of the original image, which is conducive to monitoring. Fusion refers to the process of synthesizing multiple image features into one image feature according to certain fusion rules. The fused image features provide a clearer and more accurate description of medical images and are more suitable for subsequent image processing.

[0160] The benefit of the above embodiment is that, when the diffusion model is used for denoising, auxiliary denoising channel features are introduced, which can greatly improve the denoising accuracy without consuming a lot of time and resources, thereby improving the denoising efficiency.

[0161] In summary, the medical image processing method provided by this application can be used to achieve medical image denoising, and the subsequent recognition using the target denoised medical image obtained after denoising can achieve more accurate results and higher accuracy. Adding this function to Ping An Health can enhance the user experience and increase user stickiness, and is highly practical.

[0162] See also Figure 7The present application also provides a medical image processing device, which can implement the above-mentioned medical image processing method. Figure 7 The module structure block diagram of the medical image processing device provided in the embodiment of the present application includes: an image acquisition module 701, a noise processing module 702, a multi-scale convolution module 703, a multi-scale fusion module 704, an attention processing module 705 and a denoising processing module 706. Among them, the image acquisition module 701 is used to acquire an initial medical image; the noise processing module 702 is used to perform noise processing on the initial medical image through a preset diffusion model to obtain a target noisy medical image; the multi-scale convolution module 703 is used to perform multi-scale convolution on the initial medical image to obtain at least two initial feature maps; wherein the scales of any two initial feature maps are different and the number of channels is equal; the multi-scale fusion module 704 is used to perform multi-scale fusion according to each initial feature map to obtain a target fused channel feature map; the attention processing module 705 is used to perform attention processing on the target fused channel feature map to obtain auxiliary denoising channel features; the denoising processing module 706 is used to perform denoising on the auxiliary denoising channel features and the target noisy medical image through the diffusion model to obtain a target denoised medical image.

[0163] In one embodiment, the noise processing module 702 is specifically used to: determine the noise weight according to the noise coefficient of the diffusion model; generate an initial noisy medical image according to the noise weight and the initial medical image; generate random noise according to the noise coefficient and the noise variable of the diffusion model; generate a target noisy medical image according to the initial noisy medical image and the random noise.

[0164] In one embodiment, the multi-scale fusion module 704 is specifically used to: adjust the scale of each initial feature map of the initial medical image to obtain a target feature map; wherein any two target feature maps have the same scale and the same number of channels; and perform channel splicing on multiple target feature maps to obtain a target fused channel feature map.

[0165] In one embodiment, the multi-scale fusion module 704 is further specifically used to: determine a target reference scale from the scales of each initial feature map; scale up an initial medical image smaller than the target reference scale to obtain a target feature map; scale down an initial medical image larger than the target reference scale to obtain a target feature map.

[0166] In one embodiment, the multi-scale fusion module 704 is further specifically used to: calculate the scale ratio between each initial feature map and the initial medical image to obtain an image scale ratio; round the image scale ratio to determine a target rounding difference; and determine the image scale ratio corresponding to the minimum target rounding difference as the target reference scale.

[0167] In one embodiment, the multi-scale fusion module 704 is further specifically used to: round the image width ratio to determine the width rounding difference; round the image height ratio to determine the height rounding difference; and determine the target rounding difference according to the width rounding difference and the height rounding difference.

[0168] In one embodiment, the attention processing module 705 is specifically used to: adaptively assign weights to each initial channel feature in the target fusion channel feature map to obtain adaptive channel weights; multiply the adaptive channel weights by the initial channel features to obtain target channel features; and fuse the target channel features to obtain auxiliary denoising channel features.

[0169] In one embodiment, the attention processing module 705 is further specifically used to: adaptively process each initial channel feature in the target fusion channel feature map through a preset fully connected layer to obtain an adaptive channel feature; and weight map the adaptive channel feature through a preset activation function to obtain an adaptive channel weight.

[0170] In one embodiment, the denoising processing module 706 is specifically used to: perform image encoding on the target noisy medical image through a diffusion model to obtain the target image noise characteristics; fuse the target image noise characteristics and the auxiliary denoising channel characteristics through a diffusion model to obtain the channel noise image characteristics; remove noise from the channel noise image characteristics through a diffusion model to obtain the target denoised medical image.

[0171] It should be noted that the specific implementation of the medical image processing device is basically the same as the specific implementation of the above-mentioned medical image processing method, and will not be repeated here.

[0172] The embodiment of the present application also provides an electronic device, which includes: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, and the program is executed by the processor to realize the above-mentioned medical image processing method. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0173] See also Figure 8 , Figure 8 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0174] The processor 801 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0175] The memory 802 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 can store an operating system and other application programs. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 802, and the processor 801 calls and executes the medical image processing method of the embodiments of this application;

[0176] Input / output interface 803, used to implement information input and output;

[0177] The communication interface 804 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WI FI, Bluetooth, etc.);

[0178] A bus 805 that transmits information between the various components of the device (e.g., the processor 801, the memory 802, the input / output interface 803, and the communication interface 804);

[0179] The processor 801 , the memory 802 , the input / output interface 803 and the communication interface 804 are connected to each other in communication within the device via a bus 805 .

[0180] An embodiment of the present application also provides a computer-readable storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned medical image processing method.

[0181] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0182] The medical image processing method, medical image processing device, electronic device and computer-readable storage medium proposed in the present application, in addition to introducing a diffusion model to perform noise processing on the initial medical image, also perform multi-scale convolution on the initial medical image to obtain at least two initial feature maps with different scales and equal number of channels. Then, multi-scale convolution is performed on each initial feature map to obtain a target fusion channel feature map. In this way, the obtained target fusion channel feature map has diversity and robustness. Then, attention processing is performed on the target fusion channel feature map to obtain auxiliary denoising channel features, which contribute to the denoising process to a large extent. Finally, the auxiliary denoising channel features and the target noisy medical image are denoised by the diffusion model, so as to achieve the use of the auxiliary denoising channel features to guide the denoising process of the diffusion model, while ensuring that the signal-to-noise ratio and structural similarity of the denoised medical image are similar to or higher than those of the initial medical image, thereby improving the denoising effect. In summary, this embodiment can not only ensure the accuracy of medical image denoising, but also save the time and resources required for denoising by introducing the auxiliary denoising channel features to guide the denoising process of the diffusion model.

[0183] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0184] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0185] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0186] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0187] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0188] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0189] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0190] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0191] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0192] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable an electronic device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0193] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A medical image processing method, characterized in that: The method comprises: Acquiring an initial medical image; The initial medical image is subjected to noise processing by a preset diffusion model to obtain a target noisy medical image; Performing multi-scale convolution on the initial medical image to obtain at least two initial feature maps; wherein any two of the initial feature maps have different scales and the same number of channels; Perform multi-scale fusion according to each of the initial feature maps to obtain a target fusion channel feature map; Performing attention processing on the target fusion channel feature map to obtain auxiliary denoising channel features; The auxiliary denoising channel features and the target noisy medical image are denoised using the diffusion model to obtain a target denoised medical image.

2. The method according to claim 1, characterized in that: The step of performing denoising processing on the auxiliary denoising channel features and the target noisy medical image by using the diffusion model to obtain a target denoised medical image includes: Performing image encoding on the target noisy medical image by using the diffusion model to obtain target image noise characteristics; The target image noise feature and the auxiliary denoising channel feature are fused through the diffusion model to obtain a channel noise image feature; The channel noise image feature is denoised using the diffusion model to obtain the target denoised medical image.

3. The method according to claim 1, characterized in that The attention processing is performed on the target fusion channel feature map to obtain auxiliary denoising channel features, including: Performing adaptive weight assignment on each initial channel feature in the target fusion channel feature map to obtain an adaptive channel weight; Multiplying the adaptive channel weight by the initial channel feature to obtain a target channel feature; The auxiliary denoising channel features are obtained by fusing the features of each target channel.

4. The method according to claim 3, characterized in that The step of adaptively allocating weights to each initial channel feature in the target fusion channel feature map to obtain an adaptive channel weight comprises: Adaptively process each initial channel feature in the target fusion channel feature map through a preset fully connected layer to obtain an adaptive channel feature; The adaptive channel features are weight mapped using a preset activation function to obtain the adaptive channel weight.

5. The method according to any one of claims 1 to 4, characterized in that: The multi-scale fusion is performed according to each of the initial feature maps to obtain a target fusion channel feature map, including: The initial medical image is subjected to a scale adjustment on each of the initial feature maps to obtain a target feature map; wherein any two of the target feature maps have the same scale and the same number of channels; Channel splicing is performed on a plurality of the target feature maps to obtain the target fused channel feature map.

6. The method according to claim 5, characterized in that The step of respectively scaling each of the initial feature maps of the initial medical image to obtain a target feature map includes: Determining a target reference scale from the scales of each of the initial feature maps; Enlarging the scale of the initial medical image that is smaller than the target reference scale to obtain the target feature map; The initial medical image that is larger than the target reference scale is scaled down to obtain the target feature map.

7. The method according to any one of claims 1 to 4, characterized in that: The process of performing noise addition processing on the initial medical image by using a preset diffusion model to obtain a target noisy medical image includes: Determining a noise adding weight according to a noise coefficient of the diffusion model; generating an initial noisy medical image according to the noise addition weight and the initial medical image; generating random noise according to the noise coefficient and the noise variable of the diffusion model; The target noisy medical image is generated according to the initial noisy medical image and the random noise.

8. A medical image processing device, characterized in that: The device comprises: An image acquisition module, used for acquiring an initial medical image; A noise processing module, used for performing noise processing on the initial medical image through a preset diffusion model to obtain a target noisy medical image; A multi-scale convolution module, used for performing multi-scale convolution on the initial medical image to obtain at least two initial feature maps; wherein any two of the initial feature maps have different scales and the same number of channels; A multi-scale fusion module, used to perform multi-scale fusion according to each of the initial feature maps to obtain a target fusion channel feature map; An attention processing module, used for performing attention processing on the target fusion channel feature map to obtain auxiliary denoising channel features; The denoising processing module is used to perform denoising processing on the auxiliary denoising channel features and the target noisy medical image through the diffusion model to obtain a target denoised medical image.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images