Medical image processing method and device, medical equipment and storage medium
The quality evaluation and recovery of medical images is solved through the image perception recovery model, which solves the problem of inconsistent image quality caused by different devices or users, and achieves unified evaluation and improvement of medical image quality.
Patent Information
- Application Number
- CN202411174291.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-12-13
AI Technical Summary
The image quality of different medical equipment or different models of the same equipment varies greatly, resulting in inconsistent user perception habits of image quality and inconsistent output image quality, and lack of effective solutions.
The image perception recovery model is adopted, including the image quality perception module and the image recovery module, and the quality evaluation and recovery of medical images are evaluated and restored by obtaining the trained model to achieve the unity of image quality.
Through the use of image perception recovery model, the image quality output by different devices or users can be unified, and the unified evaluation and improvement of image quality can be achieved, solving the problem of inconsistent image quality output by the device.
Smart Images
Figure CN119130860A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of medical image processing, and in particular to a medical image processing method, apparatus, medical equipment and storage medium. Background Art
[0002] Medical image quality control is generally complex. In actual scenarios, it usually involves multiple medical devices, such as computed tomography (CT) devices, magnetic resonance imaging (MRI) devices, positron emission tomography (PET) devices, digital subtraction angiography (DSA) devices, etc. The image quality of different medical devices or different models of the same medical device varies. How to unify the image quality or monitor the status of the device through the perception of image quality is a complex issue.
[0003] Different users (such as doctors, technicians, etc.) have different perception habits of image quality, which will also lead to inconsistent image quality output by the device. If the quality of the output image is close to a consistent level, it will make it more convenient for users to use (such as diagnosis, teaching, etc.).
[0004] Currently, no effective solution has been proposed to the problem of inconsistent image quality output by different devices or different users in the prior art. Summary of the invention
[0005] The embodiments of the present application provide a medical image processing method, apparatus, medical equipment and storage medium to solve the problem of inconsistent image quality output by a device caused by different devices or different users in the prior art.
[0006] In a first aspect, an embodiment of the present application provides a medical image processing method, the method comprising:
[0007] Acquire a first medical image; wherein different first medical images correspond to different beds of the short-axis PET system;
[0008] Acquire a trained image perception restoration model; the image perception restoration model includes a first image quality perception module and an image restoration module; the image restoration module includes a second image quality perception module;
[0009] Inputting the first medical image into a second image quality perception module to obtain a quality evaluation value of the first medical image;
[0010] Inputting the first medical image into an image restoration module to obtain a second medical image; wherein the second medical image corresponds to a single bed of a long-axis PET system or a whole-body PET system, and a single bed of the long-axis PET system or the whole-body PET system is equivalent to the sum of different beds of a short-axis PET system, and the quality of the second medical image is higher than that of the first medical image;
[0011] The second medical image is input into the first image quality perception module to obtain a quality evaluation value of the second medical image.
[0012] In some embodiments, inputting the first medical image into a second image quality perception module to obtain a quality evaluation value of the first medical image includes:
[0013] Performing feature extraction on the first medical image to obtain a first feature map;
[0014] Downsampling the first feature map to obtain a second feature map;
[0015] A quality evaluation value of the first medical image is determined according to the second feature map.
[0016] In some embodiments, inputting the first medical image into an image restoration module to obtain a second medical image includes:
[0017] Upsampling the second feature map to obtain a third feature map;
[0018] Concatenating the third feature map and the first feature map to obtain a fourth feature map;
[0019] Perform feature extraction on the fourth feature map to obtain the second medical image.
[0020] In some embodiments, inputting the second medical image into the first image quality perception module to obtain the quality evaluation value of the second medical image includes:
[0021] performing feature extraction on the second medical image to obtain a fifth feature map;
[0022] Downsampling the fifth feature map to obtain a sixth feature map;
[0023] A quality evaluation value of the second medical image is determined according to the sixth feature map.
[0024] In some embodiments, the method further comprises:
[0025] Build a training dataset;
[0026] The image perception restoration model is trained according to the training data set to obtain a trained image perception restoration model; the image perception restoration model includes a first image quality perception module and an image restoration module.
[0027] In some of the embodiments, the training data set includes first label data and second label data;
[0028] The first label data includes a first sample image, a second sample image, an evaluation value of the first sample image, and an evaluation value of the second sample image; the second label data includes the first sample image and the evaluation value of the first sample image; and the quality of the second sample image is higher than that of the first sample image.
[0029] In some embodiments, the step of training the image perception restoration model according to the training data set to obtain the trained image perception restoration model includes:
[0030] Training the image perception restoration model according to the first label data to obtain a first image perception restoration model;
[0031] The second image quality perception module and / or the first image quality perception module of the first image perception restoration model are trained according to the second label data to obtain a trained image perception restoration model.
[0032] In a second aspect, an embodiment of the present application provides a medical image processing device, the device comprising:
[0033] An image acquisition module, used for acquiring a first medical image; wherein different first medical images correspond to different beds of the short-axis PET system;
[0034] A model acquisition module, used to acquire a trained image perception restoration model; the image perception restoration model includes a first image quality perception module and an image restoration module;
[0035] A first perception module, used for inputting the first medical image into a second image quality perception module to obtain a quality evaluation value of the first medical image;
[0036] A restoration module, used for inputting the first medical image into an image restoration module to obtain a second medical image; wherein the second medical image corresponds to a single bed of a long-axis PET system or a whole-body PET system, and a single bed of the long-axis PET system or the whole-body PET system is equivalent to the sum of different beds of a short-axis PET system, the image restoration module includes the second image quality perception module, and the quality of the second medical image is higher than the quality of the first medical image;
[0037] The second perception module is used to input the second medical image into the first image quality perception module to obtain a quality evaluation value of the second medical image.
[0038] In a third aspect, an embodiment of the present application provides a medical device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the medical image processing method described in the first aspect.
[0039] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the medical image processing method described in the first aspect are implemented.
[0040] Compared with the prior art, a medical image processing method, apparatus, medical equipment and storage medium provided in the embodiments of the present application obtain a trained image perception restoration model, the image perception restoration model includes a first image quality perception module and an image restoration module, inputs a first medical image into a second image quality perception module, obtains a quality evaluation value of the first medical image, inputs the first medical image into the image restoration module, obtains a second medical image, the quality of the second medical image is higher than the quality of the first medical image, and inputs the second medical image into the first image quality perception module to obtain a quality evaluation value of the second medical image. Thus, a medical image with uniform quality is obtained, and the medical image with uniform quality is evaluated using a uniform quality evaluation standard, thereby solving the problem of inconsistent image quality output by different devices or different users in the prior art, and realizing a uniform evaluation of medical image quality.
[0041] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0043] Figure 1 It is a hardware structure block diagram of a terminal for executing a medical image processing method according to an embodiment of the present application;
[0044] Figure 2 is a flow chart of a medical image processing method according to an embodiment of the present application;
[0045] Figure 3 is a flow chart of another medical image processing method according to an embodiment of the present application;
[0046] Figure 4 is a structural schematic diagram of an image perception restoration model in an embodiment of the present application;
[0047] Figure 5 It is an application schematic diagram of an embodiment of the present application;
[0048] Figure 6 It is a structural block diagram of a medical image processing device according to an embodiment of the present application. DETAILED DESCRIPTION
[0049] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.
[0050] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.
[0051] The method embodiments provided in the embodiments of the present application can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 is a hardware structure block diagram of a terminal for executing a medical image processing method according to an embodiment of the present application. Figure 1 As shown, the terminal may include one or more ( Figure 1Only one is shown in the figure) processor 102 and memory 104 for storing data, wherein processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.
[0052] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as a computer program corresponding to a medical image processing method in an embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0053] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.
[0054] This embodiment provides a medical image processing method. Figure 2 is a flow chart of a medical image processing method according to an embodiment of the present application. Figure 2 As shown, the process includes the following steps:
[0055] Step S200, acquiring a first medical image; wherein different first medical images correspond to different beds of the short-axis PET system.
[0056] In this step, the first medical image is a medical image to be restored and to be evaluated for quality.
[0057] Step S210, obtaining a trained image perception restoration model; the image perception restoration model includes a first image quality perception module and an image restoration module; the image restoration module includes a second image quality perception module.
[0058] In this step, a pre-trained image perception restoration model is obtained. The image perception restoration model is used to perceive the image quality of the first medical image input into the image perception restoration model and to restore the medical image to obtain a second medical image, and to perceive the image quality of the second medical image. The image perception restoration model includes a first image quality perception module and an image restoration module, the second image quality perception module is used to perceive the image quality of the first medical image, the first image quality perception module is used to perceive the quality of the second medical image, and the image restoration module is used to restore the first medical image to obtain a second medical image of higher quality. When training the image perception restoration model, the image perception restoration model can be trained as a whole, or the second image quality perception module, the first image quality perception module and the image restoration module can be trained separately, or the image perception restoration model can be first trained as a whole, and then the image perception restoration model can be trained as a whole to fine-tune the model parameters of the image perception restoration model, thereby realizing the training of the image perception restoration model, which is not specifically limited here.
[0059] The first medical image and the second medical image here can be images acquired by medical equipment, and the medical equipment here can be a computer tomography (CT) device, a magnetic resonance imaging (MRI) device, a positron emission tomography (PET) device, a digital subtraction angiography (DSA) device, etc., which are not specifically limited here. The quality of medical images acquired by different medical equipment or different models of product equipment of the same medical equipment is different. For medical images acquired by different medical equipment or different models of product equipment of the same medical equipment, it is necessary to adopt the medical image processing method in the embodiment of the present application, and use the image perception restoration model to automatically realize image restoration and perform image quality evaluation, thereby unifying the quality of medical images output by different medical equipment or different models of product equipment of the same medical equipment, and unifying the corresponding image quality evaluation standards. The image perception restoration model here can be various types of artificial intelligence models. Exemplarily, the image perception restoration model can include but is not limited to a neural network model, a generative adversarial network model, a deep reinforcement learning model, etc., or a combination thereof.
[0060] Step S220: input the first medical image into a second image quality perception module to obtain a quality evaluation value of the first medical image.
[0061] In this step, after acquiring the first medical image, the acquired first medical image is input into the second image quality perception module to obtain the quality evaluation value of the first medical image. The quality evaluation value of the first medical image here is used to identify the image quality of the first medical image. The quality evaluation value can be a specific numerical value, and the image quality of the first medical image is identified by the correspondence between the preset numerical value and the image quality. The quality evaluation value can also be an image grade, and the image quality of the first medical image is identified by the correspondence between the preset image grade and the image quality. The image quality here can be the resolution of the image, the clarity of the image, etc. As a non-limiting example, the quality evaluation value is a specific score. Taking the 5-point system as an example, the lower the score, the worse the image quality. The second image quality perception module outputs a specific 5-point score to identify the image quality of the first medical image. For example, when the quality evaluation value corresponding to the first medical image is 1, it means that the first medical image is a low-quality image. The training data set includes the image quality scores of comprehensive doctors, technicians, and R&D personnel (specifically, image noise and image contrast). This scoring rule was used as a reference in the production of the low-quality training data set for image restoration. Taking 5 points as an example, the image restoration training data set includes data belonging to various scores of the same scanned object. The image perception restoration model based on this training can automatically perceive different image qualities and perform image restoration.
[0062] Step S230, inputting the first medical image into an image recovery module to obtain a second medical image; the image recovery module includes a second image quality perception module, and the quality of the second medical image is higher than the quality of the first medical image; wherein the second medical image corresponds to a single bed of a long-axis PET system or a whole-body PET system, and a single bed of the long-axis PET system or the whole-body PET system is equivalent to the sum of different beds of a short-axis PET system.
[0063] In this step, after acquiring the first medical image, the first medical image is input into the image restoration module to obtain a restored second medical image, wherein the quality of the second medical image is higher than the quality of the first medical image. The image restoration module is used to restore the image quality of the input first medical image, and the quality of the image restored by the image restoration module is higher than the quality of the image before restoration, that is, the quality of the second medical image is higher than the quality of the first medical image. In the process of training the image restoration module, the network parameters of the image restoration module are trained with the goal of improving the quality of the medical image output by the image restoration module. The provided supervised training data pair includes a low-quality PET image and a corresponding high-quality PET image, and the image perception restoration model trained based on this supervised training data will have the ability to restore the image (improve the image quality); usually the training data includes whole-body PET, and the restoration here refers to the reconstructed PET image after PET scanning. The image quality here can be the resolution of the image, the clarity of the image, etc., and the quality of the second medical image higher than the quality of the first medical image can be that the clarity of the second medical image is higher than the clarity of the first medical image, or the resolution of the second medical image is higher than the resolution of the first medical image. The image restoration module here can reuse the structure of the second image quality perception module, that is, the image restoration module uses the intermediate result of the second image quality perception module or the output result of the second image quality perception module to perform image restoration to obtain the second medical image, that is, the quality evaluation and restoration of the first medical image are performed simultaneously. The image restoration module may also not reuse the structure of the second image quality perception module, that is, the image restoration module and the second image quality perception module are used as separate module structures, and the first medical image is input into the second image quality perception module and the image restoration module respectively, and the quality evaluation value of the first medical image and the second medical image are obtained respectively, that is, the quality evaluation and restoration of the first medical image are performed separately and independently.
[0064] Step S240: input the second medical image into the first image quality perception module to obtain a quality evaluation value of the second medical image.
[0065] In this step, the restored second medical image is input into the first image quality perception module to obtain the quality evaluation value of the second medical image. The quality evaluation value of the second medical image here is used to identify the image quality of the second medical image. The quality evaluation value can be a specific numerical value, and the image quality of the second medical image is identified by the correspondence between the preset numerical value and the image quality. The quality evaluation value can also be an image grade, and the image quality of the second medical image is identified by the correspondence between the preset image grade and the image quality. The image quality here can be the resolution of the image, the clarity of the image, etc. The first image quality perception module here can have the same structure as the second image quality perception module, or can have a different structure. The first medical image acquired or generated by different medical devices or different models of the same medical device is restored by the image perception restoration model to obtain a second medical image with uniform image quality, and the image perception restoration model is used to perceive the second medical image to obtain the quality evaluation value of the second medical image with a uniform quality evaluation standard. The input of the image perception restoration model is the first medical image, and the output of the image perception restoration model can be the quality evaluation value of the first medical image, the second medical image, and the quality evaluation value of the second medical image. The output of the image perception restoration model can also be the quality evaluation value of the second medical image and the quality evaluation value of the second medical image. The output of the image perception restoration model is transmitted to the subsequent processing unit or processing module to facilitate the user to perform data analysis and processing. Among them, the network parameters of the second image quality perception module and the first image quality perception module can be the same or different. As a non-limiting example, the quality evaluation value is a specific score. Taking the 5-point system as an example, the lower the score, the worse the image quality. The first image quality perception module outputs a specific 5-point score to identify the image quality of the second medical image. For example, when the quality evaluation value corresponding to the second medical image is 4 or 5, it means that the second medical image is a high-quality image and the image restoration meets expectations.
[0066] In this embodiment, by obtaining a trained image perception restoration model, the image perception restoration model includes a first image quality perception module and an image restoration module, the image restoration module includes a second image quality perception module, the first medical image is input into the second image quality perception module, the quality evaluation value of the first medical image is obtained, the first medical image is input into the image restoration module, the second medical image is obtained, the quality of the second medical image is higher than the quality of the first medical image, the second medical image is input into the first image quality perception module, the quality evaluation value of the second medical image is obtained. Thus, a medical image with uniform quality is obtained, and the medical image with uniform quality is evaluated using a uniform quality evaluation standard, thereby solving the problem of inconsistent image quality output by different devices or different users in the prior art, and realizing a unified evaluation of medical image quality.
[0067] In some of the embodiments, a first medical image is input into a second image quality perception module to obtain a quality evaluation value of the first medical image, including: extracting features of the first medical image to obtain a first feature map; downsampling the first feature map to obtain a second feature map; and determining the quality evaluation value of the first medical image based on the second feature map.
[0068] In this embodiment, the first medical image is input into the second image quality perception module, feature extraction is performed on the first medical image to obtain a first feature map, the first feature map is downsampled to obtain a second feature map, and the quality evaluation value of the first medical image is determined according to the second feature map. Convolution operation can be used here, and convolution operation and nonlinear activation function can also be used to realize feature extraction. The second feature map generates the quality evaluation value of the first medical image according to the full connection operation, or the second feature map generates the quality evaluation value of the first medical image according to the full connection and activation function operation. In this embodiment, the quality evaluation value of the first medical image is generated by performing feature extraction, convolution operation and full connection operation on the first medical image, so as to realize the quality evaluation of the first medical image and realize the unified quality evaluation of the first medical image.
[0069] In some of the embodiments, the first medical image is input into an image recovery module to obtain a second medical image, including: upsampling the second feature map to obtain a third feature map; concatenating the third feature map and the first feature map to obtain a fourth feature map; and extracting features from the fourth feature map to obtain the second medical image.
[0070] In this embodiment, a convolution operation can be used, and a convolution operation and a nonlinear activation function can also be used to implement feature extraction. In this embodiment, the intermediate result of the second image quality perception module is used for splicing to form a new feature map, and the corresponding second medical image is generated according to the intermediate result of the second image quality perception module. The second medical image is generated using the intermediate result of the second image quality perception module, which improves the output efficiency of the image perception restoration model.
[0071] In some of the embodiments, the second medical image is input into the first image quality perception module to obtain a quality evaluation value of the second medical image, including: extracting features of the second medical image to obtain a fifth feature map; downsampling the fifth feature map to obtain a sixth feature map; and determining the quality evaluation value of the second medical image based on the sixth feature map.
[0072] In this embodiment, the second medical image is input into the first image quality perception module, feature extraction is performed on the second medical image to obtain a fifth feature map, the fifth feature map is downsampled to obtain a sixth feature map, and the quality evaluation value of the second medical image is determined according to the sixth feature map. Convolution operation can be used here, and convolution operation and nonlinear activation function can also be used to realize feature extraction. The sixth feature map generates the quality evaluation value of the second medical image according to the full connection operation, or the sixth feature map generates the quality evaluation value of the second medical image according to the full connection and activation function operation. In this embodiment, the quality evaluation value of the second medical image is generated by performing feature extraction, convolution operation and full connection operation on the second medical image, so as to realize the quality evaluation of the second medical image and realize the unified quality evaluation of the second medical image.
[0073] In some of the embodiments, the method further includes: constructing a training data set; training an image perception restoration model according to the training data set to obtain a trained image perception restoration model; the image perception restoration model includes a first image quality perception module and an image restoration module.
[0074] In this embodiment, the image perception restoration model includes a first image quality perception module and an image restoration module. When training the image perception restoration model, the image perception restoration model can be trained as a whole, or the second image quality perception module, the first image quality perception module and the image restoration module can be trained separately. The image perception restoration model can also be trained as a whole first, and then the image perception restoration model can be trained as a whole to fine-tune the model parameters of the image perception restoration model, thereby realizing the training of the image perception restoration model, which is not specifically limited here.
[0075] In some of the embodiments, the training data set includes first label data and second label data; wherein the first label data includes a first sample image, a second sample image, an evaluation value of the first sample image, and an evaluation value of the second sample image; the second label data includes the first sample image and the evaluation value of the first sample image; and the quality of the second sample image is higher than the quality of the first sample image. As a non-limiting example, when constructing the sample image set, the amount of data used to reconstruct the first sample image is less than the amount of data used to reconstruct the second sample, so that the quality of the second sample image is higher than the quality of the first sample image.
[0076] In this embodiment, the first label data is fully supervised label data, and the second label data is semi-supervised label data. In this embodiment, a richer and more accurate data set is constructed by constructing the first label data and the second label data, so that the trained image perception restoration model is more accurate.
[0077] In some of the embodiments, an image perceptual restoration model is trained according to a training data set to obtain a trained image perceptual restoration model, including: training the image perceptual restoration model according to first label data to obtain a first image perceptual restoration model; training the second image quality perception module and / or the first image quality perception module of the first image perceptual restoration model according to second label data to obtain a trained image perceptual restoration model.
[0078] In this embodiment, the image perception restoration model is first trained according to the first label data to obtain the first image perception restoration model, and then the second image quality perception module and / or the first image quality perception module of the first image perception restoration model are trained according to the second label data to obtain the trained image perception restoration model. That is, after the overall training of the image perception restoration model is completed, the network parameters of the non-image perception module in the image perception restoration model network are fixed, and the network parameters of the second image quality perception module and / or the first image quality perception module are fine-tuned using the more extensive second label data, thereby obtaining the trained image perception restoration model. The model training method in this embodiment trains a complete image perception restoration model based on the first label data to obtain the first image perception restoration model, and then uses the second label data to fine-tune the model parameters of the first image perception restoration model to obtain the trained image perception restoration model, thereby improving the efficiency of model training and making the trained image perception restoration model more accurate.
[0079] It should be noted that the feature map in the embodiment of the present application refers to the feature map of the network during the forward propagation process, and the first feature map, the second feature map, the third feature map, the fourth feature map, the fifth feature map, and the sixth feature map represent feature maps of positions at different layers in the network.
[0080] This embodiment also provides a medical image processing method. Figure 3 is a flowchart of another medical image processing method according to an embodiment of the present application. Figure 3 As shown, the process includes the following steps:
[0081] Step S300, constructing a training data set.
[0082] In this step, a training data set is constructed based on the quality evaluation values of medical images by different devices or different models of the same device or different users. The training data set includes two types of data labels, one is fully supervised label data and the other is semi-supervised label data.
[0083] Step S310: training the image perceptual restoration model according to the training data set to obtain a trained image perceptual restoration model.
[0084] In this step, the image perception restoration model includes a first image quality perception module and an image restoration module. When training the image perception restoration model, the image perception restoration model can be trained as a whole, or the second image quality perception module, the first image quality perception module and the image restoration module can be trained separately. The image perception restoration model can also be trained as a whole first, and then the model parameters of the image perception restoration model can be fine-tuned to achieve the training of the image perception restoration model, which is not specifically limited here.
[0085] Step S320: acquiring a first medical image.
[0086] In this step, the first medical image is a medical image to be restored and to be evaluated for quality.
[0087] Step S330: input the first medical image into the second image quality perception module of the trained image perception restoration model to obtain a quality evaluation value of the first medical image.
[0088] In this step, after acquiring the first medical image, the acquired first medical image is input into the second image quality perception module of the trained image perception restoration model to obtain a quality evaluation value of the first medical image.
[0089] Step S340, input the first medical image into the image restoration module of the trained image perception restoration model to obtain a second medical image; the image restoration module includes a second image quality perception module, and the quality of the second medical image is higher than that of the first medical image.
[0090] In this step, after acquiring the first medical image, the first medical image is input into an image restoration module to obtain a restored second medical image, wherein the quality of the second medical image is higher than the quality of the first medical image.
[0091] Step S350: input the second medical image into the first image quality perception module of the trained image perception restoration model to obtain a quality evaluation value of the second medical image.
[0092] In this step, the restored second medical image is input into the first image quality perception module of the trained image perception restoration model to obtain a quality evaluation value of the second medical image.
[0093] In this embodiment, the image perception restoration model is trained according to the training data set to obtain the trained image perception restoration model, the image perception restoration model includes a first image quality perception module and an image restoration module, the image restoration module includes a second image quality perception module, the first medical image is input into the second image quality perception module of the trained image perception restoration model to obtain the quality evaluation value of the first medical image, the first medical image is input into the image restoration module of the trained image perception restoration model to obtain the second medical image, the quality of the second medical image is higher than the quality of the first medical image, the second medical image is input into the first image quality perception module to obtain the quality evaluation value of the second medical image. Thus, a medical image with uniform quality is obtained, and the medical image with uniform quality is evaluated using a uniform quality evaluation standard, thereby solving the problem of inconsistent image quality output by different devices or different users in the prior art, and realizing a uniform evaluation of medical image quality.
[0094] The following is an example in which the image perception restoration model is a neural network model and the medical device is a PET device.
[0095] The structure of the image perception restoration model is as follows Figure 4 As shown in the figure, the image perception restoration model net is a network structure that includes both the image quality perception module and the image restoration module. It can perceive the image quality of the input image and restore the image quality based on it and output the restored image and the corresponding image quality perception score. The image perception quantification can be obtained by performing an adaptive pooling, full connection and activation function softmax operation on the deep features in the model to obtain the image quality score. Figure 4 The first evaluation value or the second evaluation value in , Figure 4 The image perception weight structures of the two parts, the original input first medical image and the second medical image obtained by image restoration, are consistent, that is, the weight structure of the second image quality perception module and the first image quality perception module are consistent, and the weight parameters can be shared. Figure 4 The blue column in the figure can be understood as the feature map obtained after the corresponding operation, and the connection between the white column and the blue column can be understood as feature combination.
[0096] The specific training process of the image perception restoration model is as follows: prepare the data collected by the long-axis system for a long time, perform random sampling at different sampling rates, and perform offline image reconstruction to obtain a series of PET images under different counts; at the same time, prepare the reconstructed images of different short-axis systems (the acquisition conditions used in the actual hospital are not limited). So far, a database containing images of different systems with high and low quality has been established. At the same time, some data in this database are paired data of different counting conditions belonging to the same individual. From the perspective of doctors and / or technicians, a set of image quality association tables is established. The purpose of this image quality association table is to associate PET images under different counts with different scores. The lower the score, the worse the image quality. The data in the database can be represented as fully supervised label data (x, x_score, y, y_score) and semi-supervised label data (x, x_score, *). The purpose of the semi-supervised label is to enrich the data set and provide a more accurate data set. Wherein, x is the image data in the database, i.e., the first medical image; x_score is the first evaluation value, i.e., the quality evaluation value of the first medical image, which is the score of the doctor and / or technician for image x; y is the restoration target of image x, i.e., the second medical image (e.g., x is a long-axis low-count image with low image quality; y is the image with the longest count of the individual to which the data belongs with the best image quality); * indicates that there is no restoration target (if it is a short-axis image, there is no restoration target because it was not collected for a long time; if it is an image with the longest long-axis count, it is the restoration target itself); y_score is the second evaluation value, i.e., the quality evaluation value of the second medical image. Based on the fully supervised and semi-supervised label data generated above, for the model training of the image perception restoration model, first, the complete image perception restoration model is trained based on (x, x_score, y, y_score); after the training, the network parameters of the non-image quality perception module in the image perception restoration model network are fixed, and the network parameters of the image quality perception module are fine-tuned on the wider semi-supervised label data (x, x_score). The trained image-aware restoration model can be used in the workflow as follows: Figure 5 As shown, after the PET image is reconstructed, the PET image is sent to the image perception restoration model. Finally, the user will get a PET image with improved image quality and perform corresponding quality assessment on the previous and next PET images.
[0097] For equipment manufacturers, providing users with a high-quality image is the goal, but equipment manufacturers have high, medium and low-end equipment, but users still use more low-end equipment. With the introduction of long-axis PET systems (high-end equipment), PET images that are closest to the Ground Truth (real data) can be obtained, but at the same time, users also have a large number of short-axis systems. Assigning the image quality of long-axis PET to different complex short-axis systems is a complex system problem. The medical image processing method provided in the embodiment of the present application can take into account both image perception and image restoration, and can automatically perceive different image qualities (even images from different manufacturers' equipment) and perform automatic matching image restoration to restore their image quality to a uniform level.
[0098] The present application also provides a medical image processing device in the embodiments, which is used to implement the above embodiments and preferred implementation modes, and will not be repeated hereafter. The terms "module", "unit", "subunit", etc. used below may implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0099] Figure 6 is a structural block diagram of a medical image processing device according to an embodiment of the present application. Figure 6 As shown, the device comprises:
[0100] The image acquisition module 600 is used to acquire a first medical image; wherein different first medical images correspond to different beds of the short-axis PET system.
[0101] The model acquisition module 610 is used to acquire the trained image perception restoration model; the image perception restoration model includes a first image quality perception module and an image restoration module; the image restoration module includes a second image quality perception module;
[0102] A first perception module 620, configured to input the first medical image into a second image quality perception module to obtain a quality evaluation value of the first medical image;
[0103] A restoration module 630 is used to input the first medical image into an image restoration module to obtain a second medical image; wherein the second medical image corresponds to a single bed of a long-axis PET system or a whole-body PET system, and a single bed of the long-axis PET system or the whole-body PET system is equivalent to the sum of different beds of a short-axis PET system, and the quality of the second medical image is higher than that of the first medical image;
[0104] The second perception module 640 is used to input the second medical image into the first image quality perception module to obtain a quality evaluation value of the second medical image.
[0105] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.
[0106] An embodiment of the present application also provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0107] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.
[0108] Optionally, in the embodiment of the present application, the processor may be configured to perform the following steps through a computer program:
[0109] S1, acquiring a first medical image; wherein different first medical images correspond to different beds of a short-axis PET system.
[0110] S2, obtaining a trained image perception restoration model; the image perception restoration model includes a first image quality perception module and an image restoration module; the image restoration module includes a second image quality perception module;
[0111] S3, inputting the first medical image into a second image quality perception module to obtain a quality evaluation value of the first medical image;
[0112] S4, inputting the first medical image into an image restoration module to obtain a second medical image; wherein the second medical image corresponds to a single bed of a long-axis PET system or a whole-body PET system, and a single bed of the long-axis PET system or the whole-body PET system is equivalent to the sum of different beds of a short-axis PET system, and the quality of the second medical image is higher than the quality of the first medical image;
[0113] S5, inputting the second medical image into the first image quality perception module to obtain a quality evaluation value of the second medical image.
[0114] It should be noted that the specific examples in the embodiments of the present application can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in the embodiments of the present application.
[0115] In addition, in combination with a medical image processing method provided in the above embodiment, a storage medium may be provided in the embodiment of the present application to implement the method. The storage medium stores a computer program; when the computer program is executed by a processor, any medical image processing method in the above embodiment is implemented.
[0116] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.
[0117] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.
[0118] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.
[0119] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.
Claims
1. A medical image processing method, characterized in that: The method comprises: Acquire a first medical image; wherein different first medical images correspond to different beds of the short-axis PET system; Acquire a trained image perception restoration model; the image perception restoration model includes a first image quality perception module and an image restoration module; the image restoration module includes a second image quality perception module; Inputting the first medical image into a second image quality perception module to obtain a quality evaluation value of the first medical image; Inputting the first medical image into an image restoration module to obtain a second medical image; wherein the second medical image corresponds to a single bed of a long-axis PET system or a whole-body PET system, and a single bed of the long-axis PET system or the whole-body PET system is equivalent to the sum of different beds of a short-axis PET system, and the quality of the second medical image is higher than that of the first medical image; The second medical image is input into the first image quality perception module to obtain a quality evaluation value of the second medical image.
2. The medical image processing method according to claim 1, characterized in that: The step of inputting the first medical image into a second image quality perception module to obtain a quality evaluation value of the first medical image includes: Performing feature extraction on the first medical image to obtain a first feature map; Downsampling the first feature map to obtain a second feature map; A quality evaluation value of the first medical image is determined according to the second feature map.
3. The medical image processing method according to claim 2, characterized in that: The step of inputting the first medical image into an image restoration module to obtain a second medical image comprises: Upsampling the second feature map to obtain a third feature map; Concatenating the third feature map and the first feature map to obtain a fourth feature map; Perform feature extraction on the fourth feature map to obtain the second medical image.
4. The medical image processing method according to any one of claims 1 to 3, characterized in that: The step of inputting the second medical image into the first image quality perception module to obtain a quality evaluation value of the second medical image includes: performing feature extraction on the second medical image to obtain a fifth feature map; Downsampling the fifth feature map to obtain a sixth feature map; A quality evaluation value of the second medical image is determined according to the sixth feature map.
5. The medical image processing method according to claim 1, characterized in that: The method further comprises: Build a training dataset; The image perception restoration model is trained according to the training data set to obtain a trained image perception restoration model; the image perception restoration model includes a first image quality perception module and an image restoration module.
6. The medical image processing method according to claim 5, characterized in that: The training data set includes first label data and second label data; The first label data includes a first sample image, a second sample image, an evaluation value of the first sample image, and an evaluation value of the second sample image; the second label data includes the first sample image and the evaluation value of the first sample image; and the quality of the second sample image is higher than that of the first sample image.
7. The medical image processing method according to claim 6, characterized in that: The step of training the image perception restoration model according to the training data set to obtain the trained image perception restoration model includes: Training the image perception restoration model according to the first label data to obtain a first image perception restoration model; The second image quality perception module and / or the first image quality perception module of the first image perception restoration model are trained according to the second label data to obtain a trained image perception restoration model.
8. A medical image processing device, characterized in that: The device comprises: An image acquisition module, used for acquiring a first medical image; wherein different first medical images correspond to different beds of the short-axis PET system; A model acquisition module, used to acquire a trained image perception restoration model; the image perception restoration model includes a first image quality perception module and an image restoration module; the image restoration module includes a second image quality perception module; A first perception module, used for inputting the first medical image into a second image quality perception module to obtain a quality evaluation value of the first medical image; A restoration module, used for inputting the first medical image into an image restoration module to obtain a second medical image; wherein the second medical image corresponds to a single bed of a long-axis PET system or a whole-body PET system, and a single bed of the long-axis PET system or the whole-body PET system is equivalent to the sum of different beds of a short-axis PET system, and the quality of the second medical image is higher than the quality of the first medical image; The second perception module is used to input the second medical image into the first image quality perception module to obtain a quality evaluation value of the second medical image.
9. A medical device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the medical image processing method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the medical image processing method according to any one of claims 1 to 7 are implemented.