An image processing method and system

Abstract

An embodiment of this specification provides an image processing method and system. Among them, the method includes: obtaining first data and second data of a first modality of a target object, reconstructing a first image according to the first data, and reconstructing a second image according to the second data; wherein, the first image corresponds to a first state of the target object, and the second image corresponds to a second state of the target object; obtaining third data of a second modality of the target object, and reconstructing a third image according to the third data; wherein, the third image corresponds to the first state of the target object; and inputting the first image, the second image, and the third image into an image processing model to obtain a fourth image corresponding to the second modality of the target object in the second state.

Description

Technical Field

[0001] This specification relates to the field of image processing, and particularly to an image processing method and system. Background Art

[0002] With the continuous development of medical imaging technology, in order to better examine patients, various technology fusion methods can be used to detect patients. For example, in the PET-CT technology, PET (Positron Emission Computed Tomography) is used to detect organs and soft tissues, and CT (Computed Tomography) is used to perform tomography on the human body. By obtaining CT images and PET images simultaneously, the advantages of the two images complement each other, enabling doctors to obtain accurate anatomical positioning while understanding biological metabolic information, thereby making a comprehensive and accurate judgment on diseases.

[0003] How to reduce the radiation dose received by patients while obtaining accurate medical images has become an urgent problem to be solved currently. Summary of the Invention

[0004] One embodiment of this specification provides an image processing method. The image processing method includes: obtaining first data and second data of a first modality of a target object, reconstructing a first image according to the first data, and reconstructing a second image according to the second data; wherein, the first image corresponds to a first state of the target object, and the second image corresponds to a second state of the target object; obtaining third data of a second modality of the target object, and reconstructing a third image according to the third data; wherein, the third image corresponds to the first state of the target object; inputting the first image, the second image, and the third image into an image processing model to obtain a fourth image of the second modality corresponding to the second state of the target object.

[0005] One embodiment of this specification provides an image processing system. The image processing system includes: a first image acquisition module, which can be used to obtain first data and second data of a first modality of a target object, reconstruct a first image according to the first data, and reconstruct a second image according to the second data; wherein, the first image corresponds to a first state of the target object, and the second image corresponds to a second state of the target object; a second image acquisition module, which can be used to obtain third data of a second modality of the target object, and reconstruct a third image according to the third data; wherein, the third image corresponds to the first state of the target object; and an image processing module, which can be used to input the first image, the second image, and the third image into an image processing model to obtain a fourth image of the second modality corresponding to the second state of the target object.

[0006] One embodiment of this specification provides an image processing apparatus, including a processor for executing the above-mentioned image processing method.

[0007] One embodiment of this specification provides a computer-readable storage medium storing computer instructions, which, when read by a computer, cause the computer to execute the above-mentioned image processing method. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] This specification will be further described by way of exemplary embodiments, which will be described in detail through the accompanying drawings. These embodiments are not restrictive. In these embodiments, the same reference numerals represent the same structures, where:

[0009] Figure 1 is a schematic diagram of an exemplary application scenario of an image processing system according to some embodiments of this specification;

[0010] Figure 2 is an exemplary block diagram of an image processing system according to some embodiments of this specification;

[0011] Figure 3 is an exemplary flowchart of an image processing method according to some embodiments of this specification;

[0012] Figure 4 is an exemplary flowchart of a method for determining a fourth image according to some embodiments of this specification;

[0013] Figure 5 is an exemplary flowchart of a model training method according to some embodiments of this specification. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0014] To more clearly illustrate the technical solutions of the embodiments of this specification, the accompanying drawings required for the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some examples or embodiments of this specification. For those of ordinary skill in the art, without creative efforts, this specification can also be applied to other similar scenarios based on these drawings. Unless obvious from the language context or otherwise stated, the same reference numerals in the drawings represent the same structures or operations.

[0015] It should be understood that the "system", "apparatus", "unit" and / or "module" used herein is a way to distinguish different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the said words can be replaced by other expressions.

[0016] As shown in this specification and the claims, unless the context clearly indicates otherwise, words such as "a", "an", "one", and / or "the" are not specifically singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. A method or device may also include other steps or elements.

[0017] Flowcharts are used in this specification to illustrate the operations performed by the systems according to the embodiments of this specification. It should be understood that the operations before or after do not necessarily have to be executed precisely in order. On the contrary, the steps can be processed in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or one or more steps can be removed from these processes.

[0018] Currently, with the continuous development of medical imaging technology, the method of fusing multiple technologies for detection is widely used in various scenarios. For example, it is used in the research of important organs or tissue structures (such as the heart, lungs, etc.) and the diagnosis of related diseases. The fusion of multiple technologies can include PET-CT, PET-MRI, etc.

[0019] Taking PET-CT as an example, PET-CT can be applied to cardiac analysis. In the related art, based on the advantages of CT scans, CT scan images can be used to perform attenuation correction on PET images, so that the PET images after attenuation correction can achieve the purpose of quantitative analysis and improve the accuracy of PET image diagnosis. Usually, there are two PET scans and one CT scan during the cardiac analysis scan. When the position of the heart does not match due to patient movement or other reasons (such as the heart in the two PET scans not being in the same position), it may affect the analysis of the scanned images. For example, it may lead to inaccurate quantification of the PET image obtained from the second PET scan. At this time, it is necessary to obtain a correct PET image by adding one more CT scan. Although this method can obtain accurate PET images, it will increase the radiation dose to the patient and the detection process.

[0020] Therefore, in some embodiments of this specification, an image processing method is proposed. By means of machine learning, based on the images of two PET scans and one CT scan, a CT image corresponding to the second PET scan can be obtained, and then an accurate PET image of the second scan can be obtained for analysis. While obtaining accurate PET images, the radiation dose is reduced and the analysis process is simplified.

[0021] It should be noted that the above examples are only for illustrative purposes and are not intended to limit the application scenarios of the technical solutions disclosed in this specification. For example, the technical solutions disclosed in the embodiments of this specification can also be applied to PET-MRI, etc. The technical solutions disclosed in this specification will be elaborated in detail below through the description of the drawings.

[0022] Figure 1 It is a schematic diagram of an exemplary application scenario of an image processing system shown in some embodiments of this specification.

[0023] In some embodiments, the image processing system 100 can obtain the first data and the second data of the first modality of the target object, reconstruct the first image according to the first data, and reconstruct the second image according to the second data; it can obtain the third data of the second modality of the target object and reconstruct the third image according to the third data; the image processing system 100 can input the first image, the second image, and the third image into an image processing model to obtain a fourth image corresponding to the second modality of the target object in the second state. For a detailed description of each image, modality, and state, reference can be made to Figure 3 the relevant description.

[0024] As Figure 1 shown, the image processing system 100 can include an imaging device 110, a network 120, a terminal 130, a processing device 140, and a storage device 150.

[0025] The imaging device 110 can be used to scan a target object to obtain scan data and form an image. The imaging device 110 can be a medical imaging device, for example, a CT (Computed Tomography) imaging device, a PET (Positron Emission Computed Tomography) imaging device, an MRI (Magnetic Resonance Imaging) imaging device, a PET-CT imaging device, a PET-MRI imaging device, etc. In some embodiments, the imaging device 110 can include a gantry 111, a detector 112, a scan area 113, and a scan bed 114. The target object can be placed on the scan bed 114 for scanning. The gantry 111 can support the detector 112. In some embodiments, the detector 112 can include one or more detector units. The detector unit can be and / or include a single-row detector and / or a multi-row detector. The detector unit can include a scintillation detector (e.g., a cesium iodide detector) and other detectors, etc. In some embodiments, the gantry 111 can rotate. For example, in a CT imaging device, the gantry 111 can rotate clockwise or counterclockwise around the gantry rotation axis. In some embodiments, the imaging device 110 can further include a radiation scan source, and the radiation scan source can rotate together with the gantry 111. The radiation scan source can emit a radiation beam (e.g., an X-ray) to the target object, and after being attenuated by the target object, it is detected by the detector 112, thereby generating an image signal. In some embodiments, the CT image can be used to correct the PET image.

[0026] The processing device 140 can process the data and / or information obtained from the imaging device 110, the terminal 130, and / or the storage device 150. For example, the processing device 140 can process the image data detected by the detector 112 to obtain an image. In some embodiments, the processing device 140 can be a single server or a server group. The server group can be centralized or distributed. In some embodiments, the processing device 140 can be local or remote. For example, the processing device 140 can access information and / or data from the imaging device 110, the terminal 130, and / or the storage device 150 through the network 120. Also for example, the processing device 140 can be directly connected to the imaging device 110, the terminal 130, and / or the storage device 150 to access information and / or data. In some embodiments, the processing device 140 can be implemented on a cloud platform. For example, the cloud platform can include one or several combinations of a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, a cross cloud, a multi-cloud, etc.

[0027] The terminal 130 may include a mobile device 131, a tablet computer 132, a laptop computer 133, etc., or any combination thereof. In some embodiments, the terminal 130 may interact with other components in the system 100 via a network. For example, the terminal 130 may send one or more control instructions to the imaging device 110 to control the imaging device 110 to scan a target object according to the instructions. In some embodiments, the terminal 130 may be part of the processing device 140. In some embodiments, the terminal 130 may be integrated with the processing device 140 to serve as an operating console of the imaging device 110. For example, a user / operator (e.g., a doctor) of the system 100 may control the operation of the imaging device 110 through this operating console, such as scanning a target object, etc.

[0028] The storage device 150 may store data (e.g., scanning data of a target object, a first image, a second image, etc.), instructions, and / or any other information. In some embodiments, the storage device 150 may store data obtained from the imaging device 110, the terminal 130, and / or the processing device 140. For example, the storage device 150 may store a first image, a second image, etc. of the target object obtained from the imaging device 110. In some embodiments, the storage device 150 may store data and / or instructions that the processing device 140 may execute or use to perform the exemplary methods described in this specification. In some embodiments, the storage device 150 may include one or a combination of a mass storage device, a removable storage device, a volatile read / write memory, a read-only memory (ROM), etc. In some embodiments, the storage device 150 may be implemented through the cloud platform described in this specification. For example, the cloud platform may include one or a combination of a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud, a multi-cloud, etc.

[0029] In some embodiments, the storage device 150 may be connected to the network 120 to enable communication with one or more components in the system 100 (e.g., the processing device 140, the terminal 130, etc.). One or more components in the system 100 may read data or instructions in the storage device 150 via the network 120. In some embodiments, the storage device 150 may be part of the processing device 140 or may be independent and directly or indirectly connected to the processing device.

[0030] Network 120 may include any suitable network capable of facilitating information and / or data exchange of the image processing system 100. In some embodiments, one or more components of the image processing system 100 (e.g., imaging device 110, terminal 130, processing device 140, storage device 150, etc.) may exchange information and / or data with one or more components of the image processing system 100 via the network 120. For example, the processing device 140 may obtain scan data from the imaging device 110 via the network 120. Network 120 may include a public network (such as the Internet), a private network (e.g., local area network (LAN), wide area network (WAN), etc.), a wired network (such as Ethernet), a wireless network (e.g., 802.11 network, wireless Wi-Fi network, etc.), a cellular network (e.g., long term evolution (LTE) network), a frame relay network, a virtual private network (VPN), a satellite network, a telephone network, a router, a hub, a server computer, etc., or a combination of one or more of them. For example, network 120 may include a wired network, an optical fiber network, a telecommunication network, a local area network, a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a network, a network, a near field communication (NFC) network, etc., or a combination of one or more of them. In some embodiments, network 120 may include one or more network access points. For example, network 120 may include wired and / or wireless network access points, such as base stations and / or Internet exchange points, through which one or more components of the system 100 may connect to the network 120 to exchange data and / or information.

[0031] Figure 2 is an exemplary module diagram of an image processing system shown according to some embodiments of this specification. As Figure 2 shown, the image processing system 200 may include a first image acquisition module 210, a second image acquisition module 220, and an image processing module 230.

[0032] The first image acquisition module 210 may be configured to acquire first data and second data of a first modality of a target object, reconstruct a first image according to the first data, and reconstruct a second image according to the second data. Wherein, the first image corresponds to a first state of the target object, and the second image corresponds to a second state of the target object. In some embodiments, the first modality is PET, and the second modality is CT or MRI.

[0033] The second image acquisition module 220 may be configured to acquire third data of a second modality of the target object and reconstruct a third image according to the third data. Wherein, the third image corresponds to the first state of the target object.

[0034] The image processing module 230 can be used to input the first image, the second image, and the third image into an image processing model to obtain a fourth image corresponding to the second modality of the target object in the second state. In some embodiments, the image processing module 230 can be used to determine the motion information between the first state and the second state of the target object based on the first image and the second image; and determine the fourth image based on the motion information and the third image. In some embodiments, the image processing module 230 can perform motion correction on the third image based on the motion information to determine the fourth image. In some embodiments, the image processing module 230 can also be used to perform image reconstruction based on the fourth image and the second data to obtain a fifth image.

[0035] In some embodiments, the image processing model can be obtained by the following method: obtaining a plurality of training samples and their corresponding labels; wherein each training sample includes a sample first image, a sample second image, and a sample third image, and the label is a sample fourth image; the sample first image and the sample second image correspond to the first modality of the target object, the sample third image and the label correspond to the second modality of the target object; the sample first image and the sample third image correspond to the first state of the target object, and the sample second image and the label correspond to the second state of the target object; inputting the training samples into the image processing model to obtain a prediction result; and adjusting the parameters of the image processing model to reduce the difference between the prediction result and the label.

[0036] For a detailed description of the above system modules, reference can be made to the flowchart part of this specification. For example, Figures 3 to 5 the relevant description, which will not be elaborated here. It should be understood that Figure 2The system and its modules shown can be implemented in various ways. For example, in some embodiments, the system and its modules can be implemented through hardware, software, or a combination of software and hardware. Among them, the hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated designed hardware. Those skilled in the art can understand that the above methods and systems can be implemented using computer-executable instructions and / or included in processor control code. For example, such code is provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The system and its modules of this specification can be implemented not only by hardware circuits such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips and transistors, or programmable hardware devices such as field programmable gate arrays and programmable logic devices, but also by software executed by various types of processors, or by a combination of the above hardware circuits and software (e.g., firmware).

[0037] It should be noted that the above description of the image processing system and its modules is only for convenience of description and does not limit this specification within the scope of the examples given. It can be understood that for those skilled in the art, after understanding the principle of the system, they may, without departing from this principle, make any combination of the various modules, or form a subsystem and connect it to other modules. In some embodiments, Figure 2 the first image acquisition module, the second image acquisition module, and the image processing module disclosed in can be different modules in a system, or a module can implement the functions of two or more of the above modules. For example, each module can share a storage module, or each module can have its own storage module respectively. Such deformations are all within the protection scope of this specification.

[0038] Figure 3 is an exemplary flowchart of an image processing method according to some embodiments of this specification. In some embodiments, process 300 can be executed by a processing device, such as processing device 140. For example, process 300 can be stored in a storage device (such as the built-in storage unit of the processing device or an external storage device) in the form of a program or instruction, and when the program or instruction is executed, process 300 can be implemented. Process 300 can include the following operations.

[0039] Step 310, obtain the first data and the second data of the first modality of the target object, reconstruct the first image according to the first data, and reconstruct the second image according to the second data. In some embodiments, step 310 can be executed by the first image acquisition module 210.

[0040] The target object may include a patient or other medical experimental subjects (e.g., laboratory mice or other animals for experiments), etc. The target object may also be a part of a patient or other medical experimental subjects, including organs and / or tissues, e.g., a patient's heart or lungs, etc. In some embodiments, the target object may also include non-biological entities, such as phantoms, artificial objects, etc.

[0041] The first modality may refer to a scanning state for imaging and scanning the target object. For example, the first modality may be PET.

[0042] The first data may refer to the image data obtained by scanning the target object in the first state through the image device of the first modality. For example, the PET image data obtained by scanning the target object in the first state. The first image may refer to the image reconstructed based on the first data.

[0043] In some embodiments, the first state may refer to a resting state. The resting state may refer to a state where the cells of the target object are not stimulated.

[0044] The second data may refer to the image data obtained by scanning the target object in the second state through the image device of the first modality. For example, the PET image data obtained by scanning the target object in the second state. The second image may refer to the image reconstructed based on the second data.

[0045] In some embodiments, the second state may refer to a stress state. The stress state may refer to a state where the cells of the target object are stimulated (e.g., drug stimulation).

[0046] In some embodiments, the image reconstruction method may include the filtered back projection method, the iterative reconstruction method, etc., which are not limited in this specification.

[0047] In some embodiments, the processing device may scan the target object through an imaging device corresponding to the first modality (e.g., a PET imaging device) to obtain the first image and the second image. For example, the processing device may first scan the target object when it is in the first state to obtain the first image. Then it may scan the target object when it is in the second state to obtain the second image.

[0048] In some embodiments, the processing device may obtain the first image and the second image by reading from a storage device, a database, or by invoking a data interface.

[0049] Step 320: Obtain the third data of the second modality of the target object, and reconstruct the third image according to the third data. In some embodiments, step 320 may be executed by the second image acquisition module 220.

[0050] The second modality may refer to another scanning state different from the first modality for imaging and scanning the target object. For example, if the first modality is PET, the second modality may be CT or MRI, etc.

[0051] The third data may refer to the image data obtained by scanning the target object in the first state with the imaging device of the second modality. The third image may refer to the image reconstructed from the point image data.

[0052] In some embodiments, the third image corresponds to the first state of the target object. For example, the third image comes from the target object in the resting state. For example, a CT image obtained by scanning the target object in the resting state.

[0053] In some embodiments, the processing device may scan the target object in the first state with the imaging device corresponding to the second modality (for example, a CT imaging device or an MRI imaging device) to obtain the third image.

[0054] In some embodiments, the processing device may obtain the third image by reading from a storage device or a database and invoking a data interface.

[0055] Step 330: Input the first image, the second image, and the third image into an image processing model to obtain a fourth image corresponding to the second modality of the target object in the second state. In some embodiments, step 330 may be executed by the image processing module 230.

[0056] The fourth image may refer to the image corresponding to the second modality of the target object in the second state. For example, when the target object is in the second state, it can be scanned with the imaging device of the second modality (such as a CT imaging device) to obtain an image (such as a CT image), and the obtained image can be used to correct the second image. The fourth image can be understood as the image corresponding to this CT image, and the difference is that the fourth image is obtained by the image processing model processing the first image, the second image, and the third image. In this way, by obtaining the fourth image through the image processing model, an image corresponding to the second modality can be obtained without performing an additional scan on the target object (i.e., scanning with the imaging device of the second modality in the second state), reducing the radiation dose to the patient and at the same time reducing the detection process.

[0057] Exemplarily, in a cardiac analysis scan, the cardiac health status of a patient can be obtained by analyzing PET images. It is necessary to analyze the PET images of the patient in the resting state and the stress state. To diagnose the patient more accurately, it is necessary to improve the accuracy of the analysis based on PET images. The PET images can be corrected by CT images or MRI images to obtain more accurate PET images. For example, after obtaining the image of the first modality (such as the first PET image) in the first state, the image of the second modality in the first state (such as the first CT image) can be obtained, and the first PET image can be corrected by the first CT image. Similarly, after obtaining the image of the first modality (such as the second PET image) in the second state, the image of the second modality in the second state (such as the second CT image) can be obtained accordingly, and the second PET image can be corrected by the second CT image. It can be understood that the fourth image corresponds to the image of the second modality obtained in the second state.

[0058] In some embodiments, the processing device may input the first image, the second image, and the third image into an image processing model, and the image processing model outputs the fourth image corresponding to the second modality of the target object in the second state.

[0059] In some embodiments, the image processing model may determine the motion information of the target object in the first state and the second state based on the input first image and second image, and then determine and output the fourth image based on the motion information and the third image. For more descriptions on determining the fourth image, reference can be made to Figure 4 and its related descriptions, which will not be elaborated here.

[0060] In some embodiments, the image processing model may include a neural network model, a deep learning model, etc.

[0061] In some embodiments, the image processing model may be trained based on multiple training samples and labels. For more descriptions on training the image processing model, reference can be made to Figure 5 and its related descriptions, which will not be elaborated here.

[0062] In some embodiments, after obtaining the fourth image, the fifth image may be obtained based on the fourth image. The fifth image may refer to the image after correcting the second image.

[0063] The correction may refer to attenuation correction of the quantification in the second image. Quantification may also be referred to as image quantification, which can be understood as the activity concentration of the tracer in the target object during the scan by the imaging device.

[0064] In some embodiments, the processing device may perform image reconstruction based on the fourth image and the second data to obtain a fifth image. For example, the fourth image may be a CT image, and the second data is PET data. The processing device may perform reconstruction by using the CT image and the PET data to obtain a fifth image.

[0065] In some embodiments, the processing device may perform attenuation correction on the second image based on the fourth image to obtain a fifth image. Attenuation correction refers to calculating the true dose by measuring the absorption coefficient of the medium for radiation and based on the attenuation law.

[0066] In some embodiments, the processing device may, in a machine learning manner (such as a deep learning model), obtain a fifth image by inputting the fourth image and the second image into a trained correction model. The correction model may be trained by multiple training samples and labels. The training samples may include a sample fourth image and a sample second image, and the label is a sample fifth image. Among them, the label may be reconstructed based on the fourth image and the second data corresponding to the second image, or obtained by other means, which is not limited in this specification. The model training method may be a common model training method, such as the gradient descent method, etc., which is also not limited in this specification.

[0067] In some embodiments, the processing device may correct the second image based on the distribution of organs and / or tissues on the fourth image. For example, when scanning the heart, if there are ribs above the heart, the radiation emitted by the tracer in the heart during scanning may be blocked by the ribs. In this case, the dose presented on the obtained second image is less than the actual dose, and at this time, attenuation correction needs to be performed on the quantification of the image. Specifically, when the ribs are located above the heart, since a part of the radiation emitted by the tracer is absorbed by the ribs after being blocked, and it can be known from the CT image that there are ribs here, then according to the absorption coefficient of the ribs for radiation and the attenuation law, the true dose here can be calculated. For example, after being blocked by the ribs, the quantification presented on the PET image may be lower than the true quantification. After calculation, it can be known how much dose is absorbed by the ribs. According to the calculated dose, the PET image can be attenuated corrected. The CT image can show the organs and / or tissue structures of the target object. For example, if there are ribs above the heart, then the dose affected by the ribs (such as the absorbed radiation dose) can be added back to the PET image. Thus, it can be seen that the accuracy of the position of the structure in the CT image is crucial. However, when performing the second PET scan, if the position of the heart changes, the position of the ribs in the CT image during the first scan is inaccurate for the PET image during the second scan. At this time, the result obtained by using the CT image of the first scan for correction will be inaccurate. Therefore, an accurate CT image corresponding to the second PET scan needs to be obtained.

[0068] In some embodiments of the present specification, based on an image processing model, by processing the first image, the second image, and the third image obtained by scanning a target object, a fourth image corresponding to scanning the target object in the second state can be obtained. Since the fourth image is obtained through a machine learning model, it is possible to avoid bringing additional radiation dose to the patient and increasing the detection process duration.

[0069] Figure 4 It is an exemplary flowchart of a method for determining a fourth image according to some embodiments of the present specification. In some embodiments, process 400 may include the following steps.

[0070] Step 410, based on the first image and the second image, determine the motion information between the first state and the second state of the target object.

[0071] The motion information may refer to the motion field of the target object or its organs and / or tissues in the first image in the first state and the second image in the second state.

[0072] In some embodiments, the motion field may refer to a registration function from one respiratory image A to another respiratory image B. Through this registration function, the image in state A can be registered to the image in state B. When performing image registration, the motion field information (deformation field) between two respiratory interval images can be calculated through an image registration algorithm, that is, the position where each pixel point in the image of respiratory cycle A can move to the image of respiratory cycle B after passing through the image registration algorithm (or any other equivalent mathematical form). In an ideal situation, after applying this motion field information to the image of respiratory cycle A, an image exactly the same as the image of respiratory cycle B can be obtained.

[0073] In some embodiments, the motion field may also refer to a registration function from an image in one state A (such as a resting state) to another state (such as a stress state). From state A to state B, the positions of the organs and / or tissues of the target object may be different in the image. For example, the heart of the target object moves from one position to another compared to the spine.

[0074] In some embodiments, the motion information may include the motion information of active motion and / or the motion information of passive motion. Active motion may include spontaneous body movement, body jitter, movement of the scanning bed, movement of the scanning device, etc.; passive motion may include involuntary movement of the organs and / or tissues of the target object in a stress state, such as the movement of organs such as the heart and lungs caused by drug stimulation and the movement of organ positions caused by respiratory movement and heartbeat movement.

[0075] In some embodiments, after the processing device inputs the first image and the second image into the image processing model, the image processing model can determine motion information through motion target detection. Motion target detection refers to detecting the changing area in a sequence of images and extracting the moving target from the images.

[0076] In some embodiments, the image processing model can determine the motion information between the first image and the second image through the frame difference method. The frame difference method can extract the motion area in the images by performing pixel-based temporal difference between image sequences (such as between the first image and the second image) through thresholding.

[0077] In some embodiments, the image processing model can determine the motion information between the first image and the second image through the background subtraction method. The background subtraction method can approximate the pixel values of the background image using the parameters of the background, and perform differential comparison between the current image and the background image (for example, taking the first image as the background image and the second image as the current image) to detect the motion area, where the pixel areas with larger differences are considered as the motion area, and the pixel areas with smaller differences are considered as the background area.

[0078] In some embodiments, the processing device can also determine the motion information between the first image and the second image through other means based on the image processing model, and this embodiment does not limit this.

[0079] Step 420: Determine the fourth image based on the motion information and the third image.

[0080] In some embodiments, the image processing model can perform motion correction on the third image based on the motion information to determine the fourth image. For example, the processing device can perform motion correction on the third image based on the motion field between the first image and the second image to obtain the fourth image corresponding to the first modality in the second state.

[0081] It should be noted that since the image processing model can correct the third image based on the motion information between the first image and the second image to obtain the fourth image. In some embodiments, after the image processing model is trained specifically, the image processing model can also determine the motion information based on the second image and the third image, and correct the third image based on the motion information to determine the fourth image. In some embodiments, the processing device can also input the second image and the third image into the image processing model, and the image processing model outputs the fourth image. For more descriptions about the image processing model, reference can be made to other relevant parts of this specification, such as Figure 3 and Figure 5 the relevant descriptions, which will not be elaborated here.

[0082] Figure 5It is an exemplary flowchart of a model training method shown in some embodiments of this specification. In some embodiments, process 500 may include the following steps.

[0083] Step 510, obtain a plurality of training samples and their corresponding labels.

[0084] The training samples may refer to image data used to train an image processing model. The image data may include historical image data.

[0085] In some embodiments, each training sample may include a sample first image, a sample second image, and a sample third image, and the label is a sample fourth image. Among them, the sample first image and the sample second image correspond to the first modality of the target object, the sample third image and the label correspond to the second modality of the target object; the sample first image and the sample third image correspond to the first state of the target object, and the sample second image and the label correspond to the second state of the target object.

[0086] In some embodiments, the first modality is PET, and the second modality is CT or MRI. The first state is the resting state, and the second state is the stress state. Correspondingly, the sample first image may be a sample first PET image, the sample second image may be a sample second PET image, the sample third image may be a sample first CT image or a sample first MRI image, and the label is a sample second CT image or a sample second MRI image.

[0087] In some embodiments, the CT image or MRI image corresponding to the label may be obtained by scanning the target object in the second state with the imaging device corresponding to the second modality. The label may be added manually or automatically, or by other means, and this embodiment does not limit this.

[0088] For the descriptions of the first modality, the second modality, the first state, and the second state, reference may be made to the relevant descriptions in this specification Figure 2 and will not be elaborated here.

[0089] In some embodiments, the processing device may obtain a plurality of training samples and their corresponding labels by reading or calling a data interface from a database or a storage device.

[0090] It should be noted that in some embodiments, the training samples used for training the image processing model can also be adjusted. It can be understood that the key for the image processing model to learn and predict the image of the second modality corresponding to the target object in the second state lies in the motion information of the images when the target object is scanned in the first state and the second state. For example, the motion field during the first scan (which can be a PET scan or a CT scan) and the second PET scan. By performing motion correction based on the motion fields during the first scan and the second PET scan, the CT image or MRI image of the second time can be obtained. Therefore, in addition to the case where the three images (the first image, the second image, and the third image) are all used as inputs in the above embodiments, it is also possible to use the first image corresponding to the first state and the third image corresponding to the second state as inputs, or use the second image in the first state and the third image in the second state as inputs. Correspondingly, the label can still be the fourth image corresponding to the second modality of the target object in the second state.

[0091] Step 520: Input the training samples into the image processing model to obtain a prediction result.

[0092] The prediction result can refer to the result obtained after the image processing model processes the training samples. In some embodiments, the prediction result is the fourth image predicted by the image processing model. The predicted fourth image corresponds to the image of the second modality of the target object in the second state.

[0093] In some embodiments, the processing device can input the training samples into the image processing model, and the image processing model outputs the prediction result.

[0094] Step 530: Adjust the parameters of the image processing model to reduce the difference between the prediction result and the label.

[0095] In some embodiments, the processing device can construct a loss function based on the prediction result and the label. The loss function can reflect the magnitude of the difference between the prediction result and the label. By adjusting the parameters of the image processing model based on the loss function, the difference between the prediction result and the label can be reduced. For example, by continuously adjusting the parameters of the image processing model, the value of the loss function is minimized.

[0096] In some embodiments, the parameters of the image processing model can also be adjusted by other common methods, and this embodiment does not limit this.

[0097] It should be noted that the above descriptions of the respective processes are only for illustration and explanation, and do not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to the processes under the guidance of this specification. However, these modifications and changes are still within the scope of this specification.

[0098] The beneficial effects that may be brought about by the embodiments of this specification include, but are not limited to: obtaining a CT image corresponding to the second PET scan without increasing the process duration and the radiation dose to the patient, and achieving accurate correction of the second PET image.

[0099] It should be noted that the beneficial effects that may be produced by different embodiments are different. In different embodiments, the beneficial effects that may be produced may be any one or several combinations of the above, or any other beneficial effects that may be obtained.

[0100] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only an example and does not constitute a limitation to this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are suggested in this specification, so such modifications, improvements, and corrections still fall within the spirit and scope of the exemplary embodiments of this specification.

[0101] At the same time, this specification uses specific terms to describe the embodiments of this specification. Such as "one embodiment", "an embodiment", and / or "some embodiments" mean a certain feature, structure, or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that the "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more at different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined.

[0102] In addition, unless clearly stated in the claims, the order of the processing elements and sequences, the use of numbers and letters, or the use of other names in this specification are not used to limit the order of the processes and methods in this specification. Although some currently considered useful invention embodiments are discussed through various examples in the above disclosure, it should be understood that such details only serve the purpose of illustration. The appended claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that conform to the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only through software solutions, such as installing the described system on existing servers or mobile devices.

[0103] Similarly, it should be noted that, in order to simplify the presentation disclosed in this specification and thus assist in the understanding of one or more embodiments of the invention, in the foregoing description of the embodiments of this specification, multiple features are sometimes grouped into one embodiment, drawing, or description thereof. However, this disclosure method does not mean that the features required by the object of this specification are more than those mentioned in the claims. In fact, the features of the embodiments are fewer than all the features of the single embodiments disclosed above.

[0104] In some embodiments, numbers are used to describe components and attribute quantities. It should be understood that such numbers used in the description of embodiments are modified by the modifiers "about", "approximate", or "substantially" in some examples. Unless otherwise stated, "about", "approximate", or "substantially" indicate that the said numbers allow a variation of ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, and such approximate values may change according to the characteristics required by individual embodiments. In some embodiments, the numerical parameters should consider the specified significant digits and adopt the method of retaining general digits. Although the numerical ranges and parameters used to confirm the breadth of the scope in some embodiments of this specification are approximate values, in specific embodiments, the setting of such numerical values is as precise as possible within the feasible range.

[0105] For each patent, patent application, patent application publication, and other materials cited in this specification, such as articles, books, specifications, publications, documents, etc., their entire contents are hereby incorporated into this specification by reference. Except for the application history documents that are inconsistent with or conflict with the content of this specification, and except for the documents that limit the broadest scope of the claims of this specification (currently or subsequently appended to this specification). It should be noted that if there are inconsistencies or conflicts between the descriptions, definitions, and / or uses of terms in the supplementary materials of this specification and the content described in this specification, the descriptions, definitions, and / or uses of terms in this specification shall prevail.

[0106] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other deformations may also fall within the scope of this specification. Therefore, by way of example and not limitation, alternative configurations of the embodiments of this specification can be considered to be in accordance with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments clearly introduced and described in this specification.

Claims

1. An image processing method, the method comprising: Obtaining first data and second data of a first modality of a target object, reconstructing a first image according to the first data, and reconstructing a second image according to the second data; wherein, the first image corresponds to a first state of the target object, the second image corresponds to a second state of the target object, the first state is a resting state, and the second state is a stress state; both the first image and the second image are PET images, and the positions of the organs of the target object on the first image and the second image are different; Obtaining third data of a second modality of the target object, and reconstructing a third image according to the third data; wherein, the third image corresponds to the first state of the target object; the third image is a CT image or an MRI image; Inputting the first image, the second image, and the third image into an image processing model to obtain a fourth image corresponding to the second modality of the target object in the second state; the image processing model is a neural network model; Performing attenuation correction on the second image based on the fourth image to obtain a fifth image.

2. The method according to claim 1, wherein, The first modality is PET, The second modality is CT or MRI.

3. The method according to claim 1, inputting the first image, the second image, and the third image into an image processing model to obtain a fourth image corresponding to the second modality of the target object in the second state includes: Determining the motion information between the first state and the second state of the target object based on the first image and the second image; And Determining the fourth image based on the motion information and the third image.

4. The method according to claim 3, the determining the fourth image based on the motion information and the third image includes: Performing motion correction on the third image based on the motion information to determine the fourth image.

5. The method according to claim 1, the image processing model is obtained based on the following process: Obtain a plurality of training samples and their corresponding labels; wherein, Each training sample includes a sample first image, a sample second image, and a sample third image, and the label is a sample fourth image; the sample first image and the sample second image correspond to the first modality of the target object, the sample third image and the label correspond to the second modality of the target object; the sample first image and the sample third image correspond to the first state of the target object, and the sample second image and the label correspond to the second state of the target object; Inputting the training sample into the image processing model to obtain a prediction result; And Adjusting the parameters of the image processing model to reduce the difference between the prediction result and the label.

6. The method according to claim 5, the first modality is PET, and the second modality is CT or MRI.

7. An image processing system, the system comprising: A first image acquisition module, configured to acquire first data and second data of a first modality of a target object, reconstruct a first image according to the first data, and reconstruct a second image according to the second data; wherein, the first image corresponds to a first state of the target object, the second image corresponds to a second state of the target object, the first state is a resting state, and the second state is a stress state; the first image and the second image are both PET images, and the positions of the organs of the target object on the first image and the second image are different; A second image acquisition module, configured to acquire third data of a second modality of a target object, and reconstruct a third image according to the third data; wherein, the third image corresponds to the first state of the target object; the third image is a CT image or an MRI image; and An image processing module, configured to input the first image, the second image, and the third image into an image processing model to obtain a fourth image corresponding to the second modality of the target object in the second state; the image processing model is a neural network model; Perform attenuation correction on the second image based on the fourth image to obtain a fifth image.

8. An image processing apparatus, comprising a processor, where the processor is configured to execute the image processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, where the storage medium stores computer instructions, and when a computer reads the computer instructions in the storage medium, the computer executes the image processing method according to any one of claims 1 to 6.

Images