Image amplification method, and method, device and equipment for removing image motion artifacts

Abstract

The invention relates to the technical field of image processing, provides an image amplification method and an image motion artifact removal method, device and equipment, and can obtain an image of a part influenced by non-autonomous motion in a human or animal body in a specific motion mode. The method comprises the following steps: acquiring a multi-temporal motion template image of a motion controlled model in a target motion mode; acquiring a motion actual image of the motion uncontrolled entity; and on the basis of registration parameters obtained by the multi-temporal motion template image and the motion actual image, amplifying the motion actual image to obtain a multi-temporal motion image of the motion uncontrolled entity in the target motion mode.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an image augmentation method, an image augmentation device, an image motion artifact removal method, an image motion artifact removal device, a computer device, a storage medium, and a computer program product. BACKGROUND

[0002] By means of CT (Computed Tomography) technology and MR (Magnetic Resonance) technology, a part (belonging to a solid object, a non-digital object) in a human or animal body can be scanned to obtain a corresponding image. In some scenarios (such as a model training scenario for removing motion artifacts), it can be necessary to obtain an image of the same part under a specific motion pattern, such as obtaining a gold standard in a model training set. However, since the human body is in motion at all times, it is difficult to obtain such an image for a part that is not in autonomous motion. SUMMARY

[0003] Therefore, it is necessary to provide an image augmentation method, an image augmentation device, an image motion artifact removal method, an image motion artifact removal device, a computer device, a storage medium, and a computer program product to solve the above technical problems.

[0004] The present application provides an image augmentation method, which comprises:

[0005] obtaining a plurality of time-phase motion template images of a motion-controlled model under a target motion pattern;

[0006] obtaining a motion actual image of a motion-uncontrolled entity;

[0007] augmenting the motion actual image based on registration parameters obtained from the plurality of time-phase motion template images and the motion actual image to obtain a plurality of time-phase motion images of the motion-uncontrolled entity under the target motion pattern.

[0008] In one embodiment, augmenting the motion actual image based on registration parameters obtained from the plurality of time-phase motion template images and the motion actual image comprises:

[0009] determining a target time-phase motion template image in the plurality of time-phase motion template images according to a part shape similarity between each time-phase motion template image in the plurality of time-phase motion template images and the motion actual image;

[0010] performing conversion processing on the plurality of time-phase motion template images based on registration parameters between the target time-phase motion template image and the motion actual image.

[0011] In one embodiment, based on the registration parameters between the target temporal motion template image and the actual motion image, the multi-temporal motion template image is converted, including:

[0012] Based on the registration parameters between the target temporal motion template image and the actual motion image, each temporal motion template image in the multi-temporal motion template image is transformed.

[0013] In one embodiment, based on the registration parameters between the target temporal motion template image and the actual motion image, the multi-temporal motion template image is converted, including:

[0014] Based on the registration parameters between the target temporal motion template image and the actual motion image, the target temporal motion template image is transformed to obtain the target temporal motion image.

[0015] The target temporal motion image is converted based on the registration parameters between the target temporal motion template image and the non-target temporal motion template images in the multi-temporal motion template images.

[0016] This application provides an image augmentation apparatus, the apparatus comprising:

[0017] The template image acquisition module is used to acquire multi-temporal motion template images of the motion-controlled model under the target motion mode;

[0018] The actual image acquisition module is used to acquire actual motion images of uncontrolled moving entities;

[0019] An image augmentation module is used to augment the actual motion image based on the registration parameters obtained from the multi-temporal motion template image and the actual motion image, to obtain a multi-temporal motion image of the uncontrolled motion entity under the target motion mode.

[0020] This application provides a method for removing motion artifacts from images, the method comprising:

[0021] Obtain multi-temporal motion images obtained according to the above-described image augmentation method embodiments;

[0022] Based on the multi-temporal motion images, simulation acquisition and reconstruction are performed to obtain multi-temporal motion artifact images of uncontrolled moving entities under the target motion mode;

[0023] The model is trained based on the multi-temporal motion images and the multi-temporal motion artifact images to obtain the artifact removal model;

[0024] The image to be removed from artifacts is input into the artifact removal model to obtain the target image.

[0025] In one embodiment, based on the multi-temporal motion images, simulated acquisition and reconstruction are performed to obtain multi-temporal motion artifact images of uncontrolled moving entities under the target motion mode, including:

[0026] For each time phase, simulation acquisition is performed based on the motion image of that time phase and multiple adjacent motion images in the multi-time phase motion image to obtain mixed-phase simulation data corresponding to that time phase;

[0027] Reconstruction is performed based on the mixed-phase simulation data corresponding to that time period to obtain the motion artifact image of that time period.

[0028] In one embodiment, simulation acquisition is performed based on the motion image of the specified time phase and multiple adjacent motion images in the multi-time phase motion images to obtain mixed-phase simulated data corresponding to that time phase, including:

[0029] The motion image of this time phase is simulated and acquired to obtain pure phase simulated raw data of the motion image of this time phase;

[0030] Simulated acquisition is performed on each neighboring temporal motion image to obtain pure phase simulated raw data for each neighboring temporal motion image;

[0031] The pure-phase simulated data of the motion image at that time phase and the pure-phase simulated data of each adjacent motion image at that time phase are stitched together to obtain the mixed-phase simulated data corresponding to that time phase.

[0032] This application provides an apparatus for removing image motion artifacts, the apparatus comprising:

[0033] The image acquisition module is used to acquire multi-temporal motion images obtained according to the above-described image augmentation method embodiments;

[0034] The simulation acquisition and reconstruction module is used to perform simulation acquisition and reconstruction based on the multi-temporal motion images to obtain multi-temporal motion artifact images of uncontrolled moving entities under the target motion mode.

[0035] The model training module is used to train the model based on the multi-temporal motion images and the multi-temporal motion artifact images to obtain an artifact removal model.

[0036] The artifact removal module is used to input the image with artifacts to be removed into the artifact removal model to obtain the target image.

[0037] This application provides a computer device, including a memory and a processor, wherein the memory stores a computer program and the processor executes the above-described method.

[0038] This application provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor using the methods described above.

[0039] This application provides a computer program product having a computer program stored thereon, the computer program being executed by a processor using the above-described method.

[0040] Since the motion-controlled model is a digital object, not a physical object inside a human or animal body, its motion process is controllable. Therefore, this application regulates the motion process of the motion-controlled model to obtain a multi-temporal motion template image of the motion-controlled model under the target motion mode. Based on the registration parameters obtained from the multi-temporal motion template image and the actual motion image of the uncontrolled entity, the actual motion image is amplified to obtain a multi-temporal motion image of the uncontrolled entity under the target motion mode. According to the scheme provided in this application, images of parts of a human or animal body affected by involuntary motion under a specific motion mode can be obtained. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart illustrating an image augmentation method in one embodiment;

[0043] Figure 2 This is a flowchart illustrating a method for removing motion artifacts from an image in one embodiment;

[0044] Figure 3 This is a schematic diagram of the overall process of an image augmentation method in one embodiment;

[0045] Figure 4 This is a structural block diagram of an image augmentation device in one embodiment;

[0046] Figure 5 This is a structural block diagram of an image motion artifact removal device in one embodiment;

[0047] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0049] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0050] The image augmentation method provided in this application can be executed by a computer device, including... Figure 1 The steps shown are as follows:

[0051] Step S101: Obtain multi-temporal motion template images of the motion-controlled model under the target motion mode.

[0052] Step S102: Obtain the actual motion image of the uncontrolled moving entity.

[0053] The motion-controlled model and the motion-uncontrolled entity belong to the same body part, such as the heart. The motion-controlled model and the motion-uncontrolled entity do not need to come from the same individual. A motion-controlled model can be set up, and it can be used for different individuals to obtain multi-temporal motion images of the corresponding motion-uncontrolled entity under the target motion mode.

[0054] Taking the heart as an example, users can use computer devices to adjust the periodic motion amplitude and motion frequency of the heart model (which belongs to the motion-controlled model) so that the heart model is in the target motion mode. In this state, the motion images of the heart model in multiple time phases are obtained. In order to distinguish them, the motion images of the heart model in multiple time phases obtained at this time are called multi-time phase motion template images.

[0055] After performing a CT or MR scan on the heart of an individual (which is a non-controlled motion entity), a motion image of the heart at at least one time phase can be obtained. This motion image is called the actual motion image.

[0056] Step S103: Based on the registration parameters obtained from the multi-temporal motion template image and the actual motion image, the actual motion image is amplified to obtain a multi-temporal motion image of the uncontrolled motion entity under the target motion mode.

[0057] Registration parameters can be obtained from the multi-temporal motion template image and the actual motion image. The actual motion image is then augmented based on these registration parameters, and the augmented image serves as a multi-temporal motion image of the uncontrolled motion entity. The actual motion image is an image acquired from a real scan, while the augmented image is an image acquired from a non-real scan. Therefore, the multi-temporal motion image can also be called a multi-temporal motion virtual image.

[0058] Since the multi-temporal motion template image is an image of the motion-controlled model in the target motion mode, the multi-temporal motion image obtained by amplifying the actual motion image based on the multi-temporal motion template image presents the target motion mode.

[0059] Correspondingly, when the multi-temporal motion template image is a full-temporal motion template image, the resulting multi-temporal motion image can be a full-temporal motion image.

[0060] In the above image augmentation method, since the motion-controlled model is a digital object, not a physical object inside a human or animal body, its motion process is controllable. Therefore, this application regulates the motion process of the motion-controlled model to obtain a multi-temporal motion template image of the motion-controlled model under the target motion mode. Based on the registration parameters obtained from the multi-temporal motion template image and the actual motion image of the uncontrolled entity, the actual motion image is augmented to obtain a multi-temporal motion image of the uncontrolled entity under the target motion mode. According to the scheme provided in this application, images of parts of a human or animal body affected by involuntary motion under a specific motion mode can be obtained.

[0061] In one embodiment, the actual motion image is augmented based on the registration parameters obtained from the multi-temporal motion template image and the actual motion image. Specifically, this may include the following steps: determining the target temporal motion template image in the multi-temporal motion template image based on the morphological similarity between each temporal motion template image in the multi-temporal motion template image and the actual motion image; and performing conversion processing on the multi-temporal motion template image based on the registration parameters between the target temporal motion template image and the actual motion image.

[0062] After obtaining the multi-temporal motion template images, the similarity of the part shape between each temporal motion template image and the actual motion image can be calculated. The motion template image with the highest part shape similarity among the multi-temporal motion template images is determined and used as the target temporal motion template image. The images in the multi-temporal motion template images other than the target temporal motion template image are non-target temporal motion template images.

[0063] Next, various registration methods (such as point cloud registration, image optical flow registration, deep learning-based registration, etc.) can be used to register the target temporal motion template image with the actual motion image to obtain registration parameters (registration parameters may include a registration vector field, which can be denoted as Θ).

[0064] The multi-temporal motion template image is converted and processed according to the registration parameters to complete the augmentation of the actual motion image, and obtain the multi-temporal motion image of the uncontrolled moving entity in the target motion mode.

[0065] In the above embodiments, the target temporal motion template image is determined based on the similarity of the part shape between each temporal motion template image and the actual motion image, so as to obtain more accurate registration parameters. With these registration parameters, more accurate multi-temporal motion images can be obtained.

[0066] In one embodiment, the conversion processing of multi-temporal motion template images based on the registration parameters between the target temporal motion template image and the actual motion image includes: converting each temporal motion template image in the multi-temporal motion template images according to the registration parameters between the target temporal motion template image and the actual motion image.

[0067] The multi-temporal motion template image includes {m1, m2, m3, m4, m5, m6, m7, m8, m9, m...} 10 Taking} as an example, the subscript of m indicates the time phase, for example, m1 represents the motion template image of the first time phase; and in the following introduction, the actual motion image is denoted as n.

[0068] If the target temporal motion template image is m3, then the registration parameters between m3 and n can be calculated, and these registration parameters can be applied to m1 to mn. 10 Each temporal motion template image in the image can be used to obtain {m'1, m'2, m'3, m'4, m'5, m'6, m'7, m'8, m'9, m' 10}. Can {m'1, m'2, m'3, m'4, m'5, m'6, m'7, m'8, m'9, m' 10} as a multi-temporal motion image of an uncontrolled moving entity in the target motion mode.

[0069] In this embodiment, based on the registration parameters between the target temporal motion template image and the actual motion image, each temporal motion template image in the multi-temporal motion template image is converted, which can quickly obtain multi-temporal motion images and reduce processing complexity.

[0070] In one embodiment, the conversion processing of the multi-temporal motion template image based on the registration parameters between the target temporal motion template image and the actual motion image includes: converting the target temporal motion template image according to the registration parameters between the target temporal motion template image and the actual motion image to obtain the target temporal motion image; and converting the target temporal motion image according to the registration parameters between the target temporal motion template image and the non-target temporal motion template images in the multi-temporal motion template image.

[0071] The multi-temporal motion template image includes {m1, m2, m3, m4, m5, m6, m7, m8, m9, m...} 10 Let's take} as an example for introduction, and in the following introduction, the actual motion image is denoted as n.

[0072] If the target temporal motion template image is m3, then the registration parameters between m3 and n can be calculated and applied to m3 to obtain m'3, which is used as the target temporal motion image.

[0073] Since m3 is the target temporal motion template image, therefore, {m1, m2, m4, m5, m6, m7, m8, m9, m 10 {m1, m2, m4, m5, m6, m7, m8, m9, m} is a non-target temporal motion template image. The values ​​{m1, m2, m4, m5, m6, m7, m8, m9, m} can be calculated. 10 The registration parameters between each non-target temporal motion template image in {m'1, m'2, m'4, m'5, m'6, m'7, m'8, m'9, m'} and m'3 are determined. Each registration parameter is then applied to m'3 to obtain {m'1, m'2, m'4, m'5, m'6, m'7, m'8, m'9, m'3}. 10 For example, the registration parameters between m1 and m3 can be calculated, and the registration parameters can be applied to m'3 to obtain m'1.

[0074] According to {m'1, m'2, m'4, m'5, m'6, m'7, m'8, m'9, m' 10} and m'3, we can obtain {m'1, m'2, m'3, m'4, m'5, m'6, m'7, m'8, m'9, m' 10}, will {m'1, m'2, m'3, m'4, m'5, m'6, m'7, m'8, m'9, m' 10} as a multi-temporal motion image of an uncontrolled moving entity in the target motion mode.

[0075] In this embodiment, the target temporal motion template image is converted according to the registration parameters between the target temporal motion template image and the actual motion image to obtain the target temporal motion image; the target temporal motion image is converted according to the registration parameters between the target temporal motion template image and the non-target temporal motion template images in the multi-temporal motion template image, which can improve the processing flexibility.

[0076] This application also provides a method for removing image motion artifacts, the method comprising: Figure 2 The steps shown are as follows:

[0077] Step S201: Obtain multi-temporal motion images obtained according to the above-described image augmentation method embodiment;

[0078] Step S202: Simulate acquisition and reconstruction based on multi-temporal motion images to obtain multi-temporal motion artifact images of uncontrolled moving entities under the target motion mode.

[0079] After obtaining multi-temporal motion images, simulation acquisition and reconstruction can be performed based on the multi-temporal motion images to reconstruct multi-temporal motion artifact images.

[0080] Step S203: Train the model based on the multi-temporal motion images and multi-temporal motion artifact images to obtain the artifact removal model.

[0081] For example, for time phase x, a motion image of time phase x can be determined from multi-temporal motion images, and the motion image of time phase x can be used as the gold standard image. A motion artifact image of time phase x can be determined from multi-temporal motion artifact images, and the motion artifact image of time phase x can be used as the image of the input model. The motion image of time phase x as the gold standard image and the motion artifact image of time phase x as the input model image form training data. The model can be trained based on the training data to obtain the artifact removal model.

[0082] Step S204: Input the image to be removed from the artifact removal model to obtain the target image.

[0083] Once motion artifacts are identified in an image that require removal, this image can be called the artifact removal image. The artifact removal image, which is generated by movements that are difficult for the human body to control, can be input into the artifact removal model. The image output by the artifact removal model can be used as the target image. Compared to the artifact removal image, the target image has less severe motion artifacts. Artifacts caused by involuntary movements include, for example, the breathing movements that some patients cannot control when scanning a certain area, and the heart's own beating during a cardiac scan.

[0084] In CT scans of the heart, the heartbeat causes significant artifacts in the resulting images. Therefore, the CT scan images can be used as artifact removal images, which are then input into an artifact removal model to obtain artifact-free images.

[0085] The images involved in this embodiment can belong to CT mode, MR mode, or PET mode. PET stands for Positron Emission Computed Tomography.

[0086] In this embodiment, the aforementioned image augmentation method can be used to obtain multi-temporal motion images of uncontrolled moving entities under the target motion mode. Based on the multi-temporal motion images, simulation acquisition and reconstruction are performed to obtain multi-temporal motion artifact images of uncontrolled moving entities under the target motion mode. These images are then used for model training to obtain a motion artifact removal model with better motion artifact removal performance, thereby improving the removal effect of motion artifacts.

[0087] In one embodiment, simulation acquisition and reconstruction based on multi-temporal motion images are used to obtain multi-temporal motion artifact images of uncontrolled moving entities under target motion modes. Specifically, this may include: for each temporal phase, simulation acquisition is performed based on the motion image of that temporal phase and multiple adjacent temporal phase motion images in the multi-temporal motion images to obtain mixed-phase simulated data corresponding to that time phase; reconstruction is performed based on the mixed-phase simulated data corresponding to that time phase to obtain the motion artifact image of that time phase.

[0088] In medical scanning imaging technology, real scans acquire data, which is then reconstructed to obtain a real image. This real scan data can be referred to as the actual acquired raw data. Simulated acquisition, on the other hand, simulates the aforementioned real scan acquisition process; therefore, the data acquired through simulated acquisition can be called simulated raw data. In CT modal medical imaging, simulated acquisition involves CT orthographic projection, while in MRI modal medical imaging, it involves MRI Fourier transform.

[0089] Taking phase y as an example, a motion image of phase y can be determined in a multi-phase motion image, and the phases adjacent to y can be determined with phase y as the center. For example, phases y-5 to y-1 and phases y+1 to y+5 can be regarded as the phases adjacent to y. The motion images of the phases adjacent to y in the multi-phase motion image are called multiple adjacent phase motion images.

[0090] In the motion image of phase y and multiple adjacent motion images of phase y, simulated acquisition is performed on each motion image of phase y to obtain simulated raw data corresponding to each motion image of phase y. This simulated raw data corresponds to that phase and is not mixed with other phases. Therefore, this simulated raw data can be called pure phase simulated raw data.

[0091] For a time-phase y-motion image and multiple neighboring time-phase motion images, the pure-phase simulated data corresponding to each time-phase motion image is synthesized. The synthesized simulated data corresponds to multiple time phases; therefore, this simulated data can be called mixed-phase simulated data. Since this mixed-phase simulated data is obtained from the time-phase y-motion image and its neighboring motion images, it corresponds to time phase y.

[0092] Reconstruction can be performed using the mixed-phase simulated data corresponding to time phase y. Since the mixed-phase simulated data corresponds to multiple time phases, the reconstructed image has severe motion artifacts. The reconstructed image is then used as the motion artifact image of time phase y.

[0093] In the above embodiments, for each time phase, simulated acquisition is performed based on the motion image of that time phase and multiple adjacent motion images in the multi-time phase motion image to obtain mixed-phase simulated raw data corresponding to that time phase; reconstruction is performed based on the mixed-phase simulated raw data corresponding to that time phase to obtain the motion artifact image of that time phase with more severe motion artifacts, and the artifact removal model is trained in this way.

[0094] In one embodiment, based on the motion image of the current time phase and multiple adjacent motion images in a multi-temporal motion image, simulated acquisition is performed to obtain mixed-phase simulated data corresponding to that time phase. Specifically, this may include: simulating acquisition of the motion image of the current time phase to obtain pure-phase simulated data of the motion image of the current time phase; simulating acquisition of each adjacent motion image of the current time phase to obtain pure-phase simulated data of each adjacent motion image of the current time phase; and stitching together the pure-phase simulated data of the motion image of the current time phase and the pure-phase simulated data of each adjacent motion image of the current time phase to obtain mixed-phase simulated data corresponding to that time phase.

[0095] Taking time phase y and its neighboring time phases y-5 to y-1 and y+1 to y+5 as examples:

[0096] For each time phase in [y-5, y+5], motion images of each time phase can be simulated and acquired to obtain pure-phase simulated data for each time phase in [y-5, y+5]. Next, partial data is extracted from the pure-phase simulated data of each time phase in [y-5, y+5]. This extracted data can be called sub-pure-phase simulated data. For example, partial data is extracted from the pure-phase simulated data corresponding to y-5, and partial data is extracted from the pure-phase simulated data corresponding to y-4. After obtaining the sub-pure-phase simulated data for each time phase in [y-5, y+5] using this extraction method, these sub-pure-phase simulated data are concatenated. The concatenated data is used as the mixed-phase simulated data corresponding to time phase y.

[0097] In one embodiment, the pure-phase simulated data of the motion image at that time phase and the pure-phase simulated data of each adjacent motion image at that time phase are stitched together to obtain mixed-phase simulated data corresponding to that time phase. Specifically, this may include: extracting sub-pure-phase simulated data of the corresponding data acquisition period from the pure-phase simulated data of each motion image at that time phase in chronological order; wherein, the earlier the time phase, the earlier the corresponding data acquisition period; and stitching together the extracted sub-pure-phase simulated data to obtain mixed-phase simulated data corresponding to that time phase.

[0098] In medical scanning imaging technology, a real scan is a scan that lasts for a period of time. Since the simulated acquisition simulates the aforementioned scanning process, the simulated acquisition process also first simulates a scanning process that lasts for a period of time and collects the data formed during the scanning process. There is a distinction between the order of acquisition of the simulated data. For example, some data in the simulated data corresponds to the data acquisition period from t0 to t1, and some data corresponds to the data acquisition period from t1 to t2.

[0099] For example, after obtaining the pure phase simulation data for each phase in [y-5, y+5], the data acquisition time corresponding to the simulation acquisition can be divided according to the number of phases contained in [y-5, y+5] to obtain multiple data acquisition time periods. The multiple data acquisition time periods formed include, for example, [t0, t1], [t1, t2], [t2, t3], [t3, t4], [t4, t5], [t5, t6], [t6, t7], [t7, t8], [t8, t9], [t9, t10], [t10, t11], and these data acquisition time periods are later and later.

[0100] From y-5 to y+5, the time phase gradually becomes later. Therefore, the data for the data acquisition period [t0, t1] is extracted from the pure phase simulated data corresponding to y-5, and this part of the data is called the sub-pure phase simulated data corresponding to y-5. The data for the data acquisition period [t1, t2] is extracted from the pure phase simulated data corresponding to y-4, and this part of the data is called the sub-pure phase simulated data corresponding to y-4. The data for the data acquisition period [t2, t3] is extracted from the pure phase simulated data corresponding to y-3, and this part of the data is called the sub-pure phase simulated data corresponding to y-3. The data for the data acquisition period [t3, t4] is extracted from the pure phase simulated data corresponding to y-2, and this part of the data is called the sub-pure phase simulated data corresponding to y-2. In this way, the sub-pure phase simulated data corresponding to y-1 to y+5 can be obtained.

[0101] After obtaining the simulated pure phase data corresponding to each of y-5 to y+5, these simulated pure phase data can be spliced ​​together, and the splicing result can be used as the simulated mixed phase data corresponding to phase y.

[0102] To better understand the above method, an application example of this application method is described in detail below. In this application example, the heart is taken as an example. Accordingly, the motion-controlled model is called the heart model, and the uncontrolled entity is the heart of an individual (individual A in this application example).

[0103] Reference Figure 3 The process involves acquiring a full-temporal motion template image of the heart model under the target motion mode, as well as an actual motion image of individual A's heart at a specific time phase. The morphological similarity between the motion template image and the actual motion image at each time phase of the full-temporal motion template image is calculated. The motion template image with the highest morphological similarity is selected as the target time phase motion template image. The actual motion image and the target time phase motion template image are then registered to obtain a registration vector field. This registration vector field is applied to the full-temporal motion template image to obtain the full-temporal motion image of individual A's heart under the target motion mode.

[0104] Taking time phase y and adjacent time phases y-5 to y-1 and y+1 to y+5 as examples, the process of obtaining motion artifact images is introduced:

[0105] In the full-time motion image, motion images for each time phase within the range [y-5, y+5] are determined, and simulation acquisition is performed on each time phase motion image to obtain pure-phase simulated data corresponding to each time phase. Following the chronological order of the time phases, sub-pure-phase simulated data for the corresponding data acquisition period are extracted from the pure-phase simulated data for each time phase. The earlier the time phase, the earlier the corresponding data acquisition period. For example, data for the data acquisition period [t0, t1] is extracted from the pure-phase simulated data corresponding to y-5, and this portion is called the sub-pure-phase simulated data corresponding to y-5. Similarly, data for the data acquisition period [t1, t2] is extracted from the pure-phase simulated data corresponding to y-4, and this portion is called the sub-pure-phase simulated data corresponding to y-4. After obtaining the sub-pure-phase simulated data corresponding to [y-5, y+5] in this way, these sub-pure-phase simulated data can be stitched together, and the stitched result serves as the mixed-phase simulated data corresponding to time phase y. The mixed-phase simulated data corresponding to time phase y is then reconstructed to obtain the motion artifact image corresponding to time phase y.

[0106] The motion image of phase y in the full-time motion image is used as the gold standard image, and the motion artifact image corresponding to phase y is used as the input image of the model. The model is trained in this way to obtain the artifact-free image.

[0107] Since the motion-controlled model is a digital object, not a physical object within a human or animal body, its motion process is controllable. Therefore, this application example regulates the motion process of the motion-controlled model to obtain multi-temporal motion template images of the motion-controlled model under the target motion mode. Based on the registration parameters obtained from the multi-temporal motion template images and the actual motion images of the uncontrolled entities, the actual motion images are amplified to obtain multi-temporal motion images of the uncontrolled entities under the target motion mode. According to the scheme provided in this application, images of parts of a human or animal body affected by involuntary motion under a specific motion mode can be obtained. Furthermore, by using the multi-temporal motion images of the uncontrolled entities under the target motion mode for simulation acquisition and reconstruction, multi-temporal motion artifact images of the uncontrolled entities under the target motion mode are obtained. These are then used for model training to obtain an artifact removal model with better motion artifact removal performance, thereby improving the removal effect of motion artifacts.

[0108] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0109] Based on the same inventive concept, this application also provides an image augmentation apparatus for implementing the image augmentation method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more image augmentation apparatus embodiments provided below can be found in the limitations of the image augmentation method described above, and will not be repeated here.

[0110] In one embodiment, such as Figure 4 As shown, an image augmentation device is provided, comprising:

[0111] The template image acquisition module 401 is used to acquire multi-temporal motion template images of the motion-controlled model under the target motion mode;

[0112] Actual image acquisition module 402 is used to acquire actual motion images of uncontrolled moving entities;

[0113] The image augmentation module 403 is used to augment the actual motion image based on the registration parameters obtained from the multi-temporal motion template image and the actual motion image to obtain a multi-temporal motion image of the uncontrolled motion entity under the target motion mode.

[0114] In one embodiment, the image augmentation module 403 is further configured to: determine a target temporal motion template image in the multi-temporal motion template image based on the similarity of the part morphology of each temporal motion template image in the multi-temporal motion template image with the actual motion image; and perform conversion processing on the multi-temporal motion template image based on the registration parameters between the target temporal motion template image and the actual motion image.

[0115] In one embodiment, the image augmentation module 403 is further configured to: perform conversion processing on each phase motion template image in the multi-phase motion template image according to the registration parameters between the target phase motion template image and the actual motion image.

[0116] In one embodiment, the image augmentation module 403 is further configured to: perform conversion processing on the target temporal motion template image according to the registration parameters between the target temporal motion template image and the actual motion image to obtain a target temporal motion image; and perform conversion processing on the target temporal motion image according to the registration parameters between the target temporal motion template image and the non-target temporal motion template images in the multi-temporal motion template images.

[0117] Each module in the aforementioned image augmentation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0118] Based on the same inventive concept, this application also provides an image motion artifact removal apparatus for implementing the image motion artifact removal method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations of one or more image motion artifact removal apparatus embodiments provided below can be found in the limitations of the image motion artifact removal method described above, and will not be repeated here.

[0119] In one embodiment, such as Figure 5 As shown, an image motion artifact removal apparatus is provided, comprising:

[0120] Image acquisition module 501 is used to acquire multi-temporal motion images obtained according to the above-described image augmentation method embodiment;

[0121] The simulation acquisition and reconstruction module 502 is used to perform simulation acquisition and reconstruction based on the multi-temporal motion images to obtain multi-temporal motion artifact images of uncontrolled moving entities under the target motion mode.

[0122] The model training module 503 is used to train the model based on the multi-temporal motion images and the multi-temporal motion artifact images to obtain an artifact removal model.

[0123] The artifact removal module 504 is used to input the image with artifacts to be removed into the artifact removal model to obtain the target image.

[0124] In one embodiment, the simulation acquisition and reconstruction module 502 is further configured to: for each time phase, perform simulation acquisition based on the motion image of that time phase and multiple adjacent motion images in the multi-time phase motion images to obtain mixed-phase simulated data corresponding to that time phase; and perform reconstruction based on the mixed-phase simulated data corresponding to that time phase to obtain the motion artifact image of that time phase.

[0125] In one embodiment, the simulation acquisition and reconstruction module 502 is further configured to: perform simulation acquisition on the temporal motion image to obtain pure phase simulated data of the temporal motion image; perform simulation acquisition on each adjacent temporal motion image to obtain pure phase simulated data of each adjacent temporal motion image; and stitch together the pure phase simulated data of the temporal motion image and the pure phase simulated data of each adjacent temporal motion image to obtain mixed phase simulated data corresponding to the time.

[0126] Each module in the aforementioned image motion artifact removal device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0127] In one exemplary embodiment, a computer device is provided, the internal structure of which can be as shown in the figure. Figure 6 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data related to the methods described above. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements an image augmentation method or an image motion artifact removal method.

[0128] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0129] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in the various method embodiments described above.

[0130] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the various method embodiments described above.

[0131] In one embodiment, a computer program product is provided having a computer program stored thereon, the computer program being executed by a processor of the steps described in the various method embodiments above.

[0132] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0133] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0134] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0135] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image augmentation method, characterized in that, The method includes: Acquire multi-temporal motion template images of a motion-controlled model under a target motion mode; Acquire actual motion images of uncontrolled moving entities; Based on the registration parameters obtained from the multi-temporal motion template image and the actual motion image, the actual motion image is augmented to obtain a multi-temporal motion image of the uncontrolled motion entity under the target motion mode.

2. The method according to claim 1, characterized in that, Based on the registration parameters obtained from the multi-temporal motion template image and the actual motion image, the actual motion image is augmented, including: Based on the similarity of the part shape of each phase motion template image in the multi-temporal motion template image with the actual motion image, the target phase motion template image in the multi-temporal motion template image is determined; Based on the registration parameters between the target temporal motion template image and the actual motion image, the multi-temporal motion template image is converted.

3. The method according to claim 2, characterized in that, Based on the registration parameters between the target temporal motion template image and the actual motion image, the multi-temporal motion template image is converted, including: Based on the registration parameters between the target temporal motion template image and the actual motion image, each temporal motion template image in the multi-temporal motion template image is transformed.

4. The method according to claim 2, characterized in that, Based on the registration parameters between the target temporal motion template image and the actual motion image, the multi-temporal motion template image is converted, including: Based on the registration parameters between the target temporal motion template image and the actual motion image, the target temporal motion template image is transformed to obtain the target temporal motion image. The target temporal motion image is converted based on the registration parameters between the target temporal motion template image and the non-target temporal motion template images in the multi-temporal motion template images.

5. A method for removing motion artifacts in images, characterized in that, The method includes: Obtain multi-temporal motion images obtained by the image augmentation method according to any one of claims 1 to 4; Based on the multi-temporal motion images, simulation acquisition and reconstruction are performed to obtain multi-temporal motion artifact images of uncontrolled moving entities under the target motion mode; The model is trained based on the multi-temporal motion images and the multi-temporal motion artifact images to obtain the artifact removal model; The image to be removed from artifacts is input into the artifact removal model to obtain the target image.

6. The method according to claim 5, characterized in that, Based on the multi-temporal motion images, simulation acquisition and reconstruction are performed to obtain multi-temporal motion artifact images of uncontrolled moving entities under the target motion mode, including: For each time phase, simulation acquisition is performed based on the motion image of that time phase and multiple adjacent motion images in the multi-time phase motion image to obtain mixed-phase simulation data corresponding to that time phase; Reconstruction is performed based on the mixed-phase simulation data corresponding to that time period to obtain the motion artifact image of that time period.

7. The method according to claim 6, characterized in that, Based on the motion image of the specified time phase and multiple adjacent motion images from the multi-time phase motion images, simulated acquisition is performed to obtain mixed-phase simulated data corresponding to that time phase, including: The motion image of this time phase is simulated and acquired to obtain pure phase simulated raw data of the motion image of this time phase; Simulated acquisition is performed on each neighboring temporal motion image to obtain pure phase simulated raw data for each neighboring temporal motion image; The pure-phase simulated data of the motion image at that time phase and the pure-phase simulated data of each adjacent motion image at that time phase are stitched together to obtain the mixed-phase simulated data corresponding to that time phase.

8. An image augmentation device, characterized in that, The device includes: The template image acquisition module is used to acquire multi-temporal motion template images of the motion-controlled model under the target motion mode; The actual image acquisition module is used to acquire actual motion images of uncontrolled moving entities; An image augmentation module is used to augment the actual motion image based on the registration parameters obtained from the multi-temporal motion template image and the actual motion image, to obtain a multi-temporal motion image of the uncontrolled motion entity under the target motion mode.

9. An apparatus for removing image motion artifacts, characterized in that, The method includes: The image acquisition module is used to acquire multi-temporal motion images obtained by the image augmentation method according to any one of claims 1 to 4; The simulation acquisition and reconstruction module is used to perform simulation acquisition and reconstruction based on the multi-temporal motion images to obtain multi-temporal motion artifact images of uncontrolled moving entities under the target motion mode. The model training module is used to train the model based on the multi-temporal motion images and the multi-temporal motion artifact images to obtain an artifact removal model. The artifact removal module is used to input the image with artifacts to be removed into the artifact removal model to obtain the target image.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

Images