Image sequence motion correction method and apparatus based on intensity correction optical flow registration

Abstract

The present description relates to an image sequence motion correction method and apparatus based on intensity correction optical flow registration. The method comprises: determining a reference image of each deformed image in a deformed-image sequence, and performing motion correction on the deformed image by means of adjusting the weight of a mechanical regularization term between the reference image and the deformed image, so as to obtain a displacement field of the deformed image relative to the reference image; on the basis of the displacement field and an intensity correction value of a finite element node, performing intensity correction on the reference image; adjusting the weight of the mechanical regularization term, performing motion correction on the deformed image, performing intensity correction on a first corrected reference image on the basis of an optimized displacement field, repeatedly executing the process until an image residual function converges, determining that the deformed image has been corrected, and recording the optimal solution of a weight value of the mechanical regularization term of the deformed image; and on the basis of the optimal solution of the weight value, determining the actual displacement of the deformed image. In the present description, the mechanical model behind physiological motion and an intensity correction technique are integrated into an image registration algorithm to reduce image noise, thereby improving the algorithm accuracy.

Description

Image sequence motion correction method and device based on intensity modified optical flow registration TECHNICAL FIELD

[0001] The present specification relates to the technical field of image registration, and particularly to an image sequence motion correction method and device based on intensity modified optical flow registration. BACKGROUND

[0002] Chemical exchange saturation transfer (CEST) magnetic resonance imaging (MRI) is a new type of in vivo molecular imaging technology based on the exchange of saturated protons and surrounding flowing water protons, which provides important basis for the diagnosis and treatment of diseases such as tumors, neurodegenerative diseases, and cardiovascular diseases, and has great potential for preclinical research and clinical application. However, during the long scanning process of CEST MRI sequence, physiological movements (such as breathing and heartbeat) and body movements (such as head movement) of the human body inevitably occur, resulting in no registration between images when scanning corresponding organs, which affects the final imaging quality and diagnostic results.

[0003] The registration method based on optical flow (Optical Flow) conservation is a conventional registration method for motion correction. This method assumes that only motion exists between image sequences while the intensity remains unchanged, but the intensity of CEST MRI images at different frequencies differs greatly, and this assumption is not applicable. Traditional registration methods introduce regularization terms (such as L1 and L2 regularization) to improve registration accuracy, but the constraint term based on statistics can only handle part of the effects of image noise, cannot be physically explained, and cannot solve the problem of optical flow non-conservation.

[0004] SUMMARY

[0005] To solve the problem of inaccurate image registration caused by large intensity and motion differences in the image registration process in the prior art, the present specification provides an image sequence motion correction method and device based on intensity modified optical flow registration.

[0006] The embodiment of the present specification provides a kind of based on intensity correction optical flow registration image sequence motion correction method, the method comprises: determining the reference image of each deformation image in deformation image sequence, wherein the deformation image sequence includes the multiple frames of deformation image caused by organ tissue movement;By adjusting the weight of the mechanical regularization term of reference image and deformation image, the deformation image is motion corrected, and the displacement field of deformation image relative to reference image is obtained, wherein the mechanical regularization term is based on mechanical equilibrium principle and is constructed;Based on the displacement field, using the intensity correction value of finite element shape function and finite element node, the intensity of the reference image is corrected, and the first corrected reference image is obtained;The weight of mechanical regularization term is adjusted iteratively, and the deformation image is motion corrected, to optimize the displacement field, the intensity of the first corrected reference image is corrected based on the optimized displacement field, and is alternately repeated, until the image residual function restrained by mechanical regularization term converges to preset value, determine that deformation image is corrected, record the optimal solution of the weight of mechanical regularization term of deformation image;According to the weight value optimal solution, the real displacement of each deformation image relative to reference image is determined.

[0007] According to one aspect of the embodiment of the present specification, before adjusting the weight of the mechanical regularization term of deformation image, the mechanical regularization term of image is determined by the following method: the deformation image is gridded, and a plurality of finite element grids of the deformation image are obtained;According to the mechanical equilibrium principle of organ tissue deformation and the force summation of internal nodes of arbitrary finite element grid, the mechanical regularization term of deformation image is constructed, as shown in the following formula: [K] represents the rectangular stiffness matrix of finite element grid and material parameter, {u} represents the required node displacement, It represents the mechanical regularization term of deformation image.

[0008] According to one aspect of the embodiment of the present specification, by adjusting the weight of the mechanical regularization term of deformation image, the motion correction of deformation image includes: determining that the initial weight of mechanical regularization term is the first numerical value;Under the weight condition of the first numerical value of mechanical regularization term, the deformation image is initially motion corrected, and the displacement field of initial motion correction is obtained.

[0009] According to one aspect of the embodiment of the present specification, the intensity of the reference image is corrected by using the intensity correction value of finite element shape function and finite element node, which includes: the reference image after initial motion correction is gridded, and a plurality of finite element grids of the reference image are obtained;The brightness correction value and contrast correction value of each node in each finite element grid are calculated, and the intensity of the reference image is corrected to reduce the intensity difference between the reference image and the deformation image;

[0010] wherein b(x) represents a whole brightness correction field of the reference image, c(x) represents a whole contrast correction field; θ k (x) represents a finite element shape function corresponding to the kth node, b k represents a brightness correction value of the kth finite element node, c k represents a contrast correction value of the kth finite element node.

[0011] According to an aspect of an embodiment of the present specification, the image residual function constrained by the mechanical regularization term is determined by the following formula: wherein, represents the image residual function, ω m represents a weight of the applied mechanical regularization term, and the value thereof is proportional to l reg 4 , l reg represents a regularization length, represents a motion residual between the reference image and the deformed image; represents a mechanical regularization term of the image; x represents a coordinate of a spatial position, I0(x) represents a pixel value in the reference image, I n (x) represents a pixel value in the deformed image, u(x, {v}) represents a displacement field for moving each pixel in the deformed image I n (x) to a corresponding position in the reference image I0(x), and I n (x+u(x, {v})) represents the deformed image corrected by the displacement field u(x, {v}).

[0012] According to an aspect of an embodiment of the present specification, the weight of the mechanical regularization term is iteratively adjusted, the motion correction is performed on the deformed image to optimize the displacement field, the intensity correction is performed on the first corrected reference image based on the optimized displacement field, and the above operations are alternately repeated until the image residual function constrained by the mechanical regularization term converges to a preset value, which includes: reducing the weight value of the mechanical regularization term, performing the motion correction on the deformed image based on the reduced weight value to obtain an updated displacement field; performing the intensity correction on the first corrected reference image according to the updated displacement field to obtain a second corrected reference image; calculating the value of the image residual function, and determining whether the value of the image residual function converges to the preset value; if yes, determining that the deformed image is corrected, and recording the weight value of the mechanical regularization term; if no, repeating the steps of reducing the weight value of the mechanical regularization term and performing the intensity correction on the second corrected reference image until the value of the image residual function converges to the preset value.

[0013] According to an aspect of the embodiments of the present specification, the determining of the reference image of each deformed image in the sequence of deformed images comprises: calculating the structural similarity of each image in the sequence of magnetic resonance images and the first frame image; grouping the images with the structural similarity higher than a first preset threshold value into a first image sequence group; grouping the images with the structural similarity lower than a second preset threshold value into a second image sequence group; taking the first frame image in the first image sequence group as the reference image of each deformed image in the first image sequence group; and taking the previous frame deformed image of each deformed image in the second image sequence group as the reference image of the deformed image.

[0014] According to an aspect of the embodiments of the present specification, when the deformed image belongs to the second image sequence group, the determining of the real displacement of each deformed image relative to the reference image comprises: determining the real displacement of the last frame image in the first image sequence group relative to the reference image, and taking the displacement as a first displacement; determining the real displacement of all deformed images before the deformed image in the second image group relative to the reference image using a forward accumulation method, and taking the real displacement as a second displacement; and adding the first displacement and the second displacement to obtain the real displacement of each deformed image in the second image sequence group relative to the reference image.

[0015] The embodiments of the present specification provide an image sequence motion correction device based on intensity correction optical flow registration, the device comprising: a reference image determination unit configured to determine a reference image of each deformed image in a sequence of deformed images, wherein the sequence of deformed images comprises a plurality of frame deformed images caused by organ tissue movement; a motion correction unit configured to correct the motion of the deformed image by adjusting the weight of the mechanical regularization term of the reference image and the deformed image to obtain the displacement field of the deformed image relative to the reference image, wherein the mechanical regularization term is constructed based on the principle of mechanical equilibrium and organ tissue movement; an intensity correction unit configured to correct the intensity of the reference image based on the displacement field using the intensity correction value of the finite element shape function and the finite element node to obtain a first corrected reference image; an iterative adjustment unit configured to iteratively adjust the weight of the mechanical regularization term to correct the motion of the deformed image to optimize the displacement field, correct the intensity of the first corrected reference image based on the optimized displacement field, and alternately repeat the execution until the image residual function constrained by the mechanical regularization term converges to a preset value, the deformed image is determined to be corrected, and the optimal solution of the weight value of the mechanical regularization term of the deformed image is recorded; and a real displacement determination unit configured to determine the real displacement of each deformed image relative to the reference image according to the optimal solution of the weight value.

[0016] The embodiment of the present specification provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the image sequence motion correction method based on intensity correction optical flow registration when executing the computer program.

[0017] The embodiment of the present specification also provides a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the image sequence motion correction method based on intensity correction optical flow registration.

[0018] The embodiment of the present specification combines the physiological motion of human tissues or organs (for example, heartbeat, respiration and posture change) to form a mechanical regularization term, and by incorporating the mechanical equilibrium condition into the image registration algorithm as an additional constraint, not only reduces the image noise, but also eliminates the motion component that violates the physical law, significantly improves the accuracy and interpretability of the algorithm. The embodiment of the present specification also introduces an intensity correction technique, which can effectively deal with the severe intensity changes in the image. It effectively reduces or eliminates the intensity distortion caused by various factors by adjusting the brightness and contrast of the image, solving the problem caused by the coupling of image intensity change and motion. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor. In the drawings:

[0020] Fig. 1 shows a flowchart of an image sequence motion correction method based on intensity correction optical flow registration according to an embodiment of the present specification;

[0021] Fig. 2 shows a flowchart of a method for determining the mechanical regularization term of an image according to an embodiment of the present specification;

[0022] Fig. 3 shows a flowchart of a method for performing motion correction according to an embodiment of the present specification;

[0023] Fig. 4 shows a flowchart of a method for intensity correction of a reference image according to an embodiment of the present specification;

[0024] Fig. 5 shows a flowchart of a method for iterative motion correction and intensity correction according to an embodiment of the present specification;

[0025] Fig. 6 shows a flowchart of a method for determining a reference image of each deformed image according to an embodiment of the present specification;

[0026] Fig. 7 shows a flow chart of a method for determining the true displacement of a deformed image relative to a reference image according to an embodiment of the present specification;

[0027] Fig. 8 shows an image sequence motion correction device based on intensity corrected optical flow registration according to an embodiment of the present specification;

[0028] Fig. 9 shows a schematic diagram of CEST MRI image sequence motion correction based on intensity corrected optical flow registration according to an embodiment of the present specification;

[0029] Fig. 10 shows a structural schematic diagram of a computer device according to an embodiment of the present specification.

[0030] BRIEF DESCRIPTION OF THE DRAWINGS 801, reference image determining unit; 802, motion correction unit; 803, intensity correction unit; 804, iterative adjustment unit; 805, true displacement determining unit; 1002, computer device; 1004, processor; 1006, memory; 1008, driving mechanism; 1010, input / output module; 1012, input device; 1014, output device; 1016, presentation device; 1018, graphical user interface; 1020, network interface; 1022, communication link; 1024, communication bus. DETAILED DESCRIPTION

[0031] In order to make the technical personnel in the technical field better understand the technical solutions in the specification, the technical solutions in the specification will be described clearly and completely in the specification below, combined with the drawings in the specification. Obviously, the described embodiments are only part of the embodiments of the specification, not all. Based on the embodiments in the specification, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the specification.

[0032] It should be noted that the terms "first", "second" and the like in the specification and claims of the specification and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the specification described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, device, product or equipment including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or equipment.

[0033] The specification provides method operation steps as described in the embodiments or flow charts, but can include more or fewer operations than shown or described, without departing from the scope of the present disclosure. The steps recited in the embodiments can be executed in any order, and are not limited to the order in which they are listed. The system or apparatus can execute the method in the order of the embodiments or the flow charts, or in parallel.

[0034] It should be noted that the image sequence motion correction method based on intensity correction optical flow registration in the specification can be applied to the field of magnetic resonance imaging, and can also be used in the field of medical technology. The application field of the image sequence motion correction method and device based on intensity correction optical flow registration in the specification is not limited.

[0035] FIG. 1 is a flow chart of an image sequence motion correction method based on intensity correction optical flow registration according to an embodiment of the present specification, which specifically includes the following steps:

[0036] Step 101, determining a reference image for each deformed image in a deformed image sequence, wherein the deformed image sequence includes a plurality of deformed images caused by organ tissue movement.

[0037] In the embodiment of the present specification, CEST MRI sequence will inevitably cause physiological movement and body movement of the human body during a long scanning process. This will cause the images obtained during the scanning process to be deformed relative to the initial images. A plurality of images obtained during the scanning process form an image sequence, and because each of the plurality of images in the image sequence is deformed to a certain extent, the image sequence is also referred to as a deformed image sequence. The deformed image sequence includes a plurality of deformed images caused by organ tissue movement of the human body.

[0038] In the embodiment of the present specification, in order to register each deformed image, a corresponding reference image needs to be determined for each deformed image. The traditional optical flow registration method only fixes to select a certain frame of image as the reference image, but there is a dramatic intensity change between the CEST MRI image sequences, which causes the traditional registration method to be unable to be used between some registered images and reference images. Therefore, the reference image needs to be changed, that is, a corresponding reference image needs to be determined for each deformed image.

[0039] Step 102, performing motion correction on the deformed image by adjusting the weight of the mechanical regularization term of the reference image and the deformed image to obtain the displacement field of the deformed image relative to the reference image, wherein the mechanical regularization term is constructed based on the principle of mechanical equilibrium.

[0040] In the embodiments of the present specification, considering the physical deformation characteristics of human tissues or organs, the mechanical equilibrium in the finite element biomechanical model is taken as a constraint condition, which can better simulate and understand the deformation observed in the image, and constitute the mechanical regularization term of the reference image and the deformation image. Among them, the corresponding weight is configured for the mechanical regularization term, and the motion correction of the deformation image is performed by adjusting the weight of the mechanical regularization term, to obtain the displacement field of the deformation image relative to the reference image, wherein the displacement field is the motion field.

[0041] In step 103, based on the displacement field, the intensity of the reference image is corrected using the finite element shape function and the intensity correction value of the finite element node, to obtain the first corrected reference image.

[0042] In the embodiments of the present specification, due to the drastic intensity change of the CEST image sequence, for example, the reference image in the CEST image sequence has high brightness, and the deformation image is usually dark. Therefore, after the coarse registration of the deformation image in the CEST image sequence, the intensity of the reference image needs to be corrected (i.e., the brightness and contrast are corrected), and the displacement field of the deformation image and the reference image is kept unchanged, to further reduce the residual error between the reference image and the deformation image

[0043] In step 104, the weight of the mechanical regularization term is iteratively adjusted to perform motion correction on the deformation image, to optimize the displacement field. Based on the optimized displacement field, the intensity of the first corrected reference image is corrected, and the alternating and repeated execution is performed until the image residual error function constrained by the mechanical regularization term converges to a preset value, the deformation image is determined to be corrected, and the optimal solution of the weight value of the mechanical regularization term of the deformation image is recorded.

[0044] Based on step 102 and step 103, after the coarse registration of the deformation image based on the reference image, the intensity of the corrected reference image is corrected, the displacement field of the coarse registration deformation image based on the reference image is obtained, and the intensity of the corrected reference image is obtained. Based on the displacement field and the corrected reference image, the weight of the mechanical regularization term is iteratively adjusted, the proportion occupied in the mechanical regularization term image residual error function is reduced, and the intensity of the reference image is further adjusted. The motion correction and the intensity correction are alternately iterated until the image residual error function finally converges.

[0045] In the embodiments of the present specification, the biomechanical model of the physiological motion (such as heartbeat, respiration and posture change) of the human tissues or organs is combined with the mechanical regularization term. By incorporating the mechanical equilibrium condition into the image registration algorithm as an additional constraint, this method not only effectively reduces the image noise, but also excludes the motion component that violates the physical law, and significantly improves the accuracy and interpretability of the algorithm.

[0046] Step 105, according to the weight value optimal solution, determine the real displacement of each deformed image relative to the reference image. When the image residual function converges, the weight value of the corresponding mechanical regularization term is the weight value optimal solution. According to the optimal solution, the mechanical regularization term can be determined. Further determine the rectangular stiffness matrix and node displacement constituting the mechanical regularization term, so as to determine the real displacement of the deformed image relative to the reference image.

[0047] The mechanical model behind the physiological motion and the strength correction technique, the image registration algorithm reduces the image noise, and improves the algorithm accuracy.

[0048] Figure 2 shows a method for determining the mechanical regularization term of an image according to an embodiment of the present specification, which specifically includes the following steps:

[0049] Step 201, grid the reference image to obtain a plurality of finite element grids of the reference image. In the embodiment of the present specification, each frame of reference image is gridded, so that each frame of reference image can be divided into a plurality of finite element units.

[0050] Step 202, according to the mechanical equilibrium principle of organ tissue deformation and any internal node of the finite element grid, construct the mechanical regularization term of the reference image and the deformed image.

[0051] In this step, the mechanical equilibrium principle of the internal node of the finite element grid is that the total force of any internal node of the finite element grid must be 0, that is, F = [K] {u} = 0. Wherein, [K] represents the rectangular stiffness matrix of the finite element grid and material parameters, and {u} represents the required node displacement. However, under normal circumstances, due to the influence of noise and other factors, [K] {u} - F is not equal to 0. Based on this principle, the formula for constructing the mechanical regularization term of the deformed image is as follows:

[0052] Wherein, [K] represents the rectangular stiffness matrix of the finite element grid and material parameters, and {u} represents the required node displacement, represents the mechanical regularization term of the deformed image. Wherein, the material parameters can be Young's modulus or Poisson's ratio, etc. In the present specification, the product of the rectangular stiffness matrix of the expected finite element grid and material parameters and the node displacement is expected to obtain the minimum value.

[0053] Figure 3 shows a method for motion correction according to an embodiment of the present specification, which specifically includes the following steps:

[0054] Step 301, determine the initial weight of the mechanical regularization term as a first numerical value.

[0055] In the embodiments of this specification, the final residual function between the reference image and the deformed image is determined based on the residual function between the reference image and the deformed image and the mechanical regularization term of the deformed image in the traditional optical flow registration method. This residual function can also be referred to as the final residual function between the reference image and the deformed image. In this residual function, the mechanical regularization term has a corresponding adjustable weight parameter.

[0056] When motion correction is initially performed on the deformed image based on the reference image, the initial weight of the mechanical regularization term is adjusted to a first value, where the first value is a large value. In this case, the mechanical considerations of tissues and organs account for a large proportion of the final residual function between the reference image and the deformed image.

[0057] In the embodiments described in this specification, the weight of the mechanical regularization term can be determined by ω. m This indicates that its value is related to the regularization length l. reg 4 Proportional. In subsequent processing steps of this specification, the regularization length l is adjusted from large to small. reg 4 The value can be used to perform motion correction on deformed images from coarse to fine.

[0058] In the embodiments of this specification, the image residual function constrained by the mechanical regularization term is determined by the following formula:

[0059] in, Represents the image residual function, ω m This represents the weight of the applied mechanical regularization term, and its value is related to l. reg 4 Proportional, l reg Indicates the regularization length. This represents the motion residual between the reference image and the deformed image; This represents the mechanical regularization term of the image.

[0060] In the embodiments described in this specification, the motion residuals of the reference image and the deformed image It is calculated using the following formula:

[0061] x represents the coordinates of the spatial location, I0(x) represents the pixel value in the reference image, and I n (x) represents the pixel value in the deformed image, and u(x,{v}) represents the deformed image I. n Each pixel in (x) is moved to the corresponding position in the reference image I0(x) by a displacement field, I n (x+u(x,{v})) represents the deformed image after correction by the displacement field u(x,{v}).

[0062] To perform global registration, the least squares method is used to evaluate across the entire region of interest (ROI). The value of is shown in the following formula:

[0063] However, using the least squares method to calculate When the minimum value is reached, the problem of no unique solution may arise. Therefore, the finite element method is often used to parameterize the displacement field u(x,{v}) to regularize and stabilize the problem. Under this framework, the registration problem can be transformed into a linear system for solution using the Gauss-Newton method: [M]{δu}={b};

[0064] Where [M] is the Hessian matrix, {b} is the residual vector, and {δu} is the correction for the mesh node displacement during each iteration. Therefore, the registration problem of the deformed image is transformed into the problem of adjusting the displacement of the finite element nodes to minimize the residual vector {b}, thereby finding the optimal displacement field u(x,{v}).

[0065] Step 302: Under the weight of the mechanical regularization term of the first value, perform initial motion correction on the deformed image to obtain the displacement field of the initial motion correction.

[0066] In this step, when the weight of the mechanical regularization term is determined to be the first value, motion correction is performed on the first deformed image of a given intensity in the deformed image sequence using the weight of the first value. Because the first value is relatively large, the mechanical regularization term accounts for a large proportion in the image residual function. Therefore, the image residual function converges quickly, resulting in the coarsely registered motion field {u0}.

[0067] Figure 4 shows a flowchart of a method for intensity correction of a reference image according to an embodiment of this specification, which specifically includes the following steps:

[0068] Step 401: Mesh the reference image to obtain multiple finite element meshes of the reference image.

[0069] Step 402: Calculate the brightness correction value and contrast correction value for each node in each finite element mesh, and perform intensity correction on the reference image to reduce the intensity difference between the reference image and the deformed image. In this specification, the contrast correction value of the reference image is calculated using the following formula:

[0070] Where b(x) represents the overall brightness correction field of the reference image, and c(x) represents the overall contrast correction field of the reference image; θ k (x) represents the finite element shape function corresponding to the k-th node, b ka brightness correction value of the kth finite element node, c k a contrast correction value of the kth finite element node.

[0071] In the embodiments of the present specification, the reference image contains a plurality of nodes, the brightness correction value of the finite element node corresponding to all nodes is multiplied by the finite element shape function to obtain the overall brightness correction value field of the reference image; the contrast correction value of the finite element node corresponding to all nodes in the reference image is multiplied by the finite element shape function to obtain the overall contrast correction field of the reference image. The overall contrast correction value and the overall brightness correction value of the reference image constitute the intensity correction value of the reference image.

[0072] In the embodiments of the present specification, by adjusting the brightness and contrast of the image, the intensity distortion caused by various factors is effectively reduced or eliminated, the problem caused by the coupling of image intensity change and motion is solved, and the stability of the registration algorithm is greatly improved.

[0073] FIG. 5 shows a flowchart of an iterative motion correction and intensity correction method according to an embodiment of the present specification, which specifically includes the following steps:

[0074] Step 501, reduce the weight value of the mechanical regularization term, and correct the deformed image based on the reduced weight value to obtain an updated displacement field.

[0075] This step adjusts the weight value of the mechanical regularization term based on steps 102 and 103. The adjusted weight value of the mechanical regularization term is less than the first value. Based on the same method as step 102, the first reference correction image and the deformed image are corrected using the reduced weight. The displacement field of the registration of the deformed image and the first reference correction image is obtained, which is considered as the update of the displacement field of the deformed image relative to the reference image in step 102.

[0076] Step 502, intensity correction is performed on the first correction reference image according to the updated displacement field to obtain a second correction reference image.

[0077] In the embodiments of the present specification, based on the updated displacement field {u1} obtained in step 501, the motion "seed" is fixed, and the displacement field {u1} is fixed. The first correction reference image is corrected for image intensity to obtain a corrected reference image

[0078] Step 503, calculate the value of the image residual function and determine whether the value of the image residual function converges to a preset value. According to the current updated displacement field {u1} and the rectangular stiffness matrix [K], the value of the mechanical regularization term and the image residual function is calculated, and it is further determined whether the value of the residual function is less than the preset value.

[0079] Step 504, if yes, it is determined that the deformed image is corrected, and the weight value of the mechanical regularization term is recorded.

[0080] Step 505, if no, the steps of reducing the weight value of the mechanical regularization term and intensity correcting the second correction reference image are repeatedly executed until the value of the image residual function converges to a preset value.

[0081] In this step, if the value of the residual function is not less than the preset value, it indicates that the registration between the deformed image and the reference image is not completed. The weight value of the mechanical regularization term is continuously reduced, and the displacement field of the deformed image and the second reference image is updated based on the new weight value. Based on the displacement field, the second correction reference image is intensity corrected to obtain a third correction reference image. In this way, the process of adjusting the weight of the mechanical regularization term, correcting the deformed image based on the reference image according to the weight to update the displacement field, and intensity correcting the reference image based on the updated displacement field is repeatedly executed.

[0082] FIG. 6 shows a method flow chart for determining the reference image of each deformed image according to an embodiment of the present specification, which specifically includes the following steps:

[0083] Step 601, the structural similarity of each image in the sequence of magnetic resonance images and the first frame image is calculated. In the embodiment of the present specification, the brightness and motion of the multiple frames of deformed images in the sequence of CEST MRI deformed images change greatly. Therefore, the same frame of reference image cannot be determined for all deformed images.

[0084] Therefore, the structural similarity (SSIM, denoted as γ) of each image in the sequence of CEST MRI magnetic resonance images and the first frame image in the sequence of images is calculated.

[0085] Step 602, multiple images with a structural similarity higher than a first preset threshold value are grouped into a first image sequence group.

[0086] In this step, according to the size of the calculated structural similarity of the image and the first preset threshold value, the images are divided into different image sequence groups. If the value of the first preset threshold value is set to 0.5, the sequence of magnetic resonance images is divided into two groups according to the first preset threshold value γ cut = 0.5. The deformed images with a first preset threshold value greater than 0.5 are grouped into a first image sequence group. In the embodiment of the present specification, for example, there are 60 frames of deformed images in the sequence of CEST MRI, among which the changes of the second to tenth frames of deformed images are small. The structural similarity of these 10 frames of images and the first frame image is greater than 0.5, and these 10 frames of images form a first image sequence.

[0087] Step 603, the plurality of images with a structure similarity lower than a first preset threshold value are grouped into a second image sequence group. If the value of the first preset threshold value is set to 0.5, the deformed images with a first preset threshold value less than 0.5 are grouped into the second image sequence group. The deformed images in the second image sequence group have a low structure similarity with the first frame image in the image sequence, indicating that the deformed images in the second image sequence group have a relatively sharp and obvious difference change. For example, in a CEST MRI sequence, there are 60 deformed images in total, among which the 20th to 39th deformed images have a sharp change, and the structure similarity of the 20 deformed images with the first frame image is calculated to be less than 0.5, so the 20 deformed images are grouped into a second image sequence.

[0088] Step 604, the first frame image in the first image sequence group is taken as a reference image of each deformed image in the first image sequence group. If γ>0.5, the first frame image I0 in the magnetic resonance image sequence is directly taken as the reference image, and the reference image can be directly correlated with the reference image, which is called direct digital image correlation (D-DIC) analysis.

[0089] Step 605, the previous frame deformed image of each deformed image in the second image sequence group is taken as the reference image of the deformed image. If γ<0.5, incremental digital image correlation (I-DIC) analysis is performed, and the previous image I n-1 is correlated with I n to overcome the sharp intensity change between the deformed images.

[0090] The adaptive reference image selection strategy of the embodiments of the present specification can automatically select the most suitable reference image according to the structure similarity of the image sequence. In addition, the incremental calculation method effectively solves the problem that the reference image and the image to be registered are difficult to match when there is a significant intensity difference.

[0091] FIG. 7 shows a method flowchart for determining the real displacement of a deformed image relative to a reference image according to an embodiment of the present specification, which specifically includes the following steps:

[0092] Step 701, the real displacement of the last frame image in the first image sequence group relative to its reference image is determined, and the displacement is recorded as a first displacement. Assuming that the last frame image is the mth frame image in the first image sequence group, the first displacement is recorded as

[0093] In the embodiments of the present specification, for example, in a CEST MRI sequence, there are 60 deformed images in total, among which the first to 20th deformed images form a first image sequence, and the displacement of the 20th deformed image relative to the reference image is recorded as

[0094] Step 702, using the forward accumulation method, sequentially determine the true displacement of all deformation images before the deformation image in the second image group relative to the reference image, and record the true displacement as the second displacement.

[0095] In the embodiments of the present specification, for example, there are 60 frames of deformation images in the CEST MRI sequence, wherein the 20th to 39th frames of deformation images constitute the second image sequence. Then the displacement of the 20th, 21st, 22nd, …, to 38th images relative to their respective reference images before the 39th deformation image needs to be determined, for example, the displacement of the 21st image relative to the 20th image is recorded as the displacement of the 38th image relative to the 37th image is recorded as Using the forward accumulation method, sequentially add the displacement of all deformation images before the deformation image relative to the reference image, and record the displacement as the second displacement, to obtain the displacement field

[0096] Step 703, add the first displacement and the second displacement to obtain the true displacement of each deformation image in the second image sequence group relative to the reference image.

[0097] Add the first displacement in step 702 and the second displacement in step 702 to obtain the true displacement of a certain frame of deformation image in the second image sequence group relative to the reference image. According to the example above, add the displacement field to the displacement field , that is, the displacement field represents the true displacement of the 38th image in the second image sequence group relative to the first image (reference image) in the CEST MRI sequence.

[0098] The application of the present specification is not limited to CEST MRI, but also applies to dynamic contrast-enhanced (DCE) MRI and positron emission computed tomography (PET-CT) images and the like, which have obvious brightness or contrast changes in the diagnosis process. RI-DIC provides a reasonable physical explanation by integrating the physical model, ensuring the accuracy of motion correction. In the application of DCE-MRI, this method effectively corrects the brightness changes caused by contrast agents in the time sequence, enhances the reliability of the imaging data, and is particularly important for lesion detection and treatment evaluation. Similarly, in the multi-modal imaging of PET-CT, RI-DIC can ensure the accurate alignment between different imaging modes, and improve the matching degree of anatomical and functional images.

[0099] As shown in Figure 8, it is a structural schematic diagram of an image sequence motion correction device based on intensity correction optical flow registration according to an embodiment of the present specification. The basic structure of the image sequence motion correction device based on intensity correction optical flow registration is described in the figure, and the functional units and modules therein can be implemented in software, general chips or specific chips. The device specifically comprises:

[0100] A reference image determination unit 801 is configured to determine a reference image of each deformed image in a deformed image sequence, wherein the deformed image sequence comprises a plurality of deformed images caused by organ tissue movement;

[0101] A motion correction unit 802 is configured to correct the motion of the deformed image by adjusting the weight of the mechanical regularization term of the reference image and the deformed image, to obtain a displacement field of the deformed image relative to the reference image, wherein the mechanical regularization term is constructed based on the principle of mechanical equilibrium and organ tissue movement;

[0102] An intensity correction unit 803 is configured to correct the intensity of the reference image based on the displacement field, using the intensity correction value of the finite element shape function and the finite element node, to obtain a first corrected reference image;

[0103] An iterative adjustment unit 804 is configured to iteratively adjust the weight of the mechanical regularization term, correct the motion of the deformed image, optimize the displacement field, correct the intensity of the first corrected reference image based on the optimized displacement field, and alternately repeat the execution until the image residual function constrained by the mechanical regularization term converges to a preset value, the deformed image is determined to be corrected, and the optimal solution of the weight value of the mechanical regularization term of the deformed image is recorded;

[0104] A real displacement determination unit 805 is configured to determine the real displacement of each deformed image relative to the reference image according to the optimal solution of the weight value.

[0105] The present specification aims to introduce the mechanical regularization term, use the biomechanical model as an additional constraint term, and provide physical interpretability for the deformation of biological tissue organs. The intensity correction technology is introduced to correct the motion and image intensity alternately, thereby solving the influence caused by the mutual coupling of the two and improving the registration accuracy. The adaptive incremental calculation method is used to optimize the reference image selection strategy, further improve the correlation of the images, and the motion correction accuracy.

[0106] FIG. 9 shows a schematic diagram of a CEST MRI image sequence motion correction based on intensity corrected optical flow registration according to an embodiment of the present specification. The differences between 9 images are shown, and the images are divided into two groups according to the structural similarity of each image to the first image, which are D-DIC group and I-DIC group. The D-DIC group and I-DIC group image sequences are corrected respectively, and the motion field of the deformed image relative to the reference image is calculated, and then the reference image is further corrected.

[0107] As shown in FIG. 10, a schematic diagram of a computer device 1002 is provided according to an embodiment of the present specification. The computer device 1002 can include one or more processors 1004, such as one or more central processing units (CPUs), each of which can implement one or more hardware threads. The computer device 1002 can also include any memory 1006 for storing any kind of information, such as code, settings, data, etc. Without limitation, for example, the memory 1006 can include any one or combination of the following: any type of RAM, any type of ROM, a flash memory device, a hard disk, an optical disk, etc. More generally, any memory can use any technology for storing information. Further, any memory can provide volatile or non-volatile retention of information. Further, any memory can represent a fixed or removable component of the computer device 1002. In one case, the computer device 1002 can perform any operation of the associated instructions when executed by the processor 1004, which are stored in any memory or combination of memories. The computer device 1002 also includes one or more drive mechanisms 1008, such as a hard disk drive mechanism, an optical disk drive mechanism, etc., for interacting with any memory.

[0108] The computer device 1002 can also include an input / output module 1010 (I / O) for receiving various inputs (via input devices 1012) and for providing various outputs (via output devices 1014). One particular output mechanism can include a presentation device 1016 and an associated graphical user interface (GUI) 1018. In other embodiments, the input / output module 1010 (I / O), input devices 1012, and output devices 1014 can also not be included, and the computer device 1002 can simply be a computer device in a network. The computer device 1002 can also include one or more network interfaces 1020 for exchanging data with other devices via one or more communication links 1022. One or more communication buses 1024 couple the above-described components together.

[0109] The communication links 1022 can be implemented in any manner, such as through a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. The communication links 1022 can include any combination of hardwired links, wireless links, routers, gateway functionality, name servers, etc., governed by any protocol or combination of protocols.

[0110] Corresponding to the method in FIG. 1 to FIG. 7, the embodiment of the present specification also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to perform the steps of the above method.

[0111] The embodiment of the present specification also provides a computer readable instruction, wherein when the processor executes the instruction, the program therein causes the processor to perform the method as shown in FIG. 1 to FIG. 7.

[0112] It should be understood that the size of the sequence number of each process described above in various embodiments of the present specification does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present specification.

[0113] It should also be understood that in the embodiment of the present specification, the term "and / or" only describes the association relationship of the associated objects, which means that there can be three relationships. For example, A and / or B can represent three cases of A alone, A and B together, and B alone. In addition, the character " / " in the present specification generally represents an "or" relationship between the associated objects before and after it.

[0114] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present specification can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been described in the above description in general terms. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present specification.

[0115] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0116] In several embodiments provided in the specification, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and the division of the units is merely a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can also be electrical, mechanical or other forms of connection.

[0117] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the specification.

[0118] In addition, the functional units in each embodiment of the specification can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0119] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the specification essentially or the part of the prior art that contributes to the technical solutions, or all or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the specification. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0120] The principles and implementation manners of the specification are described in the specific embodiments in the specification, and the above embodiment descriptions are only used to help understand the method and its core idea of the specification; at the same time, for those skilled in the art, according to the idea of the specification, the specific implementation manner and application range will be changed, and the above-mentioned content of the specification should not be understood as a limitation of the specification.

Claims

1. A method for motion correction of a sequence of images based on intensity- corrected optical flow registration, characterized in that, The method comprises: determining a reference image of each deformed image in a sequence of deformed images, wherein the sequence of deformed images comprises a plurality of frames of deformed images caused by organ tissue movement; performing motion correction on the deformed image by adjusting a weight of a mechanical regularization term of the reference image and the deformed image to obtain a displacement field of the deformed image relative to the reference image, wherein the mechanical regularization term is constructed based on a mechanical equilibrium principle; performing intensity correction on the reference image based on the displacement field using a finite element shape function and an intensity correction value of a finite element node to obtain a first corrected reference image; iteratively adjusting the weight of the mechanical regularization term to perform motion correction on the deformed image to optimize the displacement field, performing intensity correction on the first corrected reference image based on the optimized displacement field, and alternately repeating the execution until an image residual function constrained by the mechanical regularization term converges to a preset value, determining that the deformed image is corrected, and recording an optimal solution of the weight value of the mechanical regularization term; determining the true displacement of each deformed image relative to the reference image according to the optimal solution of the weight value.

2. The method of claim 1, wherein, Before adjusting the weight of the mechanical regularization term of the deformed image, the mechanical regularization term of the image is determined in the following manner: griding the reference image to obtain a plurality of finite element grids of the reference image; constructing the mechanical regularization term of the reference image and the deformed image according to the mechanical equilibrium principle of organ tissue deformation and nodes inside an arbitrary finite element grid, as shown in the following formula: where [K] represents a rectangular stiffness matrix of finite element mesh and material parameters, {u} represents the required node displacement, denotes the mechanical regularization term of the deformed image.

3. The method of claim 1, wherein, performing motion correction on the deformed image and its reference image by adjusting the weight of the mechanical regularization term of the deformed image comprises: determining an initial weight of the mechanical regularization term as a first numerical value; performing initial motion correction on the deformed image under the condition of the weight of the mechanical regularization term being the first numerical value to obtain an initial motion corrected displacement field.

4. The method of claim 3, wherein, performing intensity correction on the reference image using a finite element shape function and an intensity correction value of a finite element node comprises: griding the reference image to obtain a plurality of finite element grids of the reference image; The intensity correction is performed on the reference image by calculating the brightness correction value and the contrast correction value of each node in each finite element grid, so as to reduce the intensity difference between the reference image and the deformed image. Wherein, b(x) represents the overall brightness correction field of the reference image, c(x) represents the overall contrast correction field; θ k (x) represents the finite element shape function corresponding to the kth node, b k represents the brightness correction value of the kth finite element node, c k represents the contrast correction value of the kth finite element node.

5. The method of claim 4, wherein, determining an image residual function constrained by the mechanical regularization term in the following formula: wherein denotes the image residual function, ω m denotes the weight of the applied mechanical regularization term, whose value is proportional to l reg 4 denotes the weight of the applied mechanical regularization term, whose value is proportional to l reg denotes the regularization length, representing the motion residual of the reference image and the warped image; denotes the mechanical regularization term of the image; x denotes a coordinate of a spatial position, I0(x) denotes a pixel value in a reference image, I n (x) denotes a pixel value in a deformed image, u(x, {v}) denotes a displacement field that moves each pixel in the deformed image I n (x) to a corresponding position in the reference image I0(x), I n (x + u(x, {v})) denotes the deformed image corrected by the displacement field u(x, {v}).

6. The method of claim 1, wherein, iteratively adjusting the weight of the mechanical regularization term to perform motion correction on the deformed image to optimize the displacement field, performing intensity correction on the first corrected reference image based on the optimized displacement field, and alternately repeating the execution until the image residual function constrained by the mechanical regularization term converges to a preset value comprises: reducing the weight value of the mechanical regularization term, performing motion correction on the deformed image based on the reduced weight value to obtain an updated displacement field; performing intensity correction on the first corrected reference image according to the updated displacement field to obtain a second corrected reference image; calculating the value of the image residual function and determining whether the value of the image residual function converges to the preset value; if yes, determining that the deformed image is corrected and recording the weight value of the mechanical regularization term; if no, repeatedly performing the steps of reducing the weight value of the mechanical regularization term and performing intensity correction on the second corrected reference image until the value of the image residual function converges to the preset value.

7. The method of claim 1, wherein, determining a reference image of each deformed image in a sequence of deformed images comprises: calculate a structural similarity of each image in the sequence of magnetic resonance images to the first frame image; group a plurality of images with a structural similarity higher than a first preset threshold into a first image sequence group; group a plurality of images with a structural similarity lower than the first preset threshold into a second image sequence group; use the first frame image in the first image sequence group as a reference image of each deformed image in the first image sequence group; use a previous deformed image of each deformed image in the second image sequence group as a reference image of the deformed image.

8. The method of claim 7, wherein, when the deformed image belongs to the second image sequence group, determining the real displacement of each deformed image relative to the reference image comprises: determining a real displacement of a last frame image in the first image sequence group relative to its reference image, and recording the displacement as a first displacement; using a forward accumulation method to sequentially determine real displacements of all deformed images before the deformed image in the second image group relative to the reference image, and recording the real displacements as second displacements; adding the first displacement and the second displacement to obtain the real displacement of each deformed image in the second image sequence group relative to the reference image.

9. An apparatus for motion correction of a sequence of images based on intensity corrected optical flow registration, characterized in that, The device comprises: a reference image determination unit configured to determine a reference image of each deformed image in a deformed image sequence, wherein the deformed image sequence comprises a plurality of deformed images caused by organ tissue movement; a motion correction unit configured to correct the motion of the deformed image by adjusting a weight of a mechanical regularization term of the reference image and the deformed image to obtain a displacement field of the deformed image relative to the reference image, wherein the mechanical regularization term is constructed based on a mechanical equilibrium principle and organ tissue movement; an intensity correction unit configured to correct the intensity of the reference image based on the displacement field using a finite element shape function and an intensity correction value of a finite element node to obtain a first corrected reference image; an iterative adjustment unit configured to iteratively adjust the weight of the mechanical regularization term to correct the motion of the deformed image to optimize the displacement field, correct the intensity of the first corrected reference image based on the optimized displacement field, and alternately repeat the execution until an image residual function constrained by the mechanical regularization term converges to a preset value, determine that the correction of the deformed image is completed, and record an optimal solution of the weight value of the mechanical regularization term of the deformed image; a real displacement determination unit configured to determine a real displacement of each deformed image relative to the reference image according to the optimal solution of the weight value.

10. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method of any one of claims 1 to 8.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method of any one of claims 1 to 8.

Images