Depth camera combined magnetic resonance image distortion correction method and device

Abstract

The invention discloses a magnetic resonance image distortion correction method and device combined with a depth camera, and belongs to the field of image processing. The magnetic resonance image distortion correction method combined with the depth camera comprises the following steps: processing depth data corresponding to a to-be-measured object collected by the depth camera, and obtaining a first point cloud model corresponding to the to-be-measured object; acquiring a first magnetic resonance image corresponding to the to-be-detected object acquired by the magnetic resonance equipment in the target scanning space; processing the first magnetic resonance image and the first point cloud model to obtain a first control point set corresponding to the first magnetic resonance image and a second control point set corresponding to the first point cloud model; obtaining a target transformation function between the first magnetic resonance image and the first point cloud model based on the first control point set and the second control point set; and correcting the first magnetic resonance image based on the target transformation function to obtain a target magnetic resonance image corresponding to the to-be-detected object. The magnetic resonance image distortion correction method combined with the depth camera is good in correction effect.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a method and device for correcting magnetic resonance image distortion in combination with a depth camera. Background Art

[0002] Magnetic Resonance Imaging (MRI), as a non-invasive medical imaging technology, is widely used in clinical diagnosis and research due to its superior soft tissue contrast and multi-directional imaging capabilities. However, in the traditional MRI scanning process, geometric distortion of the imaging image is often caused by factors such as magnetic field inhomogeneity, insufficient linearity of the gradient magnetic field, or improper system parameter settings. These distortions not only affect the visual quality of the image, but may also have a negative impact on the accuracy of clinical diagnosis. There are correction methods through geometric correction or machine learning in related technologies, but the commonly used correction methods are complex to operate and have high time costs, and it is difficult to achieve high-precision correction, and the correction effect is not good. Summary of the invention

[0003] The present invention aims to solve at least one of the technical problems existing in the related art. To this end, the present invention proposes a method and device for correcting magnetic resonance image distortion in combination with a depth camera, which improves the convenience of operation, shortens the scanning preparation time, thereby improving the efficiency of clinical work, and improves the adaptability of the correction method under different equipment and scanning conditions, and has a good correction effect.

[0004] In a first aspect, the present invention provides a method for correcting magnetic resonance image distortion in combination with a depth camera, the method comprising: Processing the depth data corresponding to the object to be measured acquired by the depth camera to obtain a first point cloud model corresponding to the object to be measured; obtaining a first magnetic resonance image corresponding to the object to be measured acquired by the magnetic resonance device in a target scanning space; the first point cloud model and the first magnetic resonance image are in the same coordinate system; Processing the first magnetic resonance image and the first point cloud model to obtain a first control point set corresponding to the first magnetic resonance image and a second control point set corresponding to the first point cloud model, wherein a plurality of first control points in the first control point set correspond one-to-one to a plurality of second control points in the second control point set; acquiring a target transformation function between the first magnetic resonance image and the first point cloud model based on the first control point set and the second control point set; The first magnetic resonance image is corrected based on the target transformation function to obtain a target magnetic resonance image corresponding to the object to be measured.

[0005] According to the magnetic resonance image distortion correction method combined with a depth camera provided by an embodiment of the present invention, the distortion correction is performed by using the real three-dimensional model of the head of the object to be measured (the first point cloud model) obtained by the depth camera, without the need to use a calibration model, thereby improving the convenience of operation, shortening the scanning preparation time, and thus improving the efficiency of clinical work; by directly comparing the real three-dimensional model reconstructed by the depth camera to obtain the transformation function between the first magnetic resonance image to be corrected and the real three-dimensional model, the first magnetic resonance image is corrected based on the transformation function, reducing the dependence on a fixed reference image, improving the adaptability of the correction method under different devices and scanning conditions, and achieving a better correction effect.

[0006] According to an embodiment of the present invention, a method for correcting magnetic resonance image distortion in combination with a depth camera, the depth data corresponding to the object to be measured acquired by the depth camera is processed to obtain a first point cloud model corresponding to the object to be measured, including: Acquire first depth data corresponding to the object to be measured collected by the depth camera at multiple angles before the object to be measured enters the target scanning space, and second depth data corresponding to the object to be measured collected by the depth camera in the target scanning space; Using a point cloud processing algorithm to process the first depth data, and obtaining a three-dimensional head model corresponding to the first depth data; The second depth data and the three-dimensional head model are aligned to obtain the first point cloud model.

[0007] According to an embodiment of the present invention, a method for correcting magnetic resonance image distortion in combination with a depth camera, aligning the second depth data with the three-dimensional head model to obtain the first point cloud model includes: Acquire at least one first feature point in the second depth data; Acquire at least one second feature point matching the at least one first feature point from the three-dimensional head model; Using an iterative closest point algorithm to process the at least one first feature point and the second feature point to obtain a target registration parameter between the second depth data and the three-dimensional head model; processing the second depth data based on the target registration parameters to align the second depth data to the three-dimensional head model; The first point cloud model is acquired based on the aligned second depth data and the three-dimensional head model.

[0008] In a method for correcting magnetic resonance image distortion in combination with a depth camera according to an embodiment of the present invention, the first magnetic resonance image and the first point cloud model are processed to obtain a first control point set corresponding to the first magnetic resonance image and a second control point set corresponding to the first point cloud model, including: Performing contour extraction processing on the first magnetic resonance image and the first point cloud model respectively to obtain first contour information corresponding to the first magnetic resonance image and second contour information corresponding to the first point cloud model; The first control point set and the second control point set are acquired based on first difference information between the first contour information and the second contour information.

[0009] In a method for correcting magnetic resonance image distortion in combination with a depth camera according to an embodiment of the present invention, obtaining a target transformation function between the first magnetic resonance image and the first point cloud model based on the first control point set and the second control point set includes: Based on the formula:

[0010] Get the target transformation function, where is the target transformation function, and are the affine transformation parameters, is the position information of the target control point in the first control point set, is the nonlinear transformation weight, is the position information of the control point corresponding to the target control point in the second control point set, is the radial basis function, is an index variable used to traverse all control points in the second control point set, is an integer in the range [1, N], where N is the total number of control points in the second control point set.

[0011] According to an embodiment of the present invention, a method for correcting magnetic resonance image distortion in combination with a depth camera, the first magnetic resonance image is corrected based on the target transformation function to obtain a target magnetic resonance image corresponding to the object to be measured, including: Correcting the first magnetic resonance image based on the target transformation function to obtain an initial magnetic resonance image corresponding to the object to be measured; Acquiring second difference information between the initial magnetic resonance image and the first point cloud model; When the second difference information is smaller than a target difference threshold, the initial magnetic resonance image is determined as the target magnetic resonance image.

[0012] In a second aspect, the present invention provides a magnetic resonance image distortion correction device combined with a depth camera, comprising: A first processing module is used to process the depth data corresponding to the object to be measured acquired by the depth camera to obtain a first point cloud model corresponding to the object to be measured; and to obtain a first magnetic resonance image corresponding to the object to be measured acquired by the magnetic resonance device in a target scanning space; the first point cloud model and the first magnetic resonance image are in the same coordinate system; a second processing module, configured to process the first magnetic resonance image and the first point cloud model to obtain a first control point set corresponding to the first magnetic resonance image and a second control point set corresponding to the first point cloud model, wherein a plurality of first control points in the first control point set correspond one-to-one to a plurality of second control points in the second control point set; a third processing module, configured to acquire a target transformation function between the first magnetic resonance image and the first point cloud model based on the first control point set and the second control point set; The fourth processing module is used to correct the first magnetic resonance image based on the target transformation function to obtain a target magnetic resonance image corresponding to the object to be measured.

[0013] According to the magnetic resonance image distortion correction device combined with a depth camera provided by an embodiment of the present invention, the distortion correction is performed by using the real three-dimensional model of the head of the object to be measured (the first point cloud model) obtained by the depth camera, without the need to use a calibration model, thereby improving the convenience of operation, shortening the scanning preparation time, and thus improving the efficiency of clinical work; by directly comparing the real three-dimensional model reconstructed by the depth camera to obtain the transformation function between the first magnetic resonance image to be corrected and the real three-dimensional model, the first magnetic resonance image is corrected based on the transformation function, reducing the dependence on a fixed reference image, improving the adaptability of the correction method under different devices and scanning conditions, and achieving a better correction effect.

[0014] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for correcting magnetic resonance image distortion in combination with a depth camera as described in the first aspect above is implemented.

[0015] In a fourth aspect, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for correcting magnetic resonance image distortion in combination with a depth camera as described in the first aspect above is implemented.

[0016] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the magnetic resonance image distortion correction method combined with a depth camera as described in the first aspect above.

[0017] The above one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects: By using the real three-dimensional model of the head of the object to be tested obtained by the depth camera (the first point cloud model) for distortion correction, there is no need to use a calibration model, which improves the convenience of operation, shortens the scanning preparation time, and thus improves the efficiency of clinical work; by directly comparing the real three-dimensional model reconstructed by the depth camera to obtain the transformation function between the first magnetic resonance image to be corrected and the real three-dimensional model, the first magnetic resonance image is corrected based on the transformation function, which reduces the dependence on fixed reference images, improves the adaptability of the correction method under different equipment and scanning conditions, and has a better correction effect.

[0018] Furthermore, by adopting the thin plate spline method in combination with the first point cloud model provided by the depth camera, complex geometric distortions in the first magnetic resonance image can be accurately captured and corrected, and the correction effect is good.

[0019] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which: Figure 1 This is one of the flow charts of the magnetic resonance image distortion correction method combined with a depth camera provided in an embodiment of the present invention; Figure 2 This is one of the principle schematic diagrams of the magnetic resonance image distortion correction method combined with a depth camera provided in an embodiment of the present invention; Figure 3 This is a second flow chart of a method for correcting magnetic resonance image distortion in combination with a depth camera provided in an embodiment of the present invention; Figure 4 This is the second principle schematic diagram of the magnetic resonance image distortion correction method combined with a depth camera provided in an embodiment of the present invention; Figure 5 This is the third principle schematic diagram of the magnetic resonance image distortion correction method combined with a depth camera provided in an embodiment of the present invention; Figure 6 This is the fourth principle schematic diagram of the magnetic resonance image distortion correction method combined with a depth camera provided in an embodiment of the present invention; Figure 7This is the fifth principle schematic diagram of the magnetic resonance image distortion correction method combined with a depth camera provided in an embodiment of the present invention; Figure 8 is a schematic diagram of the result of a magnetic resonance image distortion correction method combined with a depth camera provided in an embodiment of the present invention; Fig. 9 is a structural schematic diagram of a magnetic resonance image distortion correction device combined with a depth camera provided in an embodiment of the present invention; Fig.10 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

[0021] Reference numerals: First processing module 910; second processing module 920; third processing module 930; fourth processing module 940; Electronic device 1000; processor 1001; memory 1002. DETAILED DESCRIPTION

[0022] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0023] The terms "first", "second", etc. in the specification and claims of the present invention are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0024] In conjunction with the accompanying drawings, the magnetic resonance image distortion correction method combined with a depth camera, the magnetic resonance image distortion correction device combined with a depth camera, the electronic device and the readable storage medium provided in the embodiments of the present invention are described in detail through specific embodiments and their application scenarios.

[0025] Among them, the magnetic resonance image distortion correction method combined with the depth camera can be applied to the terminal, and can be specifically executed by hardware or software in the terminal.

[0026] The terminal includes, but is not limited to, a portable communication device such as a mobile phone or a tablet computer having a touch-sensitive surface (e.g., a touch screen display and / or a touch pad). It should also be understood that in some embodiments, the terminal may not be a portable communication device, but a desktop computer having a touch-sensitive surface (e.g., a touch screen display and / or a touch pad).

[0027] In the following various embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, a mouse and a joystick.

[0028] An embodiment of the present invention provides a method for correcting the distortion of a magnetic resonance image combined with a depth camera. The execution subject of the method for correcting the distortion of a magnetic resonance image combined with a depth camera may be an electronic device or a functional module or functional entity in the electronic device that can implement the method for correcting the distortion of a magnetic resonance image combined with a depth camera. The electronic devices mentioned in the embodiment of the present invention include but are not limited to mobile phones, tablet computers, computers, cameras, and wearable devices. The following takes the electronic device as an example of the execution subject to illustrate the method for correcting the distortion of a magnetic resonance image combined with a depth camera provided in an embodiment of the present invention.

[0029] like Figure 1 As shown, the magnetic resonance image distortion correction method combined with a depth camera includes: step 110, step 120, step 130 and step 140.

[0030] Step 110: Process the depth data corresponding to the object to be measured acquired by the depth camera to obtain a first point cloud model corresponding to the object to be measured; obtain a first magnetic resonance image corresponding to the object to be measured acquired by the magnetic resonance device in the target scanning space; In this step, the depth camera can obtain the distance information between the object and the camera, which can be used to generate a 3D scene.

[0031] The depth camera emits infrared structured light or laser and receives the signal reflected by the object. It calculates the distance between the object and the sensor based on the time difference or intensity difference of the signal. This distance information is recorded to form depth data.

[0032] Each pixel in the depth data is used to represent the actual distance of the point from the depth camera.

[0033] The depth camera can capture images of the head or face of the object to be measured, and obtain the head data or face data of the object to be measured.

[0034] The head data of the object to be tested can be used to represent the complete head information of the object to be tested, and the facial data can be used to represent part of the head or facial information of the object to be tested.

[0035] The object to be detected is a target entity that needs to be examined by magnetic resonance imaging, for example, a human body.

[0036] The first point cloud model is used to represent the complete head information of the object to be measured in the coordinate system corresponding to the magnetic resonance device.

[0037] Before and after the object to be measured enters the target scanning space, a depth camera can be used to capture images of the object to be measured to obtain complete depth data corresponding to the object to be measured and depth data in the coordinate system corresponding to the magnetic resonance device, and then a first point cloud model is obtained based on the two depth data.

[0038] The target scanning space may be a scanning space of a magnetic resonance device.

[0039] like Figure 2 As shown, when it is necessary to perform a magnetic resonance examination on the object to be tested, the object to be tested can lie flat on the testing table, and the depth camera can be set directly above or diagonally above the face of the object to be tested. The present invention is not limited to this. The head of the object to be tested can be completely within the target scanning space of the magnetic resonance equipment, and the head can be wrapped by an open coil.

[0040] The first magnetic resonance image may be a two-dimensional or three-dimensional image, which may be selected based on user needs.

[0041] The first point cloud model and the first magnetic resonance image are in the same coordinate system, that is, the first point cloud model and the first magnetic resonance image may have position coordinate information in a scanning coordinate system corresponding to the magnetic resonance device.

[0042] Step 120: Process the first magnetic resonance image and the first point cloud model to obtain a first control point set corresponding to the first magnetic resonance image and a second control point set corresponding to the first point cloud model; In this step, the first control point set includes a plurality of first control points, and the second control point set includes a plurality of second control points.

[0043] The plurality of first control points correspond to the plurality of second control points one by one.

[0044] The first control point may be a key point of the head feature in the first magnetic resonance image, and the second control point may be a key point of the head feature in the first point cloud model. For example, the multiple control points may include key positions of the head, such as the forehead, cheekbones, and chin, and each control point may correspond to position information, such as coordinate information in the magnetic resonance scanning coordinate system.

[0045] Multiple groups of one-to-one corresponding control points may be obtained through an automatic matching algorithm (such as nearest neighbor matching or KD tree algorithm, etc.) or manual annotation, and the present invention is not limited thereto.

[0046] The nearest neighbor matching algorithm can calculate the geometric distance between each control point in the first magnetic resonance image and all control points in the first point cloud model, and select the point with the closest distance as a matching pair; The KD tree (K-Dimensional Tree) algorithm is a tree data structure used to organize multidimensional spatial data. It is suitable for searching key data in multidimensional space, such as range search and nearest neighbor search.

[0047] Based on the automatic matching algorithm, users can make manual adjustments. For example, when the automatic matching is inaccurate or ambiguous, users can make adjustments to improve the accuracy and robustness of control point matching.

[0048] In some embodiments, the matching result may be verified to ensure accurate correspondence of the control points in the first magnetic resonance image and the first point cloud model, for example, by calculating the error between the matched control points or by using other verification methods.

[0049] Step 130: acquiring a target transformation function between the first magnetic resonance image and the first point cloud model based on the first control point set and the second control point set; In this step, the matching first control point set and the second control point set can be used to calculate the transformation function from the coordinate system of the first magnetic resonance image to the coordinate system of the first point cloud model. The transformation function can be a rigid transformation (involving only rotation and translation), or it can be an affine transformation (including scaling, rotation and translation), or it can be a non-rigid transformation (including local deformation, etc.).

[0050] The transformation parameters can be estimated by minimizing the error between sets of control points using, for example, the least squares method or the Iterative Closest Point (ICP) algorithm.

[0051] For example, the first magnetic resonance image can be transformed according to the calculated initial transformation parameters, and the degree of alignment between the transformed first magnetic resonance image and the first point cloud model can be evaluated. If the degree of alignment is not ideal, the parameters of the transformation function can be adjusted through an iterative optimization process until the alignment requirements are met.

[0052] The quality of the transformation function can be assessed using quantitative metrics such as the root mean square error or Hausdorff distance.

[0053] Step 140: Correct the first magnetic resonance image based on the target transformation function to obtain a target magnetic resonance image corresponding to the object to be measured.

[0054] The target transformation function may be applied to the pixel or voxel coordinates of the first magnetic resonance image, for example the position of each pixel or voxel may be transformed from the coordinate system of the first magnetic resonance image to the coordinate system of the first point cloud model.

[0055] When the target transformation function is a rigid transformation, the coordinates of the first magnetic resonance image may be directly transformed. When the target transformation function is a non-rigid transformation, an interpolation method may be used to recalculate pixel values ​​in the transformed image.

[0056] The transformation may result in new coordinates that do not correspond to integer pixel positions in the original image. Image resampling may be performed. For example, an interpolation algorithm (such as nearest neighbor interpolation, bilinear interpolation, or cubic interpolation) may be used to estimate pixel values ​​at non-integer coordinates in the transformed image.

[0057] The transformation may cause part of the image content to exceed the boundaries of the original image. You can choose to crop the parts that exceed the boundaries, or use specific boundary conditions (such as mirror boundary conditions or periodic boundary conditions) to fill these areas.

[0058] The quality of the rectified MRI images can be assessed using quantitative metrics such as image registration error or anatomical consistency, and a visual check can be performed to ensure that the rectified images are anatomically consistent with the first point cloud model.

[0059] According to the magnetic resonance image distortion correction method combined with a depth camera provided by an embodiment of the present invention, the distortion correction is performed by using the real three-dimensional model of the head of the object to be measured (the first point cloud model) obtained by the depth camera, without the need to use a calibration model, thereby improving the convenience of operation, shortening the scanning preparation time, and thus improving the efficiency of clinical work; by directly comparing the real three-dimensional model reconstructed by the depth camera to obtain the transformation function between the first magnetic resonance image to be corrected and the real three-dimensional model, the first magnetic resonance image is corrected based on the transformation function, reducing the dependence on a fixed reference image, improving the adaptability of the correction method under different devices and scanning conditions, and achieving a better correction effect.

[0060] In some embodiments, processing the depth data corresponding to the object to be measured acquired by the depth camera to obtain a first point cloud model corresponding to the object to be measured may include: Obtaining first depth data corresponding to the object to be measured collected by the depth camera at multiple angles before the object to be measured enters the target scanning space, and second depth data corresponding to the object to be measured collected by the depth camera in the target scanning space; Using a point cloud processing algorithm to process the first depth data, and obtaining a three-dimensional head model corresponding to the first depth data; The second depth data and the three-dimensional head model are aligned to obtain a first point cloud model.

[0061] In this embodiment, if Figure 4 As shown, before the object to be measured enters the target scanning space, a depth camera can be used to collect depth data of the object to be measured from multiple angles, namely, first depth data. Multi-angle collection helps to capture comprehensive information of the object to be measured to reconstruct a three-dimensional model of the head of the object to be measured.

[0062] After the object to be tested enters the target scanning space, the depth camera can be used again to collect depth data, i.e., second depth data. When the object to be tested lies in the magnetic resonance scanning space, the head of the object to be tested is usually wrapped by an open coil, and the depth camera can obtain depth data of part of the face.

[0063] A point cloud processing algorithm, such as the PCL (Point Cloud Library) library or the Open3D library, may be used to process the first depth data to construct a three-dimensional model based on the depth information. During the three-dimensional model reconstruction process, a noise filtering algorithm may be applied to remove noise points in the point cloud. The point clouds scanned from different angles may be registered to form a complete head model.

[0064] The point cloud data can be converted into a three-dimensional mesh model using a surface reconstruction algorithm (such as Poisson surface reconstruction or Delaunay triangulation). The reconstructed three-dimensional model can be surface smoothed to reduce the unevenness and roughness of the model surface. For example, this can be achieved using algorithms such as Laplace smoothing or mesh simplification.

[0065] The second depth data and the three-dimensional head model may be converted into a scanning coordinate system corresponding to the magnetic resonance device, and the second depth data and the three-dimensional head model may be aligned in the same coordinate system.

[0066] For example, feature points of more prominent parts of the face can be extracted from the second depth data, and corresponding feature points can be extracted on the three-dimensional head model at the same time. Then, a feature matching algorithm (such as an ICP algorithm or a RANSAC (Random Sample Consensus) algorithm) can be used to match the feature points in the second depth data with the feature points on the three-dimensional head model. Then, based on the result of the feature matching, the second depth data can be translated, rotated, scaled, and other operations can be performed to align it with the three-dimensional head model in space.

[0067] After alignment, the second depth data may be fused with the three-dimensional head model to obtain a first point cloud model, wherein the first point cloud model has accurate position coordinate information in a scanning coordinate system of the magnetic resonance device.

[0068] In some embodiments, aligning the second depth data with the three-dimensional head model to obtain the first point cloud model may include: Acquire at least one first feature point in the second depth data; Acquire at least one second feature point matching the at least one first feature point from the three-dimensional head model; Using an iterative closest point algorithm to process at least one first feature point and a second feature point to obtain a target registration parameter between the second depth data and the three-dimensional head model; processing the second depth data based on the target registration parameter to align the second depth data to the three-dimensional head model; A first point cloud model is obtained based on the aligned second depth data and the three-dimensional head model.

[0069] In this embodiment, the first feature points are significant and stable feature points in the second depth data. For example, at least one first feature point may include key points of key parts such as facial contour, glasses, nose and mouth.

[0070] The matched feature points can be processed using an iterative closest point algorithm, which can optimize the alignment of two point clouds by iteratively finding the closest point pairs and minimizing the distance between them.

[0071] After multiple iterations, the iterative closest point algorithm can output a transformation matrix (including rotation and translation parameters), namely, the target registration parameters, which can then be used to transform the second depth data so that it is spatially aligned with the three-dimensional head model.

[0072] like Figure 5 As shown, in the actual implementation process, the key feature points (first feature points) in the partial point cloud (i.e., the second depth data) can be extracted first, and the corresponding second feature points can be found in the complete three-dimensional head model. By minimizing the distance error between the two, the alignment parameters can be gradually optimized to finally achieve seamless fusion of the two parts of the point cloud. After the alignment is completed, the generated complete head point cloud model (first point cloud model) will have accurate position coordinates in the coordinate system scanned by the magnetic resonance device, so that subsequent image processing steps can be performed in the same coordinate system.

[0073] like Figure 6 As shown, in some embodiments, step 120 may include: Performing contour extraction processing on the first magnetic resonance image and the first point cloud model respectively to obtain first contour information corresponding to the first magnetic resonance image and second contour information corresponding to the first point cloud model; A first control point set and a second control point set are acquired based on first difference information between the first contour information and the second contour information.

[0074] In this embodiment, for the first magnetic resonance image, an edge detection algorithm (such as Canny edge detection or Sobel operator, etc.) can be applied to extract the outer contour of the head, that is, the first contour information, and the two-dimensional contour point set can be mapped to the three-dimensional space as needed for comparison with the three-dimensional first point cloud model.

[0075] For the first point cloud model, a grid-based edge detection algorithm or a voxel-based surface extraction method can be used to obtain an outer contour point set of the head, that is, the second contour information.

[0076] For example, the surface mesh can be extracted from the first point cloud model using an isosurface extraction algorithm (Marching Cubes), and then the outer contour boundary can be identified using edge detection technology.

[0077] By comparing the first contour information and the second contour information, the geometric distortion region in the first magnetic resonance image can be identified and located, and according to the comparison result, a group of corresponding control points can be selected.

[0078] In some embodiments, step 130 may include: The first control point set and the second control point set are processed by a thin plate spline method to obtain a target transformation function.

[0079] In this embodiment, a thin plate splines (TPS) method can calculate a transformation function according to a selected set of control points. The TPS transformation function includes an affine transformation part and a nonlinear transformation part, and can effectively capture and correct complex geometric distortions.

[0080] like Figure 7 As shown, the optimal TPS transformation parameters can be determined based on the source control point set (first control point set) and the target control point set (second control point set) by minimizing the transformation energy function.

[0081] In some embodiments, step 130 may include: Based on the formula:

[0082] Get the target transformation function.

[0083] In this embodiment, is the target transformation function, and are the affine transformation parameters, is the location information of the target control point in the first control point set, is the nonlinear transformation weight, is the control point corresponding to the target control point in the second control point set ( control points), is the radial basis function, is the index variable used to traverse all control points in the second control point set, is an integer in the range [1, N], where N is the total number of control points in the second control point set.

[0084] Among them, the radial basis function is usually: , for and The Euclidean distance between .

[0085] In some embodiments, the weight parameters in the TPS transformation function may be dynamically adjusted according to the importance of different control points to improve the flexibility and accuracy of the transformation effect.

[0086] In some embodiments, parallel computing or graphics processing unit (GPU) acceleration technology may be used to reduce computing time and improve the efficiency of transformation computing.

[0087] According to the magnetic resonance image distortion correction method combined with a depth camera provided in an embodiment of the present invention, by adopting a thin plate spline method combined with a first point cloud model provided by the depth camera, it is possible to accurately capture and correct complex geometric distortions in the first magnetic resonance image, and the correction effect is good.

[0088] In some embodiments, step 140 may include: Correcting the first magnetic resonance image based on the target transformation function to obtain an initial magnetic resonance image corresponding to the object to be tested; acquiring second difference information between the initial magnetic resonance image and the first point cloud model; When the second difference information is smaller than the target difference threshold, the initial magnetic resonance image is determined as the target magnetic resonance image.

[0089] In this embodiment, applying the target transformation function to the first magnetic resonance image can minimize the difference between the first magnetic resonance image and the first point cloud model.

[0090] The initial magnetic resonance image is an image that is corrected from the first magnetic resonance image and is close to the first point cloud model.

[0091] The second difference information between the initial magnetic resonance image and the first point cloud model may be compared, and the second difference information may include the degree of spatial alignment, shape similarity, and feature point matching degree between the image and the point cloud.

[0092] The second difference information may be a quantitative value, for example, it may be an error value between the initial magnetic resonance image and the first point cloud model. The target difference threshold may be user-defined, for example, the target difference threshold may be 5%, or 2%, or may be other values, which are not limited in the present invention.

[0093] For example, a quantitative evaluation method may be used to calculate a mean square error (MSE) or a mean absolute error (MAE) between the initial magnetic resonance image and the first point cloud model to obtain the second difference information.

[0094] When the second difference information is less than the target difference threshold, it can be determined that the initial magnetic resonance image is sufficiently close to the first point cloud model, and the initial magnetic resonance image can be determined as the target magnetic resonance image.

[0095] In the case where the second difference information is greater than or equal to the target difference threshold, the target transformation function may be readjusted and the correction process may be repeated until the condition is met.

[0096] like Figure 8 The schematic diagram of the comparison before and after correction of the first magnetic resonance image is illustrated. According to the test by the inventor, the first magnetic resonance image is corrected by using the magnetic resonance image distortion correction method combined with a depth camera provided in an embodiment of the present invention. The mean square error between the target magnetic resonance image obtained and the first point cloud model is reduced by about 30% compared with the traditional calibration phantom-based method. The present invention can more effectively reduce image distortion and improve the geometric consistency of the image. The calculation time of the correction process is shortened by about 40% compared with traditional image registration and deep learning methods, which can meet the demand for fast image processing in clinical diagnosis, optimize the resource utilization of the system, and reduce the dependence on high-performance computing equipment; Moreover, the present invention shows consistent correction results on magnetic resonance equipment of different models and brands, with a stable error range of less than 1 mm, is applicable to different equipment and scanning conditions, has a wide range of applicable scenarios, and has high reliability and consistency of correction results; The magnetic resonance image distortion correction method combined with a depth camera provided in an embodiment of the present invention can more realistically reflect the head structure of the object to be tested by correcting the magnetic resonance image, which helps medical workers to more accurately identify and analyze lesions in the image, and provides more reliable reference data for fields that require precise preoperative planning, thereby reducing surgical risks and improving the safety and effectiveness of treatment.

[0097] An embodiment of the present invention also provides a magnetic resonance image distortion correction system combined with a depth camera.

[0098] The magnetic resonance image distortion correction system combined with a depth camera includes: a depth camera module, a magnetic resonance imaging module, an image processing module, a TPS conversion module, and a display and verification module.

[0099] In this embodiment, if Figure 3 As shown, the object to be tested is in the scanning room, and the depth camera module can use a high-precision depth camera to perform multi-angle depth scanning on the head of the object to be tested to obtain three-dimensional depth data (first depth data) of the head. The first depth data can be converted into a high-precision three-dimensional head model through a point cloud processing algorithm.

[0100] The magnetic resonance imaging module can use an MRI device (magnetic resonance device) to scan the head of the same object to be tested and obtain a two-dimensional or three-dimensional MRI image (first magnetic resonance image) with geometric distortion. The acquired MRI image is then transmitted to the image processing module through an interface and pre-processed, including denoising and standardization, to improve image quality and consistency.

[0101] In the image processing module, since the human head is wrapped by an open coil when the human body lies in the MRI scanning space, the depth camera can only capture a part of the human facial point cloud image. This part of the point cloud must first be aligned with the complete point cloud reconstructed in the previous step to obtain the position coordinates of the complete head point cloud under MRI, so that the MRI image and the complete head point cloud image can be displayed in the same coordinate system.

[0102] Then, a grid-based edge detection algorithm can be used to extract the outer contour point set from the three-dimensional head model reconstructed by the depth camera, and the Canny edge detection algorithm can be used to process the MRI image to extract the outer contour of the head and map it to the three-dimensional space through stereo matching or three-dimensional reconstruction methods. The system can select corresponding control point sets to make the first control point set and the second control point set accurately correspond in the MRI image and the depth camera model.

[0103] The TPS transformation module can calculate the transformation function based on the selected control point set using the thin plate spline (TPS) method. After the calculation is completed, the target transformation function is applied to the original MRI image, and the geometric mapping from the distorted image to the real model is achieved through the bilinear interpolation algorithm to generate a corrected high-precision MRI image (target magnetic resonance image).

[0104] The corrected image is verified through the display and verification module. The MRI images before and after correction can be superimposed and displayed with the first point cloud model reconstructed by the depth camera through visual comparison to intuitively check the correction effect of the geometric distortion. Then the system can use quantitative evaluation methods, such as calculating the mean square error (MSE) and mean absolute error (MAE), to evaluate the accuracy and robustness of the correction results to ensure the geometric accuracy and reliability of the corrected MRI images in clinical applications.

[0105] In the present invention, a multimodal data fusion method is used to comprehensively process the multi-view depth data from the depth camera and the MRI image. The depth data obtained by multi-angle scanning is aligned and fused using the point cloud fusion algorithm ICP algorithm to generate a more complete and accurate three-dimensional head model. The synchronous optimization algorithm in the image processing module can ensure the consistency of the MRI image and the three-dimensional model in space and time, and improve the accuracy of multimodal data fusion by combining rigid registration with non-rigid registration.

[0106] The system can adopt a multi-dimensional correction result verification method. First, the MRI images before and after correction can be superimposed and displayed with the 3D model reconstructed by the depth camera through the visual comparison function to visually check the correction effect of geometric distortion, such as Figure 8 The schematic diagram of the comparison of MRI images before and after correction is shown, which clearly shows the consistency of the geometric structure after distortion correction.

[0107] Secondly, the system can calculate the geometric error indicators between the image and the three-dimensional model before and after correction, such as mean square error (MSE), mean absolute error (MAE) and Hausdorff distance, through the quantitative error evaluation module. Through statistical analysis, the superiority of the method of the present invention in error indicators is verified. For example, in the experiment, the MSE of the image corrected by the method provided by the present invention is reduced by about 30% compared with the traditional method, the MAE is reduced by 25%, and the Hausdorff distance is shortened by 20%, thereby improving the correction accuracy.

[0108] Finally, the system was tested in an actual clinical environment, and doctors conducted diagnostic tests on MRI images corrected by the method of the present invention, and collected data on diagnostic accuracy and time efficiency. The test results showed that the corrected images helped doctors identify and locate lesions more accurately, shortened the diagnosis time by about 15%, and improved the diagnostic accuracy by about 20%. These data verified the practicality and effectiveness of the method provided by the present invention in clinical applications.

[0109] The following is a description of the magnetic resonance image distortion correction device combined with a depth camera provided by the present invention. The magnetic resonance image distortion correction device combined with a depth camera described below and the magnetic resonance image distortion correction method combined with a depth camera described above can be referred to each other.

[0110] The magnetic resonance image distortion correction method combined with a depth camera provided in an embodiment of the present invention may be executed by a magnetic resonance image distortion correction device combined with a depth camera. In the embodiment of the present invention, the magnetic resonance image distortion correction method combined with a depth camera is performed by a magnetic resonance image distortion correction device combined with a depth camera as an example to illustrate the magnetic resonance image distortion correction device combined with a depth camera provided in an embodiment of the present invention.

[0111] An embodiment of the present invention also provides a magnetic resonance image distortion correction device combined with a depth camera.

[0112] like Fig. 9 As shown, the magnetic resonance image distortion correction device combined with a depth camera includes: a first processing module 910, a second processing module 920, a third processing module 930 and a fourth processing module 940.

[0113] The first processing module 910 is used to process the depth data corresponding to the object to be measured collected by the depth camera to obtain a first point cloud model corresponding to the object to be measured; obtain a first magnetic resonance image corresponding to the object to be measured collected by the magnetic resonance device in the target scanning space; the first point cloud model and the first magnetic resonance image are in the same coordinate system; A second processing module 920 is used to process the first magnetic resonance image and the first point cloud model to obtain a first control point set corresponding to the first magnetic resonance image and a second control point set corresponding to the first point cloud model, wherein the first control points in the first control point set correspond to the second control points in the second control point set in a one-to-one manner; A third processing module 930 is used to obtain a target transformation function between the first magnetic resonance image and the first point cloud model based on the first control point set and the second control point set; The fourth processing module 940 is used to correct the first magnetic resonance image based on the target transformation function to obtain a target magnetic resonance image corresponding to the object to be measured.

[0114] According to the magnetic resonance image distortion correction device combined with a depth camera provided by an embodiment of the present invention, the distortion correction is performed by using the real three-dimensional model of the head of the object to be measured (the first point cloud model) obtained by the depth camera, without the need to use a calibration model, thereby improving the convenience of operation, shortening the scanning preparation time, and thus improving the efficiency of clinical work; by directly comparing the real three-dimensional model reconstructed by the depth camera to obtain the transformation function between the first magnetic resonance image to be corrected and the real three-dimensional model, the first magnetic resonance image is corrected based on the transformation function, reducing the dependence on a fixed reference image, improving the adaptability of the correction method under different devices and scanning conditions, and achieving a better correction effect.

[0115] In some embodiments, the first processing module 910 may also be used to: Obtaining first depth data corresponding to the object to be measured collected by the depth camera at multiple angles before the object to be measured enters the target scanning space, and second depth data corresponding to the object to be measured collected by the depth camera in the target scanning space; Using a point cloud processing algorithm to process the first depth data, and obtaining a three-dimensional head model corresponding to the first depth data; The second depth data and the three-dimensional head model are aligned to obtain a first point cloud model.

[0116] In some embodiments, the first processing module 910 may also be used to: Acquire at least one first feature point in the second depth data; Acquire at least one second feature point matching the at least one first feature point from the three-dimensional head model; Using an iterative closest point algorithm to process at least one first feature point and a second feature point to obtain a target registration parameter between the second depth data and the three-dimensional head model; processing the second depth data based on the target registration parameter to align the second depth data to the three-dimensional head model; A first point cloud model is obtained based on the aligned second depth data and the three-dimensional head model.

[0117] In some embodiments, the second processing module 920 may also be used to: Performing contour extraction processing on the first magnetic resonance image and the first point cloud model respectively to obtain first contour information corresponding to the first magnetic resonance image and second contour information corresponding to the first point cloud model; A first control point set and a second control point set are acquired based on first difference information between the first contour information and the second contour information.

[0118] In some embodiments, the third processing module 930 may also be used to: Based on the formula:

[0119] Get the target transformation function, where is the target transformation function, and are the affine transformation parameters, is the location information of the target control point in the first control point set, is the nonlinear transformation weight, is the position information of the control point corresponding to the target control point in the second control point set, is the radial basis function, is the index variable used to traverse all control points in the second control point set, is an integer in the range [1, N], where N is the total number of control points in the second control point set.

[0120] In some embodiments, the fourth processing module 940 may also be used to: Correcting the first magnetic resonance image based on the target transformation function to obtain an initial magnetic resonance image corresponding to the object to be tested; acquiring second difference information between the initial magnetic resonance image and the first point cloud model; When the second difference information is smaller than the target difference threshold, the initial magnetic resonance image is determined as the target magnetic resonance image.

[0121] The magnetic resonance image distortion correction device combined with a depth camera in the embodiment of the present invention can be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present invention.

[0122] The magnetic resonance image distortion correction device combined with a depth camera in the embodiment of the present invention may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present invention.

[0123] The magnetic resonance image distortion correction device combined with a depth camera provided in the embodiment of the present invention can achieve Figures 1 to 8 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0124] In some embodiments, Fig.10 As shown, an embodiment of the present invention further provides an electronic device 1000, including a processor 1001, a memory 1002, and a computer program stored in the memory 1002 and executable on the processor 1001. When the program is executed by the processor 1001, each process of the embodiment of the magnetic resonance image distortion correction method combined with a depth camera is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0125] It should be noted that the electronic devices in the embodiments of the present invention include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0126] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the various processes of the above-mentioned magnetic resonance image distortion correction method embodiment combined with a depth camera, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0127] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the various processes of the above-mentioned magnetic resonance image distortion correction method combined with a depth camera are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0128] On the other hand, an embodiment of the present invention further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the various processes of the above-mentioned magnetic resonance image distortion correction method embodiment combined with a depth camera, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0129] It should be understood that the chip mentioned in the embodiment of the present invention may also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0130] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0131] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for correcting magnetic resonance image distortion in combination with a depth camera, characterized in that: include: Processing the depth data corresponding to the object to be measured collected by the depth camera to obtain a first point cloud model corresponding to the object to be measured; Acquire a first magnetic resonance image corresponding to the object to be measured, which is acquired by a magnetic resonance device in a target scanning space; The first point cloud model and the first magnetic resonance image are in the same coordinate system; Processing the first magnetic resonance image and the first point cloud model to obtain a first control point set corresponding to the first magnetic resonance image and a second control point set corresponding to the first point cloud model, wherein a plurality of first control points in the first control point set correspond one-to-one to a plurality of second control points in the second control point set; acquiring a target transformation function between the first magnetic resonance image and the first point cloud model based on the first control point set and the second control point set; The first magnetic resonance image is corrected based on the target transformation function to obtain a target magnetic resonance image corresponding to the object to be measured.

2. The method for correcting magnetic resonance image distortion in combination with a depth camera according to claim 1, characterized in that: The processing of the depth data corresponding to the object to be measured collected by the depth camera to obtain a first point cloud model corresponding to the object to be measured includes: Acquire first depth data corresponding to the object to be measured collected by the depth camera at multiple angles before the object to be measured enters the target scanning space, and second depth data corresponding to the object to be measured collected by the depth camera in the target scanning space; Using a point cloud processing algorithm to process the first depth data, and obtaining a three-dimensional head model corresponding to the first depth data; The second depth data and the three-dimensional head model are aligned to obtain the first point cloud model.

3. The method for correcting magnetic resonance image distortion in combination with a depth camera according to claim 2, characterized in that: The aligning the second depth data with the three-dimensional head model to obtain the first point cloud model includes: Acquire at least one first feature point in the second depth data; Acquire at least one second feature point matching the at least one first feature point from the three-dimensional head model; Using an iterative closest point algorithm to process the at least one first feature point and the second feature point to obtain a target registration parameter between the second depth data and the three-dimensional head model; processing the second depth data based on the target registration parameters to align the second depth data to the three-dimensional head model; The first point cloud model is acquired based on the aligned second depth data and the three-dimensional head model.

4. The method for correcting magnetic resonance image distortion in combination with a depth camera according to any one of claims 1 to 3, characterized in that: The processing of the first magnetic resonance image and the first point cloud model to obtain a first control point set corresponding to the first magnetic resonance image and a second control point set corresponding to the first point cloud model includes: Performing contour extraction processing on the first magnetic resonance image and the first point cloud model respectively to obtain first contour information corresponding to the first magnetic resonance image and second contour information corresponding to the first point cloud model; The first control point set and the second control point set are acquired based on first difference information between the first contour information and the second contour information.

5. The method for correcting magnetic resonance image distortion in combination with a depth camera according to any one of claims 1 to 3, characterized in that: The acquiring, based on the first control point set and the second control point set, a target transformation function between the first magnetic resonance image and the first point cloud model comprises: Based on the formula: Get the target transformation function, where is the target transformation function, and are the affine transformation parameters, is the position information of the target control point in the first control point set, is the nonlinear transformation weight, is the position information of the control point corresponding to the target control point in the second control point set, is the radial basis function, is an index variable used to traverse all control points in the second control point set, is an integer in the range [1, N], where N is the total number of control points in the second control point set.

6. The method for correcting magnetic resonance image distortion in combination with a depth camera according to any one of claims 1 to 3, characterized in that: The correcting the first magnetic resonance image based on the target transformation function to obtain a target magnetic resonance image corresponding to the object to be measured includes: Correcting the first magnetic resonance image based on the target transformation function to obtain an initial magnetic resonance image corresponding to the object to be measured; Acquiring second difference information between the initial magnetic resonance image and the first point cloud model; When the second difference information is smaller than a target difference threshold, the initial magnetic resonance image is determined as the target magnetic resonance image.

7. A magnetic resonance image distortion correction device combined with a depth camera, characterized in that: include: A first processing module, used to process the depth data corresponding to the object to be measured collected by the depth camera, and obtain a first point cloud model corresponding to the object to be measured; Acquire a first magnetic resonance image corresponding to the object to be measured, which is acquired by a magnetic resonance device in a target scanning space; The first point cloud model and the first magnetic resonance image are in the same coordinate system; a second processing module, configured to process the first magnetic resonance image and the first point cloud model to obtain a first control point set corresponding to the first magnetic resonance image and a second control point set corresponding to the first point cloud model, wherein a plurality of first control points in the first control point set correspond one-to-one to a plurality of second control points in the second control point set; a third processing module, configured to acquire a target transformation function between the first magnetic resonance image and the first point cloud model based on the first control point set and the second control point set; The fourth processing module is used to correct the first magnetic resonance image based on the target transformation function to obtain a target magnetic resonance image corresponding to the object to be measured.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the magnetic resonance image distortion correction method combined with a depth camera as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for correcting magnetic resonance image distortion combined with a depth camera as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for correcting magnetic resonance image distortion combined with a depth camera as described in any one of claims 1 to 6 is implemented.

Images