Automatic and precise splicing method, device, equipment and medium for magnetic resonance full spine images

Abstract

The present application provides a method, device, equipment and medium for automatic and precise stitching of magnetic resonance full spine images. By obtaining at least two images to be stitched and corresponding scanning information, each of the images to be stitched is pre-processed, and one or more pairs of images to be stitched are determined based on the overlapping area; the image feature points of each pair of images to be stitched are extracted and roughly aligned to obtain the corresponding alignment matrix; the image feature points are sampled and precisely aligned to optimize the alignment matrix; each pair of images to be stitched is stitched based on the optimized alignment matrix, and the stitched images are fused. The present application has high alignment accuracy, and the average deviation value of the inliers is significantly reduced after precise alignment. By optimizing the alignment matrix, better alignment effects can also be obtained for slices far away from the center of the magnetic field.

Description

Technical Field

[0001] The present application relates to the technical field of image stitching and processing, and in particular to a method, device, equipment and medium for automatic and precise stitching of magnetic resonance full spine images. Background Art

[0002] Due to the limitation of FOV (field of view), it is often impossible to obtain the entire spine information in one go in magnetic resonance imaging. Multiple shots are required, and the acquired partial spine images are stitched together to obtain the entire spine image.

[0003] Currently, the following problems exist in the whole spine image stitching process:

[0004] 1) Stitched images often use the same pixel resolution. For relatively small tissues such as the head, the pixels are larger and the FOV is larger, which will lead to problems such as long scanning time and decreased image contrast.

[0005] 2) The characteristic information of magnetic resonance images is not obvious, the characteristic information of intervertebral discs is similar, and the matching accuracy is not high.

[0006] 3) Magnetic resonance images that deviate from the center of the magnetic field are prone to deformation and other problems, so the stitching error of edge slice images increases.

[0007] 4) After the magnetic resonance images are stitched, the stitching seams of the stitched images are prone to deformation, making it more difficult to fuse the stitched images at this part. Summary of the Invention

[0008] In view of the shortcomings of the prior art described above, the purpose of this application is to provide a method, device, equipment and medium for automatic and accurate splicing of magnetic resonance full spine images to solve the problems existing in the prior art in splicing magnetic resonance full spine images.

[0009] To achieve the above-mentioned objectives and other related objectives, the present application provides a method for automatic and precise stitching of magnetic resonance full-spine images, the method comprising: obtaining at least two images to be stitched and corresponding scanning information, preprocessing each of the images to be stitched, and determining one or more image pairs to be stitched based on the overlapping area; extracting image feature points of each of the image pairs to be stitched and performing coarse registration to obtain a corresponding registration matrix; sampling the image feature points and performing fine registration to optimize the registration matrix; stitching each of the image pairs to be stitched based on the optimized registration matrix, and fusing the stitched images.

[0010] In one embodiment of the present application, the scanning information includes any one or more combinations of image resolution, pixel resolution, image direction, patient position, image layer thickness, number of image layers, and patient number.

[0011] In one embodiment of the present application, obtaining at least two images to be stitched and corresponding scanning information, and preprocessing each of the images to be stitched, includes: determining the stitching range and order of the images to be stitched; unifying the pixel resolution and image resolution of each of the images to be stitched based on the scanning information; calculating a rotation matrix based on the scanning information, unifying the images to be stitched into a coordinate system through the rotation matrix, and obtaining slice images of the same body part in the same slice at different scanning positions through an interpolation method; calculating the three-dimensional coordinates of all slice images on the bed coordinate system when the human body is scanned based on the unified coordinate system, and calculating the overlapping area of ​​the three-dimensional positions of the images to be stitched based on the coordinate position of each part in the bed coordinate system to determine one or more pairs of images to be stitched.

[0012] In one embodiment of the present application, the extraction of image feature point information of each pair of images to be stitched is performed by traversing and calculating feature points using a feature template; the feature template can be set to different shapes according to image characteristics.

[0013] In one embodiment of the present application, the sampling and fine registration of the image feature points to optimize the registration matrix includes: using the image feature points of one image to be stitched in the pair of images to be stitched as a two-dimensional image point set to be stitched, and the image feature points of the other image to be stitched as a target image point set; sampling the two-dimensional image point set to be stitched and the target image point set respectively; calculating the position distances between the inner points and the outer points and between the points respectively, and iteratively optimizing the registration matrix based on the ratio of the number of inner points and outer points and the average distance between the points; on this basis, combining the registration matrix of each layer and the overall FOV position information of the layer to further optimize the registration matrix.

[0014] In one embodiment of the present application, the method includes: after searching for each feature point on the two-dimensional image point set to be registered and calculating the registration matrix obtained through the current iteration, determining whether it has a nearest neighbor point on the target image point set; if a nearest neighbor point exists, it is considered to be an internal point; if not, it is considered to be an external point.

[0015] In one embodiment of the present application, the fusion processing includes: calculating the final set of inliers and outliers based on the inliers and outliers calculated during fine registration; setting weights based on the deviation of the distance of each inlier from the center of the image to be registered; using a template frame, and multiplying the template frame by the weight to obtain a matrix, and sliding it through the intersection of the two images to be registered to complete the gradient correction of the boundary.

[0016] To achieve the above-mentioned objectives and other related objectives, the present application provides a device for automatic and precise stitching of magnetic resonance full-spine images, the device comprising: a preprocessing module for acquiring at least two images to be stitched and corresponding scanning information, preprocessing each of the images to be stitched, and determining one or more image pairs to be stitched based on the overlapping area; a coarse registration module for extracting image feature points of each of the image pairs to be stitched and performing coarse registration to obtain a corresponding registration matrix; a fine registration module for sampling and processing the image feature points and performing fine registration to optimize the registration matrix; and a fusion module for stitching each of the image pairs to be stitched based on the optimized registration matrix, and performing fusion processing on the stitched images.

[0017] To achieve the above-mentioned purpose and other related purposes, the present application provides a computer device, which includes: a memory, a processor and a communicator; the memory is used to store computer instructions; and the processor executes the computer instructions to implement the above-mentioned method.

[0018] To achieve the above-mentioned purpose and other related purposes, the present application provides a computer-readable storage medium storing computer instructions, which execute the above-mentioned method when executed.

[0019] In summary, the present application provides a method, device, equipment and medium for automatic and precise stitching of magnetic resonance full spine images. By obtaining at least two images to be stitched and corresponding scanning information, each of the images to be stitched is pre-processed, and one or more image pairs to be stitched are determined based on the overlapping area; the image feature points of each image pair to be stitched are extracted and coarsely aligned to obtain a corresponding alignment matrix; the image feature points are sampled and finely aligned to optimize the alignment matrix; each image pair to be stitched is stitched according to the optimized alignment matrix, and the stitched images are fused.

[0020] It has the following beneficial effects:

[0021] The registration accuracy of this application is high, and the average deviation value of the inner point is significantly reduced after fine registration. By optimizing the registration matrix, good registration effect can be obtained for slices far away from the center of the magnetic field. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Shown is a flow chart of a method for automatic and precise stitching of magnetic resonance whole spine images in one embodiment of the present application.

[0023] Figure 2 Shown is a schematic diagram of a spinal column image in one embodiment of the present application.

[0024] Figure 3Shown is a structural schematic diagram of a feature template in one embodiment of the present application.

[0025] Figures 4A-4B They respectively show simulation diagrams of coarse registration and fine registration between feature points of a pair of images to be stitched in one embodiment of the present application.

[0026] Figures 5A-5B Schematic diagrams respectively show partial spinal images to be spliced ​​that are scanned in the same sequence at different distances from the magnet center in one embodiment of the present application.

[0027] Figures 6A-6B Schematic diagrams respectively show overlapping areas of stitched images after stitching in one embodiment of the present application.

[0028] Figures 7A-7B They are schematic diagrams showing feature point extraction of images to be stitched in one embodiment of the present application.

[0029] Figure 8 They are schematic diagrams showing the entire spine after splicing and fusion in one embodiment of the present application.

[0030] Figure 9 Shown is a module diagram of a device for automatic and precise stitching of magnetic resonance whole spine images in one embodiment of the present application.

[0031] Figure 10 Shown is a schematic structural diagram of a computer device in one embodiment of the present application. DETAILED DESCRIPTION

[0032] The following describes the embodiments of the present application through specific examples. Those skilled in the art can easily understand the other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0033] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application. Although the illustrations only show components related to the present application and are not drawn according to the number, shape and size of components in actual implementation, the type, quantity and proportion of each component in actual implementation can be changed arbitrarily, and the component layout type may also be more complex.

[0034] As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It should be further understood that the terms "comprise" and "include" indicate the presence of the described features, steps, operations, elements, components, items, kinds, and / or groups, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Thus, "A, B or C" or "A, B and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B and C". Exceptions to this definition occur only when the combination of elements, functions, steps or operations are inherently mutually exclusive in some way.

[0035] like Figure 1 FIG. 1 is a flow chart of a method for automatically and accurately stitching magnetic resonance whole spine images according to an embodiment of the present invention. As shown in the figure, the method includes:

[0036] Step S101: obtaining at least two images to be stitched and corresponding scanning information, pre-processing each of the images to be stitched, and determining one or more pairs of images to be stitched according to overlapping areas.

[0037] In this application, at least two images to be stitched can be obtained by scanning the entire spine with magnetic resonance imaging (MRI), for example, 2 to 3 images are required for an adult. The images to be stitched contain a lot of important information, including but not limited to: image resolution, pixel resolution (Pixelspacing), image orientation, patient position, image layer thickness, number of image layers, and any one or more combinations of patient numbers. Based on the relevant image information, corresponding processing information can be provided for subsequent pre-processing steps. For example, image resolution and pixel resolution are crucial for unifying the pixels of the stitched images.

[0038] In one embodiment of the present application, pre-processing is performed on each of the images to be stitched, including:

[0039] A. Screening the range and order of image stitching based on the patient number in the scan information. For example, the images can be preliminarily screened based on information such as the patient number to determine whether they fall within the stitching range and to determine the stitching order.

[0040] B. Unifying the pixel resolution and image resolution of each of the images to be spliced ​​based on the scanning information.

[0041] Simply put, for the images to be stitched, the same pixel resolution is often selected. For relatively small tissues such as the head, using the same resolution will result in larger pixels and reduced contrast. In addition, since the same large FOV as large tissue is used, the scanning time is invisibly increased.

[0042] To this end, this application considers the use of flexible FOV and pixel resolution. For example, for smaller tissue areas, a small FOV scan can be selected to improve scanning efficiency and image contrast. Therefore, this step requires unifying the pixel resolution and image resolution based on the image information and the corresponding scan information. For example, interpolation can be used to unify the small FOV area with the large FOV area.

[0043] C. Obtaining a rotation matrix of the images to be stitched according to the scanning information, unifying the images to be stitched into a unified coordinate system using the rotation matrix, and obtaining slice images of the overlapping area in the same slice by an interpolation method.

[0044] Specifically, the rotation matrix corresponding to the stitched images can be calculated based on image orientation, patient position, and other information contained in the image information, and the images to be stitched can be unified into a unified coordinate system. Here, the rotation matrix refers to the transformation matrix of the image coordinate system of any of the 3D images to be stitched relative to the unified coordinate system, which is the coordinate system of the bed used for the body scan.

[0045] Taking into account the convenience of the doctor's operation during the scanning process, there is a possibility that the coordinates of the scanned image layers of different parts will change. Therefore, in this step, based on the unified coordinate system, interpolation and other methods will be used to obtain image information of the same layer of the same body part in different scanning parts.

[0046] D. Calculate the three-dimensional coordinates of all slice images in the bed coordinate system during human body scanning based on the unified coordinate system, and calculate the overlapping area of ​​the three-dimensional positions of the images to be stitched based on the coordinate position of each part in the bed coordinate system to determine one or more pairs of images to be stitched.

[0047] In simple terms, in the unified coordinate calculation, the three-dimensional coordinates of all slice images in the bed coordinate system where the slice images are located during human body scanning are calculated. Therefore, the three-dimensional position overlap area of ​​the images can be calculated according to the coordinate position of each part (head 1, head 2...head n, spine 1, spine 2...spine n) in the bed coordinate system, and the splicing range is determined. The splicing image pairs are selected from the image group to be spliced. Preferably, the splicing image pairs mentioned here refer to the image pairs on the same layer after interpolation that have a certain proportion of overlapping areas. It should be noted that the layer unification in this application is mainly aimed at the splicing of 2D images.

[0048] Step S102: extracting image feature points of each pair of images to be stitched and performing rough registration to obtain a corresponding registration matrix.

[0049] In some examples, image feature information is obtained, and feature calculation methods may be used to extract feature information of each pair of images, such as Sift, Surf, Orb, and other feature extraction algorithms to extract feature point information.

[0050] It should be noted that there are usually areas with similar local information and similar feature information in spinal images, such as the similar morphology of the lumbar disc area. For such areas, it is easy to have the problem of single feature point mismatching, such as Figure 2 The image features of rectangle 1 and rectangle 2 are similar.

[0051] This problem can be solved by expanding the feature calculation template, but expanding the template will undoubtedly increase the calculation time and may cause the loss of boundary information. To solve this problem, in addition to using conventional 3*3, 5*5 and other square templates, this application also uses special templates such as cross and straight shapes to obtain more and more effective feature information while reducing or maintaining the amount of calculation. The feature templates are as follows: Figure 3 As shown in the figure, (a) is a rectangular template, (b) is a cross template, (c) is a linear template, and (d) is an elliptical template. For example, the redundancy analysis of each template has the following relationship: w = L1 = a = 25 (pixels), h = L2 = L = b = 30 (pixels), W1 = W2 = 5 (pixels), α = 90 (degrees), then the number of pixels contained in the four scopes can be: 3111 for the rectangular template, 1111 for the cross template, 61 for the linear template, and 2347 for the elliptical template.

[0052] As can be seen from the figure, if the difference in feature information only appears in certain specific directions, using linear, cross-shaped, and other feature calculation templates can not only obtain most of the feature information, but also greatly reduce the amount of calculation.

[0053] For example, when acquiring new feature information, if feature templates are used to traverse and calculate feature points, special templates such as crosses or straight lines can be selected based on the characteristics of the image. This can reduce calculation time and improve the accuracy of feature information, provided that the feature information is identical, similar, or large. Feature points can be extracted using a single algorithm or multiple algorithms simultaneously. The extraction algorithm can consider point features such as Harris and Susan, line features such as the log operator, or surface features. In this example, methods with high scale invariance are primarily considered.

[0054] In addition to the above processing, this application adopts a global feature point registration method. First, for different types of initial images, a traditional registration method is used or an initial coarse registration matrix is ​​obtained based on mechanical coordinate information.

[0055] The coarse matching described here can be obtained based on the image's coordinate information or calculated using an algorithm, such as feature matching. During preprocessing, depending on the original image information, one or more operations, such as interpolation, pixel resolution unification, and coordinate system unification, are performed on the image. Therefore, the coarse matching method used in this example is selected based on the original image and its preprocessing.

[0056] Step S103: sampling the image feature points and performing precise registration to optimize the registration matrix.

[0057] On the basis of coarse registration, this application performs fine registration on the feature points, calculates the position distance between points, and minimizes the average distance between points, thereby obtaining a fine registration matrix and achieving a more accurate registration effect.

[0058] In one embodiment of the present application, step S103 specifically includes:

[0059] A. Using the image feature points of one of the image pairs to be stitched as a two-dimensional image point set to be registered, and the image feature points of the other image to be stitched as a target image point set.

[0060] B. Sampling the two-dimensional image point set to be registered and the target image point set respectively.

[0061] C. Calculate the position distances of the inner points and the outer points as well as the distances between the points respectively, and iteratively optimize the registration matrix according to the ratio of the number of inner points to the outer points and the average distance between the points.

[0062] D. On this basis, the registration matrix of each layer and the overall FOV position information of the layer are combined to further optimize the registration matrix.

[0063] In short, this application uses a method based on singular value decomposition to estimate the morphological pose of two-dimensional points to perform a precise calculation of the two-dimensional full spine image registration matrix. Before fine registration, the feature point information of the image to be registered is first obtained. Then, based on the initial matching matrix of each pair of images to be spliced ​​and the image feature point information, the images are finely registered to obtain the registration matrix.

[0064] For example, feature extraction algorithms such as Sift, Surf, and Orb are used to extract feature point information. These points are used as the two-dimensional image point set to be registered and the target image point set. The feature point set is then sampled. The sampled set is further optimized to calculate the inliers and outliers, and the position distances between the points. Based on the ratio of the number of inliers and outliers, as well as the average distance between the points, the registration matrix is ​​iteratively optimized, preferably to minimize the average distance between the points. The specific iterative process is as follows:

[0065] Let R represent the rotation matrix, t represent the translation matrix, P2 represent the image point set to be registered, and P1 represent the target image point set. Then the registration matrix F(R, t) is expected to achieve the following formula:

[0066]

[0067] Here F is the objective function, that is, the average distance between points that is expected to be minimized as described above.

[0068] Let the center be down:

[0069] at the same time:

[0070] but:

[0071] make:

[0072] Through singular value decomposition (SVD): H = U ∧ V T ;

[0073] The optimized rotation matrix is: R = VU T ;

[0074] The translation information can be further calculated through the rotation matrix R.

[0075] In this embodiment, after each feature point on the two-dimensional image point set to be registered is queried through the registration matrix calculated through the current iteration by searching, it is determined whether it has a nearest neighbor point on the target image point set; if there is a nearest neighbor point, it is considered to be an internal point; if not, it is considered to be an external point.

[0076] In the above calculation process, the concepts of inliers and outliers are mentioned. This application searches for each feature point on the moving image (image 1 to be registered) after being transformed by the registration matrix calculated by the current iteration to see if it has a nearest neighbor point on the target image (image 2 to be registered). The search can be performed using methods such as Kdtree. If a nearest neighbor point exists, it is considered an inlier; if not, it is considered an outlier.

[0077] The purpose of this registration calculation method is to optimize the registration matrix so that the position difference between the feature points of the images to be registered calculated by the optimized registration matrix is ​​minimized. The above method can achieve the optimization purpose. Calculate the rotation matrix to improve the registration accuracy. Figure 4A The simulation diagram shows the rough registration between the feature points of the image pair to be stitched. Figure 4B The figure shows a simulation of the precise registration between the feature points of the image pair to be stitched. The position indicated by the rectangular box in the figure is the stitching position.

[0078] In one or more embodiments of the present application, the method described in the present application further includes: further optimizing or fine-tuning the registration matrix by combining image information.

[0079] Specifically, the image registration matrix is ​​optimized by integrating image information. This information includes, but is not limited to, the characteristic information of each image to be stitched, and the positional relationship between each pair of images to be stitched, which can be obtained during preprocessing. Image information acquired at different levels varies, and image distortion varies between areas far from and near the center of the magnetic field. Consequently, image quality varies. For the same patient, the registration matrix for each pair of images to be stitched, acquired during the same scan (e.g., spine 1, spine 2, ... spine n), should be identical. The registration matrix for each slice is examined, and the matrix rotation angle and translation are calculated. Fine-tuning is then performed on the rotation matrix after precise registration.

[0080] In some embodiments, the stitching matrix of slices far from the center of the magnetic field can be fine-tuned by stitching matrices of several slices close to the center of the magnetic field. For another example, the rotation angle between all slices should be less than a threshold value. Within this threshold value range, the average value and variance of all matrix rotation angles are calculated, and the matrix rotation angle and translation amount with larger variance are fine-tuned according to the average value to make them close to the average value. Through adjustment, a more accurate registration purpose is achieved. The purpose of this step is to optimize the slice image registration matrix with poor effect based on the slice image registration matrix with better quality, so as to achieve a more accurate registration purpose.

[0081] It should be noted that due to FOV (field of view) limitations, obtaining complete information about the adult spine often requires capturing two to three partial spinal images. The purpose of image stitching is to calculate the positional relationship between the images to be stitched based on their overlapping portions. Through image fusion technology, a seamless, clear, and large-scale panoramic view is obtained, helping doctors obtain more information from the images and ultimately assisting in diagnosis.

[0082] The significance of obtaining a seamless, clear, large-scale panoramic view lies in integrating the information of multiple images, and in the process of integration, it is necessary to ensure the integrity of the original information as comprehensively as possible, and the integrated image should not have any connection gaps, so it can be viewed more clearly. One of the key points is seamlessness, which requires the accuracy of image registration. However, due to some characteristics of the MR image itself, it may have problems such as unclear feature information and repeated feature information. For example, for layers far away from the center of the magnet, the image is prone to deformation, the feature information is not obvious, and the stitching error of the edge slice image is prone to increase, such as Figures 5A-5B The following are the images of the spine to be spliced ​​from the same sequence of scans of the same person. Figure 5A For images far away from the center of the magnet, Figure 5B is the image close to the center of the magnet. By comparison, we can see that Figure 5A The farther the center of the medium-range magnet is, the more likely the image will be distorted and the less obvious the feature information will be.

[0083] To address this issue, after the registration calculation is completed, this application locates the 3D spatial position of the image based on the image's position information, and combines the calculated registration matrix group to optimize data such as the registration matrix and image boundary deformation to improve the accuracy of image registration and obtain a more complete stitching result.

[0084] Step S104: each of the to-be-stitched image pairs is stitched together according to the optimized registration matrix, and a fusion process is performed on the stitched images.

[0085] Although an accurate registration matrix has been obtained, considering that the image may be deformed to a certain extent due to problems such as magnetic field inhomogeneity, the purpose of this step is to correct and connect these contact areas.

[0086] In one embodiment of the present application, the fusion process includes:

[0087] A. Calculate the final set of inliers and inliers and their centers based on the inliers and inliers calculated during fine alignment.

[0088] B. Set the weight according to the deviation of the distance between each inner point and the center of the image to be registered;

[0089] C. Using a template frame, the matrix obtained by multiplying the template frame by the weight is slid across the intersection of the two images to be registered to complete the gradual correction of the boundary.

[0090] In simple terms, the present application may adopt a gradual method. For example, in the present application, the inner points and outer points may be calculated during the precise registration process. The center of the inner points, that is, the average value of the inner point coordinates, may be calculated through the calculated final set of inner points. Since image deformation mostly occurs at the image boundary, the deviation of the distance between the inner points and the center point at the boundary of the two images to be spliced ​​is likely to increase. According to the deviation of the distance between each inner point and the center of the two images to be registered, a weight is set, and a template frame is used. The matrix obtained by multiplying the template frame by the weight slides across the intersection of the images, preferably sliding across in a direction perpendicular to the image seam. For example, if the image is spliced ​​left and right, the matrix slides vertically to complete the gradual correction of the boundary.

[0091] In addition to deformation, due to the different signal intensities of image pixels, there may be brightness differences in the overlapping parts, such as Figure 6A As shown in the middle rectangle, this part can be transitioned by gradually entering and exiting. Figure 6A Simplified overlapping area diagram of Figure 6B As shown, the position coordinates of the overlapping part 2 are calculated. Let the gradient coefficient be b. According to the distance between each pixel in the overlapping area and the upper boundary of the overlapping part 2, b changes from 0 to 1, that is, the upper boundary b = 0, and the lower boundary b = 1. Let p1 represent the pixel value of image 1, p2 represent the pixel value of image 2, and the subscript i represent the i-th row of the overlapping area, that is, the upper boundary of the overlap i = 0, then the pixel value after fusion is:

[0092] pi=p1i×(1-b)+b×p2i;

[0093] By fading in and out, the pixel values ​​of image 1 are smoothly transitioned to the pixel values ​​of image 2.

[0094] In addition, in some feasible embodiments, the present application also includes some other fusion processing methods.

[0095] 1) Adjust the overall image and seamlessly splice it.

[0096] The key to this step is seamlessness. Grayscale information for the same area may vary between different images, and images of the same tissue may also vary significantly. For example, when using the FatSat-FSE sequence for fat suppression scanning, fat suppression over a wide range and off-center imaging are generally not effective. Off-center areas may have poor fat suppression, resulting in high fat signal on the image. This step aims to adjust the connection between the two stitched images and the overall image after stitching, ensuring a clearer and more seamless display.

[0097] 2) Adjust the image window width and window position to display the spliced ​​image

[0098] The window width and window position of images in different parts are different. Adjusting the window width and window position is conducive to clearer image display. This step sets the window width and window position of the spliced ​​image according to the window width and window position information of the image to be spliced ​​for better display.

[0099] In this example, we tried a variety of feature point extraction methods. Through simulation, we found that after extracting feature points, the feature point information of the same part of different images is also different, such as Figure 7A and 7B As shown, it can be seen that in Figure 7A The feature points in the rectangular box are Figure 7B The feature points within the rectangular boxes are not identical. These boxes represent overlapping areas. Although the scans represent the same tissue, variations in feature information may exist due to factors such as scanning time. Algorithms such as RANSAC (Random Sample Consensus) and BFMatcher (Brute Force Matching) can help optimize the registration matrix, but these registration matrices may still exhibit some deviation. Using precise registration methods can significantly improve this optimization.

[0100] In summary, the registration accuracy of this application is high, and the average deviation of the inliers is significantly reduced after fine registration. By optimizing the registration matrix, good registration effects can be obtained for slices far away from the center of the magnetic field.

[0101] like Figure 9 FIG. 1 is a block diagram of a device for automatically and accurately stitching magnetic resonance whole spine images according to an embodiment of the present invention. As shown in the figure, the device 900 includes:

[0102] A pre-processing module 901 acquires at least two images to be stitched and corresponding scanning information, pre-processes each of the images to be stitched, and determines one or more pairs of images to be stitched based on overlapping areas;

[0103] A coarse registration module 902 is used to extract image feature points of each pair of images to be stitched and perform coarse registration to obtain a corresponding registration matrix;

[0104] A fine registration module 903 is used to sample and process the image feature points and perform fine registration to optimize the registration matrix;

[0105] The fusion module 904 is configured to stitch the image pairs to be stitched together according to the optimized registration matrix and perform a fusion process on the stitched images.

[0106] It should be noted that the information interaction, execution process, etc. between the modules / units of the above-mentioned system are based on the same concept as the method embodiment described in this application, and the technical effects they bring are the same as those of the method embodiment of this application. For specific contents, please refer to the description in the method embodiment shown above in this application, and no further details will be given here.

[0107] It should also be noted that the division of the various modules of the above system is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into a single physical entity or physically separated. Furthermore, these units can be implemented entirely in software called by processing elements, or entirely in hardware. Alternatively, some modules can be implemented in software called by processing elements, while others can be implemented in hardware. For example, fusion module 904 can be a separate processing element, or it can be integrated into a chip of the above system. Furthermore, it can be stored in the memory of the above system in the form of program code, and called by a processing element of the above system to perform the functions of the above fusion module 904. The implementation of other modules is similar. Furthermore, these modules can be fully or partially integrated together, or implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each of the above modules can be completed by hardware integrated logic circuits in the processor element or by software instructions.

[0108] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented by scheduling program code through a processing element, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0109] like Figure 10As shown, it is a schematic diagram of the structure of a computer device in one embodiment of the present application. As shown in the figure, the computer device 1000 includes: a memory 1001 and a processor; the memory 1001 is used to store computer instructions; the processor 1002 executes the computer instructions to implement the following Figure 1 The method described.

[0110] In some embodiments, the number of the memory 1001 in the computer device 1000 can be one or more, the number of the processor 1002 can be one or more, and Figure 10 Take one as an example.

[0111] In one embodiment of the present application, the processor 1002 in the computer device 1000 will follow the following steps: Figure 1 The steps described above load one or more instructions corresponding to the application process into the memory 1001, and the processor 1002 runs the application stored in the memory 1001, thereby achieving the following Figure 1 The method described.

[0112] The memory 1001 may include random access memory (RAM) or non-volatile memory, such as at least one disk storage. The memory 1001 stores an operating system and operating instructions, executable modules, or data structures, or subsets thereof, or extended sets thereof, wherein the operating instructions may include various operating instructions for implementing various operations. The operating system may include various system programs for implementing various basic services and processing hardware-based tasks.

[0113] The processor 1002 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0114] In some specific applications, the various components of the computer device 1000 are coupled together via a bus system, wherein the bus system may include a power bus, a control bus, a status signal bus, etc. in addition to a data bus. However, for the sake of clarity, Figure 10 In Chinese, all kinds of buses are called bus systems.

[0115] In one embodiment of the present application, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following Figure 1 The method described.

[0116] At any possible level of technical detail combination, the present application may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present application.

[0117] Computer-readable storage media can be a tangible device that can hold and store the instructions used by the instruction execution device. Computer-readable storage media can be, for example, (but not limited to) an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. Computer-readable storage media used herein is not interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagated by waveguides or other transmission media (for example, light pulses by fiber optic cables), or electrical signals transmitted by wires.

[0118] The computer-readable program described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0119] The computer program instructions for performing the operation of the present application can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data or source code or object code written in any combination of one or more programming languages, wherein the programming language includes object-oriented programming languages ​​such as Smalltalk, C++, and procedural programming languages ​​such as "C" language or similar programming languages. Computer-readable program instructions can be executed completely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or executed completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer by any type of network including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (such as by using an Internet service provider to connect to the Internet). In certain embodiments, by utilizing the state information of computer-readable program instructions to personalize electronic circuits, such as programmable logic circuits, field programmable gate arrays (FPGAs) or programmable logic arrays (PLAs), the electronic circuits can execute computer-readable program instructions, thereby realizing various aspects of the present application.

[0120] In summary, the present application provides a method, device, equipment and medium for automatic and precise stitching of magnetic resonance full spine images, which obtains at least two images to be stitched and corresponding scanning information, preprocesses each of the images to be stitched, and determines one or more image pairs to be stitched based on the overlapping area; extracts the image feature points of each image pair to be stitched and performs coarse alignment to obtain a corresponding alignment matrix; samples the image feature points and performs fine alignment to optimize the alignment matrix; each image pair to be stitched is stitched according to the optimized alignment matrix, and the stitched images are fused.

[0121] This application effectively overcomes various shortcomings of the prior art and has high industrial utilization value.

[0122] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit the present invention. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by persons of ordinary skill in the art without departing from the spirit and technical concepts disclosed herein shall be encompassed by the claims of this application.

Claims

1. A method for automatic and accurate splicing of magnetic resonance whole spine images, characterized in that: The method comprises: Obtaining at least two images to be stitched and corresponding device scanning information, preprocessing each of the images to be stitched based on the information, and determining one or more pairs of images to be stitched based on overlapping areas; Extracting image feature points of each pair of images to be stitched and performing rough registration to obtain a corresponding registration matrix; extracting image feature point information of each pair of images to be stitched using a feature template to traverse and calculate feature points; the feature template can be set to different shapes according to image characteristics; The image feature points are sampled and precisely registered to optimize the registration matrix; the method includes: The image feature points of one image to be stitched in the pair of images to be stitched are used as a two-dimensional image point set to be registered, and the image feature points of the other image to be stitched are used as a target image point set; The two-dimensional image point set to be registered and the target image point set are sampled respectively; Calculating the position distances of the inner points and the outer points as well as the distances between the points respectively, and iteratively optimizing the registration matrix according to the ratio of the number of the inner points to the outer points and the average distance between the points; On this basis, the registration matrix of each layer and the overall FOV position information of the layer are combined to further optimize the registration matrix; The pairs of images to be stitched are stitched together according to the optimized registration matrix, and a fusion process is performed on the stitched images.

2. The method according to claim 1, characterized in that The image information includes any one or more combinations of image resolution, pixel resolution, image direction, patient position, image layer thickness, number of image layers and patient number.

3. The method according to claim 2, characterized in that The acquiring of at least two images to be stitched and corresponding device scanning information, and preprocessing the images to be stitched accordingly, includes: Determining the stitching scope and order of the images to be stitched; Unifying the pixel resolution and image resolution of each of the images to be stitched according to the image information and the device scanning information; Calculate the coordinate system corresponding to the sextant based on the image information and the device scanning information, rotate and translate the images to be stitched into a unified coordinate system, and obtain slice images of different body parts in the same slice by interpolation; The three-dimensional coordinates of all slice images in the bed coordinate system at the time of human body scanning are calculated based on the unified coordinate system, and the three-dimensional position overlapping area of ​​the images to be stitched is calculated based on the coordinate position of each part in the bed coordinate system to determine one or more pairs of images to be stitched.

4. The method according to claim 1, wherein The method comprises: After querying each feature point on the two-dimensional image point set to be registered through the registration matrix calculated by the current iteration, it is determined whether there is a nearest neighbor point on the target image point set; If a nearest neighbor exists, it is considered an interior point; if not, it is considered an exterior point.

5. The method according to claim 1, wherein The fusion process includes: Based on the inliers and outliers calculated during fine alignment, calculate the final inlier set and the center of the inliers; The weight is set according to the deviation of the distance between each inner point and the center of the two images to be registered; A template frame is used, and a matrix obtained by multiplying the template frame by the weight is slid across the intersection of the two images to be registered to complete the gradual correction of the boundary.

6. A device for automatically and accurately stitching magnetic resonance whole spine images, characterized in that: The device comprises: a preprocessing module configured to obtain at least two images to be stitched and corresponding device scanning information, preprocess each of the images to be stitched accordingly, and determine one or more pairs of images to be stitched based on overlapping areas; A coarse registration module is used to extract image feature points of each pair of images to be stitched and perform coarse registration to obtain a corresponding registration matrix; the extracted image feature point information of each pair of images to be stitched is calculated by traversing a feature template; the feature template can be set to different shapes according to image characteristics; a fine registration module for sampling and fine registration the image feature points to optimize the registration matrix; the method includes: using the image feature points of one image to be stitched in the pair of images to be stitched as a two-dimensional image point set to be registered, and the image feature points of the other image to be stitched as a target image point set; sampling the two-dimensional image point set to be registered and the target image point set separately; calculating the position distances between inliers and outliers and between points, and iteratively optimizing the registration matrix based on the ratio of the number of inliers and outliers and the average distance between points; and further optimizing the registration matrix based on this ratio by combining the registration matrix of each layer and the overall FOV position information of the layer; A fusion module is used to stitch the pairs of images to be stitched together according to the optimized registration matrix, and to perform fusion processing on the stitched images.

7. A computer device, characterized in that: The device comprises: a memory and a processor; the memory is used to store computer instructions; the processor executes the computer instructions to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that Computer instructions are stored, and when the computer instructions are executed, the method according to any one of claims 1 to 5 is performed.

Images