A method and system for organ reconstruction based on medical images
By using a multi-angle medical image data alignment method, the problems of high computational complexity and low accuracy of three-dimensional coordinates in existing technologies are solved, and efficient three-dimensional organ reconstruction is achieved.
Patent Information
- Application Number
- CN202411059874.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-05
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-08-05
AI Technical Summary
Existing technologies for reconstructing three-dimensional models of target organs based on two-dimensional ultrasound images suffer from high computational complexity and low accuracy of three-dimensional coordinates.
By acquiring multiple medical images of the target organ from multiple angles, a coarse 3D model is established based on the coordinate range. The image data is then aligned by identifying similar local areas and offsets, and the 3D reconstruction of the target organ is completed step by step.
It reduces image processing workload, improves reconstruction results, and enhances the accuracy of 3D models.
Smart Images

Figure CN118587366B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of medical imaging and data processing technology, and in particular to a method and system for organ reconstruction based on medical imaging. Background Art
[0002] As one of the important research fields of artificial intelligence technology, medical technology has played a huge role in medical imaging, clinical decision support, drug development, and pathology. The integration of medicine and technology can help doctors make diagnoses quickly and accurately. With the continuous updating and development of medical imaging equipment, for the same patient, multiple imaging technologies such as CT and MRI can be used to collect the patient's pathological information from multiple angles and aspects.
[0003] Some clinical procedures use complex computational methods to generate anatomical representations of organs such as heart chambers. For example, U.S. Patent Application Publication No. 2014 / 0152653 describes a method for processing two-dimensional ultrasound images obtained from an intracardiac ultrasound imaging catheter, which provides improved image quality and can generate three-dimensional composite images of the heart, but the scheme has high computational complexity.
[0004] By performing ultrasound scanning on the target organ, a three-dimensional model of the target organ can be reconstructed based on the two-dimensional ultrasound image. However, the accuracy of the three-dimensional coordinates of the points in the three-dimensional model reconstructed by the current technology is low. Summary of the invention
[0005] The embodiments of the present application provide a method and system for organ reconstruction based on medical images, which are used to complete three-dimensional reconstruction of a target organ based on medical images, so as to reduce the amount of image calculation and improve the reconstruction effect.
[0006] The present application embodiment proposes a method for organ reconstruction based on medical images, comprising:
[0007] Acquire multiple medical image data of the target organ at multiple angles, and determine the coordinate range of the target organ in the coordinate system based on the multiple medical image data;
[0008] According to the plurality of medical image data, establishing a rough three-dimensional model of the target organ based on the coordinate range;
[0009] Identifying a plurality of reference positions from the plurality of medical image data, wherein the reference positions include similar first local areas existing in at least two medical image data;
[0010] For any identified reference position, selecting a medical image containing the reference position as a reference medical image, and aligning the image with the rough three-dimensional model;
[0011] For other medical images including any one of the reference positions, matching is performed with the other medical images and any one of the reference positions in the benchmark medical image as centers, so as to match a similar second local area;
[0012] determining, according to the second local area, an offset between the other medical image and the reference medical image;
[0013] Aligning the other medical images on the rough three-dimensional model according to the offset, any one of the reference positions, and a contour boundary of the rough three-dimensional model;
[0014] The multiple medical image data acquired one by one are aligned based on the reference position to complete the reconstruction of the target organ.
[0015] Optionally, the plurality of medical image data of the target organ at multiple angles are obtained under the same shooting parameters;
[0016] According to the plurality of medical image data, establishing a rough three-dimensional model of the target organ based on the coordinate range includes:
[0017] Determine the maximum distance in the three-dimensional direction and the contour data at the corresponding angle from the plurality of medical image data;
[0018] A rough three-dimensional model of the target organ is constructed based on the determined maximum distance value and contour data in the three-dimensional direction and in combination with the empirical model of the target organ.
[0019] Optionally, identifying a plurality of reference positions from the plurality of medical image data includes:
[0020] Segmenting multiple medical image data to obtain multiple image segmentations;
[0021] Extract local contour data from multiple image segmentations and calculate the similarity between the extracted contour data;
[0022] When the similarity between the extracted contour data is greater than a first preset similarity threshold, it is determined as suspected similar image segmentation;
[0023] Determine the similarity between the local contour data of the image segmentation adjacent to any suspected similar image segmentation, and determine that any suspected similar image segmentation is a similar image segmentation when the proportion of adjacent image segmentations with a similarity greater than a second similarity threshold is greater than a specified proportion;
[0024] From the determined similar image segmentations, a plurality of reference positions are determined based on the local contour data of the plurality of image segmentations with the greatest similarity.
[0025] Optionally, from among the determined similar image segmentations, based on the contour data of several image segmentations with the greatest similarity, determining several reference positions includes:
[0026] Determine multiple nodes in the contour data of any two image segments with the greatest similarity;
[0027] According to the line similarity of contour lines covered by any node and the number of contour lines, any node is determined as a reference position, wherein the higher the line similarity of contour lines covered by any node and the greater the number, the greater the probability of being selected as a reference position.
[0028] Optionally, for any identified reference position, selecting a medical image containing the any reference position as a reference medical image includes:
[0029] Prioritizing selecting a medical image having a smaller number of reference positions between the medical image and the reference medical image, and performing an alignment step; and,
[0030] The reference positions in the aligned reference medical image are eliminated, and other medical images associated with the reference medical image and containing a small number of reference positions are selected one by one to perform the alignment step until the reconstruction of the target organ is completed.
[0031] Optionally, performing matching with the other medical image and any reference position in the benchmark medical image as the center to match a similar second local area includes:
[0032] Selecting the image segment with the greatest similarity outside the preset pixel distance from the neighboring image segments whose similarity is greater than the second similarity threshold, and determining whether the similarity meets the alignment requirement;
[0033] If the selected similarity does not meet the alignment requirement, performing secondary segmentation on the image segmentation with the maximum similarity value, and determining whether the secondary segmentation meets the alignment requirement;
[0034] The image segments that meet the alignment requirements are matched one by one as the second local area.
[0035] Optionally, determining, according to the second local area, an offset between the other medical image and the reference medical image includes:
[0036] Determining a proportional relationship between the reference positions in the reference medical image and the other medical images, and between the second local areas;
[0037] Establishing a mathematical relationship based on the proportional relationship and the pixel distance between the reference position and the second local area to calculate the offset;
[0038] When the image segmentation with the greatest similarity outside the preset pixel distance cannot obtain the second local area that meets the alignment requirement, the offset is calculated based on the contour boundary of the rough three-dimensional model.
[0039] Optionally, aligning the other medical images on the rough three-dimensional model according to the offset, any one of the reference positions, and a contour boundary of the rough three-dimensional model includes:
[0040] Using the contour boundary of the rough three-dimensional model as a constraint, rotating or moving the other medical images based on any reference position according to the offset to align them on the rough three-dimensional model; and
[0041] When the other medical images are directly aligned with the contour boundary of the rough three-dimensional model based on any one of the reference positions, the position of the second local area is retained.
[0042] An embodiment of the present application further proposes a medical image-based organ reconstruction system, comprising a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the steps of the aforementioned medical image-based organ reconstruction method are implemented.
[0043] The method of the embodiment of the present application completes the alignment of the image of the target organ in the rough three-dimensional model based on the medical images of the target organ acquired from multiple angles. After the alignment, the three-dimensional reconstruction of the target organ can be completed. The method of the present application can reduce the amount of image calculation and improve the reconstruction effect.
[0044] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:
[0046] Figure 1 The following is a schematic diagram of the basic process of the medical image-based organ reconstruction method of this embodiment. DETAILED DESCRIPTION
[0047] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.
[0048] The present application embodiment proposes a method for organ reconstruction based on medical images, such as Figure 1 As shown, the following steps are included:
[0049] In step S101, multiple medical image data of the target organ at multiple angles are obtained, and the coordinate range of the target organ in the coordinate system is determined based on the multiple medical image data. In some examples, multi-modal medical images of the target organ can be obtained, and after multi-modal registration, the coordinate range of the target organ is determined in a unified three-dimensional coordinate system based on the multiple medical image data.
[0050] In step S102, a rough 3D model of the target organ is established based on the coordinate range according to the plurality of medical image data. According to the coordinate range determined above, the contour of the rough 3D model can be supplemented with the contour detected by ultrasound or the 3D contour structure of a typical target organ.
[0051] In step S103, a plurality of reference positions are identified from the plurality of medical image data, wherein the reference positions include similar first local areas existing in at least two medical image data.
[0052] In an embodiment of the present application, for medical imaging data of a target organ acquired from multiple angles, there are partially overlapping areas within the acquired organs, or the same tissue or structure within the target organ is acquired. The embodiment of the present application identifies the partially overlapping areas, or the same tissue or structure, and completes the alignment and three-dimensional reconstruction steps. Please refer to the subsequent embodiments for details.
[0053] In step S104, for any identified reference position, a medical image containing the reference position is selected as a reference medical image and aligned with the rough 3D model. In some examples, the reference medical image can be based on the maximum image containing the target organ in a certain direction, which is most likely to have the same structure or tissue as other medical images, and the reference medical image is directly aligned to the maximum value of the rough 3D model in this direction.
[0054] In step S105, other medical images including any of the reference positions are matched with the other medical images and any of the reference positions in the reference medical image as centers to match a similar second local area. In the embodiment of the present application, a second local area of partially overlapping or identical tissue or structure between the other image and the reference medical image is further matched.
[0055] In step S106, the offset between the other medical image and the reference medical image is determined according to the second local area. According to the above example, the offset is determined by combining the second local area with any reference position.
[0056] In step S107, the other medical images are aligned with the rough three-dimensional model according to the offset, any one of the reference positions, and the contour boundary of the rough three-dimensional model. In some examples, if the second local area is a non-offset structure or tissue actually contained in the organ, the contour boundary range of the rough three-dimensional model is corrected after the alignment is completed according to the offset and any one of the reference positions.
[0057] In step S108, the acquired multiple medical image data are aligned based on the reference position to complete the reconstruction of the target organ.
[0058] The method of the embodiment of the present application completes the alignment of the image of the target organ in the rough three-dimensional model based on the medical images of the target organ acquired from multiple angles. After the alignment, the three-dimensional reconstruction of the target organ can be completed. The method of the present application can reduce the amount of image calculation and improve the reconstruction effect.
[0059] In some embodiments, the plurality of medical image data of the target organ at multiple angles are obtained under the same shooting parameters; and establishing a rough three-dimensional model of the target organ based on the coordinate range according to the plurality of medical image data includes:
[0060] Determine the maximum distance value in the three-dimensional direction and the contour data at the corresponding angle from the plurality of medical image data. In some examples, the maximum distance value may be, for example, the maximum distance value in multiple directions such as horizontal and vertical directions of the plurality of medical image data.
[0061] According to the determined three-dimensional distance maximum and contour data, combined with the empirical model of the target organ, a rough three-dimensional model of the target organ is constructed. In the case of having ultrasound contour data, the three-dimensional model of the target organ can be constructed directly based on the ultrasound data.
[0062] In some embodiments, identifying a plurality of reference positions from the plurality of medical image data comprises:
[0063] The plurality of medical image data are segmented to obtain a plurality of image segmentations. For example, the medical image can be segmented according to a set segmentation specification.
[0064] Extract local contour data from multiple image segmentations and calculate the similarity between the extracted contour data. The specific method for calculating the similarity can use methods such as PSNR, MSE, and cosine similarity.
[0065] When the similarity between the extracted contour data is greater than a first preset similarity threshold, it is determined to be a suspected similar image segmentation.
[0066] Determine the similarity between the local contour data of the image segmentation adjacent to any suspected similar image segmentation, and determine that any suspected similar image segmentation is a similar image segmentation when the proportion of adjacent image segmentations with a similarity greater than a second similarity threshold is greater than a specified proportion. In some implementations, the second similarity threshold can be set smaller than the first similarity threshold, for example, the second similarity threshold can be set to 70%, and the first similarity threshold can be set to 90%. By calculating the proportion of adjacent image segmentations with a similarity greater than 70%, the approximate alignment direction and range can be determined, which is convenient for further matching of the second position for alignment.
[0067] From the determined similar image segmentations, a plurality of reference positions are determined based on the local contour data of the plurality of image segmentations with the greatest similarity.
[0068] In some embodiments, determining a plurality of reference positions based on contour data of a plurality of image segmentations with the greatest similarity from among the determined similar image segmentations includes:
[0069] Determine multiple nodes in the contour data of any two image segmentations with the greatest similarity. Specifically, any two image segmentations may be sorted according to their similarity, and the two segmentations with the greatest similarity may be selected to further determine the contour in the segmentation, thereby obtaining multiple nodes contained in the contour.
[0070] According to the line similarity of the contour lines covered by any node and the number of contour lines, any node is determined as a reference position, wherein the higher the line similarity of the contour lines covered by any node and the greater the number, the greater the probability of being selected as a reference position. The contour lines covered by the nodes referred to in the implementation of this application refer to the lines extending outward from the nodes. In some examples, the relevant lines represent the outward extension of the structure or organization of the node position. According to the line similarity and the number of contour lines, the higher the similarity, the more likely the two nodes are the same structure or organization, and the greater the number, the reflection of another aspect of the same structure or organization, and therefore it is determined as a reference position based on this.
[0071] In some embodiments, for any identified reference position, selecting a medical image containing the any reference position as a reference medical image includes:
[0072] Prioritizing selecting a medical image having a smaller number of reference positions between the medical image and the reference medical image, and performing an alignment step; and,
[0073] The reference positions in the aligned reference medical image are eliminated, and other medical images associated with the reference medical image and containing a small number of reference positions are selected one by one to perform the alignment step until the reconstruction of the target organ is completed. In some embodiments, the alignment is performed preferentially on the medical images with a small number of reference positions, and the other aligned medical images are used as auxiliary to determine the reference positions of the subsequently aligned medical images, thereby improving the alignment speed on the one hand, and reducing the number of reference positions on the other hand, which can reduce the amount of image processing operations.
[0074] In some embodiments, taking the other medical image and any reference position in the reference medical image as the center, matching to match a similar second local area includes:
[0075] From the neighboring image segmentations whose similarity is greater than the second similarity threshold, select the image segmentation with the greatest similarity outside the preset pixel distance, and judge whether the similarity meets the alignment requirement. Based on the aforementioned neighboring image segmentation, it provides a search direction for determining the second local area. The embodiment of the present application further sets a shortest pixel distance for the convenience of alignment. Outside the shortest pixel distance, select the image segmentation with the greatest similarity. For example, if the similarity required by the alignment is also 90%, it is judged that the alignment requirement is met.
[0076] If the selected similarity does not meet the alignment requirement, the image segmentation with the maximum similarity is segmented twice, and it is determined whether the secondary segmentation meets the alignment requirement. Based on the above, for the segmentation that does not meet the alignment requirement, a secondary segmentation is performed.
[0077] The image segments that meet the alignment requirements are matched one by one as the second local area. For the secondary segmentation, the secondary segmentation area that meets the conditions is matched as the second local area.
[0078] In some embodiments, determining, according to the second local area, an offset between the other medical image and the reference medical image comprises:
[0079] Determine the proportional relationship between the reference positions in the reference medical image and the other medical images, and between the second local areas. In some examples, the proportional relationship of the present application may be, for example, a proportional relationship between pixel areas.
[0080] Based on the proportional relationship and the pixel distance between the reference position and the second local area, a mathematical relationship is established to calculate the offset. Specifically, the offset pixel distance can be calculated as the offset based on the image similarity principle and the corresponding pixel distance.
[0081] When the image segmentation with the greatest similarity outside the preset pixel distance cannot obtain the second local area that meets the alignment requirement, the offset is calculated based on the contour boundary of the rough three-dimensional model.
[0082] In some embodiments, aligning the other medical images on the rough three-dimensional model according to the offset, any one of the reference positions, and the contour boundary of the rough three-dimensional model includes:
[0083] Using the contour boundary of the rough three-dimensional model as a constraint, rotating or moving the other medical images based on any reference position according to the offset to align them on the rough three-dimensional model; and
[0084] In the case where the other medical images are directly aligned with the contour boundary of the rough three-dimensional model based on any reference position, the position of the second local area is retained. After completing the alignment of multiple medical image data, for the vacant areas inside the three-dimensional model, each identical tissue or structure can be fitted at the corresponding position, and other areas can be filled according to empirical data to obtain a complete three-dimensional reconstruction of the target organ.
[0085] An embodiment of the present application further proposes a medical image-based organ reconstruction system, comprising a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the steps of the aforementioned medical image-based organ reconstruction method are implemented.
[0086] In addition, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present disclosure with equivalent elements, modifications, omissions, combinations (e.g., various embodiments intersecting schemes), adaptations or changes. It is not limited to the examples described in this specification or during the implementation of this application, and its examples will be interpreted as non-exclusive.
[0087] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more of them) can be used in combination with each other. For example, those of ordinary skill in the art may use other embodiments when reading the above description.
[0088] The above embodiments are merely exemplary embodiments of the present disclosure. Those skilled in the art may make various modifications or equivalent substitutions to the present invention within the essence and protection scope of the present disclosure, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present invention.
Claims
1. A method for organ reconstruction based on medical images, characterized in that: include: Acquire multiple medical image data of the target organ at multiple angles, and determine the coordinate range of the target organ in the coordinate system based on the multiple medical image data; According to the plurality of medical image data, establishing a rough three-dimensional model of the target organ based on the coordinate range; Identifying a plurality of reference positions from the plurality of medical image data, wherein the reference positions include similar first local areas existing in at least two medical image data; For any identified reference position, selecting a medical image containing the reference position as a reference medical image, and aligning the image with the rough three-dimensional model; For other medical images including any one of the reference positions, matching is performed with the other medical images and any one of the reference positions in the benchmark medical image as centers, so as to match a similar second local area; determining, according to the second local area, an offset between the other medical image and the reference medical image; Aligning the other medical images on the rough three-dimensional model according to the offset, any one of the reference positions, and a contour boundary of the rough three-dimensional model; Sequentially aligning the acquired multiple medical image data based on the reference position to complete the reconstruction of the target organ; From the plurality of medical image data, identifying a number of reference positions includes: Segmenting multiple medical image data to obtain multiple image segmentations; Extract local contour data from multiple image segmentations and calculate the similarity between the extracted contour data; When the similarity between the extracted contour data is greater than a first preset similarity threshold, it is determined as suspected similar image segmentation; Determine the similarity between the local contour data of the image segmentation adjacent to any suspected similar image segmentation, and determine that any suspected similar image segmentation is a similar image segmentation when the proportion of adjacent image segmentations with a similarity greater than a second similarity threshold is greater than a specified proportion; From the determined similar image segments, a plurality of reference positions are determined based on local contour data of a plurality of image segments with the greatest similarity; Taking the other medical image and any reference position in the benchmark medical image as the center, matching to match a similar second local area includes: Selecting the image segment with the greatest similarity outside the preset pixel distance from the neighboring image segments whose similarity is greater than the second similarity threshold, and determining whether the similarity meets the alignment requirement; If the selected similarity does not meet the alignment requirement, performing secondary segmentation on the image segmentation with the maximum similarity value, and determining whether the secondary segmentation meets the alignment requirement; The image segments that meet the alignment requirements are matched one by one as the second local area.
2. The organ reconstruction method based on medical images according to claim 1, characterized in that: The plurality of medical image data of the target organ at multiple angles are obtained under the same shooting parameters; According to the plurality of medical image data, establishing a rough three-dimensional model of the target organ based on the coordinate range includes: Determine the maximum distance in the three-dimensional direction and the contour data at the corresponding angle from the plurality of medical image data; A rough three-dimensional model of the target organ is constructed based on the determined maximum distance value and contour data in the three-dimensional direction and in combination with the empirical model of the target organ.
3. The organ reconstruction method based on medical images according to claim 1, characterized in that: From the determined similar image segments, based on the contour data of several image segments with the greatest similarity, several reference positions are determined, including: Determine multiple nodes in the contour data of any two image segments with the greatest similarity; According to the line similarity of contour lines covered by any node and the number of contour lines, any node is determined as a reference position, wherein the higher the line similarity of contour lines covered by any node and the greater the number, the greater the probability of being selected as a reference position.
4. The organ reconstruction method based on medical images according to claim 3, characterized in that: For any identified reference position, selecting a medical image containing the reference position as a reference medical image includes: Prioritizing the selection of a medical image that has the least number of reference locations with the reference medical image, and performing an alignment step; and, The reference positions in the aligned reference medical image are eliminated, and other medical images associated with the reference medical image and containing the least number of reference positions are selected one by one to perform the alignment step until the reconstruction of the target organ is completed.
5. The organ reconstruction method based on medical images according to claim 1, characterized in that: Determining, according to the second local area, an offset between the other medical image and the reference medical image includes: Determining a proportional relationship between the reference positions in the reference medical image and the other medical images, and between the second local areas; Establishing a mathematical relationship based on the proportional relationship and the pixel distance between the reference position and the second local area to calculate the offset; When the image segmentation with the greatest similarity outside the preset pixel distance cannot obtain the second local area that meets the alignment requirement, the offset is calculated based on the contour boundary of the rough three-dimensional model.
6. The organ reconstruction method based on medical images according to claim 5, characterized in that: Aligning the other medical images on the rough three-dimensional model according to the offset, any one of the reference positions, and the contour boundary of the rough three-dimensional model includes: Using the contour boundary of the rough three-dimensional model as a constraint, rotating or moving the other medical images based on any reference position according to the offset to align them on the rough three-dimensional model; and When the other medical images are directly aligned with the contour boundary of the rough three-dimensional model based on any one of the reference positions, the position of the second local area is retained.
7. An organ reconstruction system based on medical images, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the organ reconstruction method based on medical images as described in any one of claims 1 to 6 are implemented.
