Medical image intelligent interpolation method and device

Abstract

The application discloses a medical image slice interpolation and padding method and device, and relates to the medical treatment technical field.The method comprises the following steps: segmenting a medical image of a case by using a deep learning segmentation network model to obtain image segmentation slices of the medical image; locating missing segmentation slices in the image segmentation slices of the medical image; performing preliminary interpolation and padding processing on the missing segmentation slices to obtain preliminary padding segmentation slices; judging whether the preliminary padding segmentation slices meet preset conditions; if yes, the medical image slice interpolation and padding processing is ended; if not, the missing segmentation slices enter an iterative interpolation and padding processing flow; judging whether iteration padding segmentation slices obtained by each iteration in the iterative interpolation and padding processing flow meet preset conditions; if yes, the iteration is ended; if not, the iteration is continued.The application improves the accuracy of medical image segmentation slice padding, makes the padding segmentation slices more in line with physiological rules, and more accurately covers organs, which helps to improve the diagnosis efficiency of doctors.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, specifically to a method and apparatus for interpolating and completing medical image slices. Background Technology

[0002] In the field of CT image segmentation, the main segmentation methods are manual annotation by physicians, traditional algorithm segmentation, and deep learning segmentation. Manual annotation by physicians requires a significant investment of time and effort from professional physicists, and due to its high time and economic costs, it is gradually being abandoned. The drawback of traditional algorithms is their heavy reliance on the edges and internal features of organs. However, organs in CT imaging vary greatly across different slices, making it difficult to achieve good segmentation results using a single traditional algorithm. Deep learning segmentation is currently the most commonly used automatic segmentation method in CT segmentation, and it also offers the best segmentation results with the fewest missing slices. Through learning from large amounts of data, this method exhibits better robustness.

[0003] However, most existing deep learning segmentation methods are data-driven, and these methods often suffer from segmentation errors, such as incomplete boundary segmentation or missing organ slices. In practical applications, segmentation algorithms often struggle to cover every slice. The edges and shapes of some small organs are not clearly defined, leading to omissions in many slices. This can cause errors in doctors' observation of patients' conditions, affecting their judgment. Existing methods mostly use linear, bilinear, or trilinear interpolation to fill in the missing slices, but this method has low accuracy. Simple interpolation often covers areas outside the delineated organs and lesions, affecting the visualization of the patient's condition.

[0004] In view of this, this invention patent is hereby proposed. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a method and apparatus for interpolating and completing medical image slices, specifically employing the following technical solution:

[0006] A method for interpolating and completing medical image slices, comprising:

[0007] A deep learning segmentation network model is used to segment medical images of medical cases, resulting in image segmentation slices of medical images;

[0008] Locate the missing segment in the image segmentation slice of a medical image, and simultaneously obtain the complete segmentation slices on both sides before and after the missing segment;

[0009] Preliminary interpolation and imputation processing is performed on the missing segment to obtain the preliminary imputed segment;

[0010] Determine whether the preliminary completed segmentation slices meet the preset conditions. If they do, the medical image slice interpolation and completion process ends. If they do not meet the conditions, proceed to the missing segmentation slice iterative interpolation and completion process.

[0011] In the iterative interpolation completion process, each iteration of the iteratively completed segmentation is used to determine whether it meets the preset conditions. If it does, the iteration ends; otherwise, the iteration continues.

[0012] As an optional embodiment of the present invention, in a medical image slice interpolation and completion method of the present invention, the iterative interpolation and completion process includes: performing SDF-based interpolation and completion processing on the initially completed segmented slice to obtain iteratively completed segmented slices.

[0013] Optionally, the step of performing SDF-based interpolation completion processing on the initially completed segmented slices to obtain iteratively completed segmented slices includes:

[0014] The complete segmented slices Xi-1 and Xi+1 on both sides before and after the initial completed segmented slice Xi are transformed into directed distance maps Mi-1 and Mi+1 by SDF transformation. The Lerp interpolation function is used on the directed distance map to obtain the interpolated completed distance map Mi. The inverse transformation is performed on the interpolated completed distance map Mi to obtain the iteratively completed segmented slice.

[0015] As an optional embodiment of the present invention, in a medical image slice interpolation and completion method of the present invention, the step of determining whether the iteratively completed segmented slice meets the preset conditions after each iteration processing includes:

[0016] Principal component analysis (PCA) feature extraction is performed based on iteratively completed segmentation slices and complete segmentation slices on both sides, extracting the completion features in the iteratively completed segmentation slices and the initial features corresponding to the completion features in the complete segmentation slices on both sides.

[0017] Calculate the cosine similarity between the completed feature and the initial feature, and compare it with a preset threshold;

[0018] If the calculated cosine similarity is less than or equal to the preset threshold, the iteration ends; if the calculated cosine similarity is greater than the preset threshold, the iteration continues.

[0019] As an optional embodiment of the present invention, in a medical image slice interpolation and completion method of the present invention, the preliminary interpolation and completion processing for the missing segment slice to obtain the preliminary completed segment slice includes:

[0020] SIFT feature point matching is performed between the missing segmentation slice and the complete segmentation slices on the front and back sides.

[0021] A perspective transformation is performed, and the intersection of the mask results of the complete segmented slices on the front and back sides is mapped to the missing segmented slice to obtain the preliminary completed segmented slice.

[0022] As an optional embodiment of the present invention, in a medical image slice interpolation and completion method of the present invention, the step of locating the missing segment in the image segmentation slice of the medical image and simultaneously obtaining the complete segmentation slices on both sides before and after the missing segment includes:

[0023] If the located missing segment is a segment of the first image or a segment of the last image, then obtain the complete segment behind or the complete segment before the missing segment.

[0024] In the preliminary interpolation and completion process for missing segments, the perspective transformation result of the complete segment slices before or after the missing segment slice is directly mapped to the missing segment slice to obtain the preliminary completed segment slice.

[0025] As an optional embodiment of the present invention, in a medical image slice interpolation and completion method of the present invention, the step of determining whether the initially completed segmented slice meets the preset conditions includes:

[0026] Principal component analysis (PCA) feature extraction is performed based on the preliminary completed segmentation slices and the complete segmentation slices on the front and back sides or both sides. The completed features in the preliminary completed segmentation slices and the initial features corresponding to the completed features in the complete segmentation slices on the front and back sides or both sides are extracted.

[0027] Calculate the cosine similarity between the completed feature and the initial feature, and compare it with a preset threshold;

[0028] If the calculated cosine similarity is less than or equal to the preset threshold, the medical image slice interpolation and completion process ends. If the calculated cosine similarity is greater than the preset threshold, the iterative interpolation and completion process for missing segmented slices begins.

[0029] As an optional embodiment of the present invention, in a medical image slice interpolation and completion method of the present invention, the missing segmented slices in the image segmentation slices of the medical image include:

[0030] Obtain the category matrix of each organ in the image segmentation slices of medical images;

[0031] Based on the segmentation order of the three-dimensional medical images, determine whether there are corresponding segmentation results for the two images before and after the current image segmentation slice B, namely image segmentation slices A and C.

[0032] If image segmentation slices A and C have segmentation results, but the current image segmentation slice B has no segmentation results, then the current image segmentation slice B is a missing segmentation slice of the corresponding category.

[0033] As an optional embodiment of the present invention, a method for interpolating and completing medical image slices according to the present invention includes:

[0034] After locating the missing segment in the image segmentation slice of the medical image and obtaining the complete segmentation slices on both sides before and after the missing segmentation slice, select the minimum bounding rectangle of the complete segmentation slices on both sides before and after the missing segmentation slice.

[0035] During the initial interpolation and completion process for missing segmented slices, the segmented slice image within the minimum bounding rectangle is processed.

[0036] In the iterative interpolation completion process for missing segmented slices, the segmented slice image within the minimum bounding rectangle is processed.

[0037] As an optional embodiment of the present invention, in a medical image slice interpolation and completion method of the present invention, the minimum bounding rectangle of the complete slices on both sides corresponding to the missing slice includes:

[0038] When the minimum bounding rectangles of the complete segments on the front and back sides of the missing segment are not equal, the smaller minimum bounding rectangle is selected.

[0039] This invention also provides an apparatus for implementing the medical image slice interpolation and completion method, comprising:

[0040] The medical image segmentation module uses a deep learning segmentation network model to segment medical images of medical cases, obtaining image segmentation slices of medical images;

[0041] The missing segmentation slice localization module locates the missing segmentation slice in the image segmentation slices of medical images, and simultaneously acquires the complete segmentation slices on both sides before and after the missing segmentation slice.

[0042] The preliminary interpolation and completion module performs preliminary interpolation and completion processing on the missing segment slices to obtain preliminary completed segment slices. It then determines whether the preliminary completed segment slices meet the preset conditions. If they do, the medical image slice interpolation and completion processing ends; otherwise, it enters the iterative interpolation and completion processing flow for the missing segment slices.

[0043] The iterative interpolation completion module executes the iterative interpolation completion processing flow. For each iteration of the iteratively completed segmentation, it determines whether a preset condition is met. If the condition is met, the iteration ends; otherwise, it continues. Compared with the prior art, the beneficial effects of this invention are:

[0044] This invention discloses a medical image slice interpolation and completion method, which is an adaptive interpolation method for completing missing organ slices. The basic idea is to perform preliminary interpolation and completion processing on the missing slice and the two adjacent unmissing slices, followed by iterative interpolation and completion processing based on the missing slice. The results of the iterative interpolation and completion processing are compared with preset conditions until the preset conditions are met, at which point the iteration ends. In this embodiment, the medical image slice interpolation and completion method already has a high confidence level after the preliminary interpolation and completion processing. Then, the missing slice undergoes iterative interpolation and completion processing followed by iterative interpolation calculations, resulting in more accurate interpolation results and smoother completion.

[0045] Therefore, the medical image slice interpolation and completion method of the present invention improves the accuracy of slice completion, makes the completed slices more in line with physiological laws, and the completion results more accurately cover organs, while also improving the diagnostic efficiency of doctors. Attached image description:

[0046] Figure 1 A flowchart illustrating a specific example of a medical image slice interpolation and completion method according to an embodiment of the present invention;

[0047] Figure 2 A flowchart illustrating a specific example of the preliminary interpolation and completion process in a medical image slice interpolation and completion method according to an embodiment of the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0049] Therefore, the following detailed description of embodiments of the present invention is not intended to limit the scope of the claimed invention, but merely illustrates some embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0050] It should be noted that, unless otherwise specified, the embodiments and features and technical solutions in the embodiments of the present invention can be combined with each other.

[0051] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0052] In the description of this invention, it should be noted that the terms "upper," "lower," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of this invention is in use, or the orientation or positional relationship commonly understood by those skilled in the art. These terms are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention. In addition, the terms "first," "second," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0053] This embodiment of a medical image slice interpolation and completion method includes:

[0054] A deep learning segmentation network model is used to segment medical images of medical cases, resulting in image segmentation slices of medical images;

[0055] Locate the missing segment in the image segmentation slice of a medical image, and simultaneously obtain the complete segmentation slices on both sides before and after the missing segment;

[0056] Preliminary interpolation and imputation processing is performed on the missing segment to obtain the preliminary imputed segment;

[0057] Determine whether the preliminary completed segmentation slices meet the preset conditions. If they do, the medical image slice interpolation completion process ends. If they do not meet the conditions, proceed to the iterative interpolation completion process for the missing segmentation slices.

[0058] In the iterative interpolation completion process, each iteration of the iteratively completed segmentation is used to determine whether it meets the preset conditions. If it does, the iteration ends; otherwise, the iteration continues.

[0059] This embodiment presents a medical image slice interpolation and completion method, which is an adaptive interpolation method for completing missing organ slices. The basic idea is to perform preliminary interpolation and completion processing on the missing segment and the two preceding unmissing segments. Then, iterative interpolation and completion processing is performed based on the missing segment. The results of the iterative interpolation and completion processing are compared with preset conditions until the preset conditions are met, at which point the iteration ends. In this embodiment, the confidence level of the preliminary interpolation and completion results is already high. Then, the missing segment undergoes iterative interpolation and completion processing followed by iterative interpolation calculations, making the interpolation results more accurate and the completion smoother.

[0060] Therefore, the medical image slice interpolation and completion method of this embodiment improves the accuracy of slice completion, makes the completed slices more in line with physiological laws, and the completion results more accurately cover organs, while also improving the diagnostic efficiency of doctors.

[0061] As an optional implementation of this embodiment, in a medical image slice interpolation and completion method of this embodiment, the iterative interpolation and completion process includes: performing SDF-based interpolation and completion processing on the initially completed segmented slice to obtain iteratively completed segmented slices.

[0062] Optionally, the step of performing SDF-based interpolation completion processing on the initially completed segmented slices to obtain iteratively completed segmented slices includes:

[0063] The complete segmented slices Xi-1 and Xi+1 on both sides before and after the initial completed segmented slice Xi are transformed into directed distance maps Mi-1 and Mi+1 by SDF transformation. The Lerp interpolation function is used on the directed distance map to obtain the interpolated completed distance map Mi. The inverse transformation is performed on the interpolated completed distance map Mi to obtain the iteratively completed segmented slice.

[0064] SDF stands for Signed Distance Field, which is generally translated as Directed Distance Field. It is developed from UDF (Unsigned Distance Field). UDF represents the distance to points inside an object as 0, and the distance to points outside the object as a positive number, storing the distance to the nearest object. SDF (Signed Distance Field) adds a positive and negative relationship to UDF to indicate whether the distance is inside or outside the object.

[0065] The properties of the directed distance graph obtained by the SDF transformation map it to [0,1]. After mapping it, 0.5 represents the boundary of the object. We can use this property to perform an inverse transformation, and finally obtain the iterative completion segmentation of the interpolation completion.

[0066] The iterative interpolation and completion process in this embodiment uses the 8SSEDT algorithm to generate a two-dimensional directed distance map from a grayscale image, i.e., the SDF transformation. The 8SSEDT algorithm, or 8-point signed sequential distance transform, has a time complexity of O(N). It obtains the directed distance map of the slice by scanning the slice twice, from top to bottom and from bottom to top.

[0067] The Lerp interpolation function is a linear mapping that predicts the directed distance map of the N missing intermediate segmentation maps using two consecutive slice segmentation maps.

[0068] The characteristic of the iterative interpolation completion process in this embodiment is that after the model outputs the initially completed segment, it does not directly perform interpolation completion on it. Instead, it converts the initially completed segment into a directed distance map and performs interpolation completion in the directed distance domain. This invention improves the accuracy of missing segment completion and makes the completed segment more in line with physiological laws, while also improving the diagnostic efficiency of doctors.

[0069] Therefore, the iterative interpolation and completion process in the medical image slice interpolation and completion method of this embodiment has the following technical effects:

[0070] 1) An important property of the two-dimensional directed distance map after mapping to [0,1] is that 0.5 represents the boundary of the object, that is, the segmentation edge. Moreover, the graphics card can perform linear mapping at the hardware level, which can ensure that no matter how many times the directed distance map is magnified, as long as the threshold is set to 0.5, the accuracy of the segmentation edge can be guaranteed.

[0071] 2) The advantage of performing SDF transformation on the image before interpolation is that it more clearly represents the relationship between the image pixels and the segmentation edges, making the interpolation smoother than performing interpolation directly.

[0072] 3) The inverse SDF transform can be easily implemented by setting a threshold of 0.5 based on the properties of the directed distance graph, with low computational complexity.

[0073] Meanwhile, in a medical image slice interpolation and completion method of this embodiment, the step of determining whether the iteratively completed segmented slice meets the preset conditions after each iteration includes:

[0074] Principal component analysis (PCA) feature extraction is performed based on iteratively completed segmentation slices and complete segmentation slices on both sides, extracting the completion features in the iteratively completed segmentation slices and the initial features corresponding to the completion features in the complete segmentation slices on both sides.

[0075] Calculate the cosine similarity between the completed feature and the initial feature, and compare it with a preset threshold;

[0076] If the calculated cosine similarity is less than or equal to the preset threshold, the iteration ends; if the calculated cosine similarity is greater than the preset threshold, the iteration continues.

[0077] In this embodiment of a medical image slice interpolation and completion method, the missing segment slice and the non-missing segment slice are subjected to PCA calculation, and then the cosine similarity is calculated as the marker to end the iteration. The advantage of doing so is that it can more accurately interpolate and complete the missing segment slice. PCA only calculates the organ part of the completed feature, which is faster and more accurate. In addition, PCA has a well-encapsulated algorithm package, which is simple and easy to use.

[0078] As an optional implementation of this embodiment, in a medical image slice interpolation and completion method of this embodiment, the preliminary interpolation and completion processing for the missing segment slice to obtain the preliminary completed segment slice includes:

[0079] SIFT feature point matching is performed between the missing segmentation slice and the complete segmentation slices on the front and back sides.

[0080] A perspective transformation is performed, and the intersection of the mask results of the complete segmented slices on the front and back sides is mapped to the missing segmented slice to obtain the preliminary completed segmented slice.

[0081] The preliminary completion process of segmented medical image slice interpolation and completion method in this embodiment has the following technical effects:

[0082] 1) After SIFT feature point localization, the confidence level of the initial interpolation completion result is already very high. Then, the missing segment slices are subjected to SDF transformation and iterative interpolation completion processing to make the interpolation result more accurate and the completion smoother.

[0083] 2) The intersection of the feature points extracted by SIFT and the perspective transformation is very close to the real mask part. Using this result as the initial interpolation result for iteration, the final interpolation result is more accurate.

[0084] In this invention, the SIFT algorithm is used in the initial interpolation completion process. The initial interpolation completion result can be achieved through various feature point matching algorithms, such as SURF and ORB.

[0085] In a medical image slice interpolation and completion method of this embodiment, the step of locating the missing segment in the image segmentation slice of the medical image and simultaneously obtaining the complete segmentation slices on both sides before and after the missing segment includes:

[0086] If the located missing segment is a segment of the first image or a segment of the last image, then obtain the complete segment behind or the complete segment before the missing segment.

[0087] In the preliminary interpolation and completion process for missing segments, the perspective transformation result of the complete segment slices before or after the missing segment slice is directly mapped to the missing segment slice to obtain the preliminary completed segment slice.

[0088] See Figure 2 The diagram shown is an example flowchart of the preliminary interpolation and completion process of a medical image slice interpolation and completion method in this embodiment.

[0089] In a medical image slice interpolation and completion method of this embodiment, the step of determining whether the initially completed segmented slice meets the preset conditions includes:

[0090] Principal component analysis (PCA) feature extraction is performed based on the preliminary completed segmentation slices and the complete segmentation slices on the front and back sides or both sides. The completed features in the preliminary completed segmentation slices and the initial features corresponding to the completed features in the complete segmentation slices on the front and back sides or both sides are extracted.

[0091] Calculate the cosine similarity between the completed feature and the initial feature, and compare it with a preset threshold;

[0092] If the calculated cosine similarity is less than or equal to the preset threshold, the medical image slice interpolation and completion process ends. If the calculated cosine similarity is greater than the preset threshold, the iterative interpolation and completion process for missing segmented slices begins.

[0093] In this embodiment of a medical image slice interpolation and completion method, after the initial completion of the segmented slices, PCA is performed on the missing segmented slices and the non-missing segmented slices, and then the cosine similarity is calculated as a marker to end the iteration. The advantage of doing this is that it can more accurately interpolate and complete the missing segmented slices. PCA only calculates the organ part of the mask, which is faster and more accurate. In addition, PCA has a well-encapsulated algorithm package, which is simple and easy to use.

[0094] In addition, in a medical image slice interpolation and completion method of this embodiment, after the preliminary interpolation and completion process obtains the preliminary completed segmented slice, if the preliminary completed segmented slice meets the preset conditions, the medical image slice interpolation and completion ends, and the preliminary completed segmented slice is directly used as the final medical image slice interpolation and completion result, without performing the iterative interpolation and completion process.

[0095] As an optional implementation of this embodiment, in the medical image slice interpolation and completion method described in this embodiment, the missing segmented slices in the image segmentation slices of the medical image include:

[0096] Obtain the category matrix of each organ in the image segmentation slices of medical images;

[0097] Based on the segmentation order of the three-dimensional medical images, determine whether there are corresponding segmentation results for the two images before and after the current image segmentation slice B, namely image segmentation slices A and C.

[0098] If image segmentation slices A and C have segmentation results, but the current image segmentation slice B has no segmentation results, then the current image segmentation slice B is a missing segmentation slice of the corresponding category.

[0099] The medical image slice interpolation and completion method described in this embodiment includes:

[0100] After locating the missing segment in the image segmentation slice of the medical image and obtaining the complete segmentation slices on both sides before and after the missing segmentation slice, select the minimum bounding rectangle of the complete segmentation slices on both sides before and after the missing segmentation slice.

[0101] During the initial interpolation and completion process for missing segmented slices, the segmented slice image within the minimum bounding rectangle is processed.

[0102] In the process of iterative interpolation and completion of missing segmented slices, the segmented slice images within the minimum bounding rectangle are processed.

[0103] This embodiment uses the minimum bounding rectangle for preliminary interpolation and completion processing, which reduces the amount of computation and improves processing efficiency.

[0104] Specifically, in a medical image slice interpolation and completion method of this embodiment, the minimum bounding rectangle of the complete slices on both sides of the missing slice includes:

[0105] When the minimum bounding rectangles of the complete segments on the front and back sides of the missing segment are not equal, the smaller minimum bounding rectangle is selected.

[0106] As an optional implementation of this embodiment, in the medical image slice interpolation and completion method described in this embodiment, a deep learning segmentation network model is used to perform preliminary segmentation of the medical image of the case. The deep learning segmentation network model is mainly responsible for performing preliminary segmentation of the input case and outputting organ segmentation results.

[0107] As an optional implementation method of this embodiment, see Figure 1 The present embodiment illustrates a specific implementation process for the medical image slice interpolation and completion method. This method is an adaptive interpolation method for completing missing organ slices. Its basic idea is to perform SIFT feature point matching and mask perspective transformation on the missing slice and the non-missing slices on the front and back sides to perform iterative preliminary interpolation and completion processing. Then, iterative processing is performed based on the cosine similarity of PCA features between the organs in the mask region of the missing slice and the organs in the boundary mask. The result is compared with a preset threshold. The iterative calculation uses a method based on SDF transformation for interpolation until the result is below the threshold, at which point the iteration ends.

[0108] This embodiment also provides an apparatus for implementing the medical image slice interpolation and completion method, comprising:

[0109] The medical image segmentation module uses a deep learning segmentation network model to segment medical images of medical cases, obtaining image segmentation slices of medical images;

[0110] The missing segmentation slice localization module locates the missing segmentation slice in the image segmentation slices of medical images, and simultaneously acquires the complete segmentation slices on both sides before and after the missing segmentation slice.

[0111] The preliminary interpolation and completion module performs preliminary interpolation and completion processing on the missing segment slices to obtain preliminary completed segment slices. It then determines whether the preliminary completed segment slices meet the preset conditions. If they do, the medical image slice interpolation and completion processing ends; otherwise, it enters the iterative interpolation and completion processing flow for the missing segment slices.

[0112] The iterative interpolation completion module executes the iterative interpolation completion processing flow. For each iteration of the iterative interpolation completion processing flow, it determines whether the iteratively completed segmentation slices meet the preset conditions. If they do, the iteration ends; otherwise, the iteration continues.

[0113] This embodiment also provides a computer-readable storage medium storing a computer-executable program. When the computer-executable program is executed, it implements the method for interpolating and completing medical image slices as described above.

[0114] The computer-readable storage medium described in this embodiment may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0115] This embodiment also provides an electronic device, including a processor and a memory, wherein the memory is used to store a computer-executable program, and when the computer program is executed by the processor, the processor executes the aforementioned method for implementing medical image slice interpolation and completion.

[0116] The electronic device is manifested in the form of a general-purpose computing device. It may contain one or more processors that work collaboratively. This invention also does not preclude distributed processing, meaning that processors may be distributed across different physical devices. The electronic device of this invention is not limited to a single entity, but may also be the sum of multiple physical devices.

[0117] The memory stores a computer-executable program, typically machine-readable code. The computer-readable program can be executed by the processor to enable the electronic device to perform the method of the present invention, or at least some steps of the method.

[0118] The memory includes volatile memory, such as random access memory (RAM) and / or cache memory, and may also be non-volatile memory, such as read-only memory (ROM).

[0119] It should be understood that the electronic device of the present invention may also include elements or components not shown in the examples above. For example, some electronic devices also include display units such as a display screen, and some electronic devices also include human-computer interaction elements such as buttons and keyboards. Any electronic device capable of executing a computer-readable program in its memory to implement the method of the present invention or at least some steps of the method can be considered as an electronic device covered by the present invention.

[0120] From the above description of the embodiments, those skilled in the art will readily understand that the present invention can be implemented by hardware capable of executing specific computer programs, such as the system of the present invention, and the electronic processing unit, server, client, mobile phone, control unit, processor, etc. included in the system. The present invention can also be implemented by computer software that executes the methods of the present invention, for example, by control software executed by a microprocessor, electronic control unit, client, server, etc. However, it should be noted that the computer software executing the methods of the present invention is not limited to execution in one or a specific set of hardware entities; it can also be implemented in a distributed manner by unspecified hardware. For computer software, the software product can be stored in a computer-readable storage medium (such as a CD-ROM, USB flash drive, portable hard drive, etc.) or distributed across a network, as long as it enables electronic devices to execute the methods according to the present invention.

[0121] The above embodiments are only used to illustrate the present invention and are not intended to limit the technical solutions described herein. Although the present invention has been described in detail with reference to the above embodiments, the present invention is not limited to the specific embodiments described above. Therefore, any modifications or equivalent substitutions to the present invention, as well as all technical solutions and improvements that do not depart from the spirit and scope of the invention, are covered within the scope of the claims of the present invention.

Claims

1. A method for interpolating and completing medical image slices, characterized in that, include: A deep learning segmentation network model is used to segment medical images of medical cases, resulting in image segmentation slices of medical images; Locate the missing segment in the image segmentation slice of a medical image, and simultaneously obtain the complete segmentation slices on both sides before and after the missing segment; Preliminary interpolation and imputation processing is performed on the missing segment to obtain the preliminary imputed segment; Determine whether the preliminary completed segmentation slices meet the preset conditions. If they do, the medical image slice interpolation and completion process ends. If they do not meet the conditions, proceed to the missing segmentation slice iterative interpolation and completion process. In the iterative interpolation completion process, each iteration of the iteratively completed segmentation slice is judged to see if it meets the preset conditions. If it does, the iteration ends; otherwise, the iteration continues. The preliminary interpolation and completion process for the missing segment slices to obtain the preliminary completed segment slices includes: SIFT feature point matching is performed between the missing segment and the complete segment on both sides; Perform perspective transformation, and map the intersection of the mask results of the complete segmented slices on the front and back sides to the missing segmented slice to obtain the preliminary completed segmented slice; The iterative interpolation completion process includes: performing SDF-based interpolation completion on the initially completed segmented slices to obtain iteratively completed segmented slices.

2. The method for interpolating and completing medical image slices according to claim 1, characterized in that, The step of performing SDF-based interpolation completion processing on the initially completed segmented slices to obtain iteratively completed segmented slices includes: The complete segmented slices Xi-1 and Xi+1 on both sides before and after the initial completed segmented slice Xi are transformed into directed distance maps Mi-1 and Mi+1 by SDF transformation. The Lerp interpolation function is used on the directed distance map to obtain the interpolated completed distance map Mi. The inverse transformation is performed on the interpolated completed distance map Mi to obtain the iteratively completed segmented slice.

3. The method for interpolating and completing medical image slices according to claim 2, characterized in that, After each iteration of interpolation and completion processing is completed, it is determined whether the iteratively completed segmentation slice meets the preset conditions, including: Principal component analysis (PCA) feature extraction is performed based on iteratively completed segmentation slices and complete segmentation slices on both sides, extracting the completion features in the iteratively completed segmentation slices and the initial features corresponding to the completion features in the complete segmentation slices on both sides. Calculate the cosine similarity between the completed feature and the initial feature, and compare it with a preset threshold; If the calculated cosine similarity is less than or equal to the preset threshold, the iteration ends; if the calculated cosine similarity is greater than the preset threshold, the iteration continues.

4. The method for interpolating and completing medical image slices according to claim 1, characterized in that, The process of locating missing segmented slices in the image segmentation slices of the medical image, and simultaneously acquiring complete segmented slices on both sides before and after the missing segmented slice, includes: If the located missing segment is a segment of the first image or a segment of the last image, then obtain the complete segment behind or the complete segment before the missing segment. In the preliminary interpolation and completion process for missing segments, the perspective transformation result of the complete segment slices before or after the missing segment slice is directly mapped to the missing segment slice to obtain the preliminary completed segment slice.

5. The method for interpolating and completing medical image slices according to claim 4, characterized in that, The determination of whether the preliminary completed segmentation slices meet the preset conditions includes: Principal component analysis (PCA) is performed to extract features from the preliminary completed segmentation slices and the complete segmentation slices on the front and back sides or both sides. The completed features in the preliminary completed segmentation slices and the initial features corresponding to the completed features in the complete segmentation slices on the front and back sides or both sides are extracted. Calculate the cosine similarity between the completed feature and the initial feature, and compare it with a preset threshold; If the calculated cosine similarity is less than or equal to the preset threshold, the medical image slice interpolation and completion process ends. If the calculated cosine similarity is greater than the preset threshold, the iterative interpolation and completion process for missing segmented slices begins.

6. The method for interpolating and completing medical image slices according to claim 1, characterized in that, The missing segmentation slices in the image segmentation slices of the localized medical image include: Obtain the category matrix of each organ in the image segmentation slices of medical images; Based on the segmentation order of the three-dimensional medical images, determine whether there are corresponding segmentation results for the two images before and after the current image segmentation slice B, namely image segmentation slices A and C. If image segmentation slices A and C have segmentation results, but the current image segmentation slice B has no segmentation results, then the current image segmentation slice B is a missing segmentation slice of the corresponding category.

7. The method for interpolating and completing medical image slices according to claim 1, characterized in that, include: After locating the missing segment in the image segmentation slice of the medical image and obtaining the complete segmentation slices on both sides before and after the missing segmentation slice, select the minimum bounding rectangle of the complete segmentation slices on both sides before and after the missing segmentation slice. During the initial interpolation and completion process for missing segmented slices, the segmented slice image within the minimum bounding rectangle is processed. In the iterative interpolation completion process for missing segmented slices, the segmented slice image within the minimum bounding rectangle is processed.

8. The method for interpolating and completing medical image slices according to claim 7, characterized in that, The minimum bounding rectangle of the complete segmented slices on both sides of the selected missing segmented slice includes: When the minimum bounding rectangles of the complete segments on the front and back sides of the missing segment are not equal, the smaller minimum bounding rectangle is selected.

9. An apparatus for implementing the medical image slice interpolation and completion method as described in any one of claims 1-8, characterized in that, include: The medical image segmentation module uses a deep learning segmentation network model to segment medical images of medical cases, obtaining image segmentation slices of medical images; The missing segmentation slice localization module locates the missing segmentation slice in the image segmentation slices of medical images, and simultaneously acquires the complete segmentation slices on both sides before and after the missing segmentation slice. The preliminary interpolation and completion module performs preliminary interpolation and completion processing on the missing segment slices to obtain preliminary completed segment slices. It then determines whether the preliminary completed segment slices meet the preset conditions. If they do, the medical image slice interpolation and completion processing ends; otherwise, it enters the iterative interpolation and completion processing flow for the missing segment slices. The iterative interpolation completion module executes the iterative interpolation completion processing flow. For each iteration of the iterative interpolation completion processing flow, it determines whether the iteratively completed segmentation slices meet the preset conditions. If they do, the iteration ends; otherwise, the iteration continues.

Images