Image processing apparatus, image processing method, and computer program product

Abstract

The present application provides an image processing device, an image processing method and a computer program product capable of reducing the amount of calculation and improving the efficiency of post-processing. The image processing device processes a primary segmentation processing result obtained by performing segmentation processing on an image at a segmentation resolution, and comprises: a boundary point searching unit configured to extract a set of specific boundary points on the image according to a probability map as the primary segmentation processing result; a positioning unit configured to map the extracted specific boundary points back to the probability map to obtain coordinates of the specific boundary points; and a resampling unit configured to perform first resampling processing on the specific boundary points on the probability map, perform second resampling processing different from the first resampling processing on other points on the probability map except the specific boundary points, and obtain a segmentation processing result at a target resolution.

Description

Technical Field

[0001] The embodiments relate to image processing apparatus, image processing method, and computer program product for performing image segmentation processing, and particularly to image processing apparatus, image processing method, and computer program product for performing post-processing in image segmentation processing. Background Technology

[0002] Currently, medical image segmentation models are frequently used to segment medical images. In this case, the medical image segmentation models used are usually trained within a certain resolution range. Therefore, to obtain optimal accuracy, before using the medical image segmentation model for inference, the medical image needs to be processed to the specifications used during training, such as pixel value normalization and sampling to the inference resolution, before being input into the medical image segmentation model for inference. Furthermore, after obtaining the model's inference results, post-processing is required to obtain the segmentation results corresponding to the input image. The segmentation model is a model that performs region extraction-related processing on medical images, such as segmentation, and the segmentation result is, for example, the region extraction result.

[0003] Existing post-processing procedures include different steps depending on the segmentation model. However, due to the difference between the original image resolution and the inference resolution (segmentation resolution), a resampling process is generally unavoidable, which is the process of sampling the model inference results to the original image resolution.

[0004] In the resampling process during post-processing, to obtain optimal accuracy, the probability map of each category channel is typically resampled independently using a linear method, and then the category with the highest probability is selected as the category for that channel. The time and memory consumption required for this process are directly proportional to the number of target categories and the image size. Therefore, it significantly impacts the efficiency of image segmentation. Summary of the Invention

[0005] The present invention was made in view of the above problems, and its purpose is to provide an image processing apparatus, image processing method and computer program product that can reduce the amount of computation and thus improve post-processing efficiency.

[0006] One of the technical solutions of the present invention is an image processing apparatus that processes a primary segmentation result obtained by segmenting an image at a segmentation resolution. The apparatus comprises: a boundary point search unit that extracts a set of specific boundary points on the image based on a probability map that is the primary segmentation result; a localization unit that maps the extracted specific boundary points back to the probability map to obtain the coordinates of the specific boundary points; and a resampling unit that performs a first resampling process on the specific boundary points on the probability map and performs a second resampling process, different from the first resampling process, on other points on the probability map besides the specific boundary points, to obtain a segmentation result at a target resolution.

[0007] Furthermore, one of the technical solutions, an image processing method, processes the initial segmentation result obtained by segmenting the image at a segmentation resolution. This includes: a boundary point search step, which extracts a set of specific boundary points within a specified range on the image based on a probability map that serves as the initial segmentation result; a localization step, which maps the extracted specific boundary points back to the probability map to obtain the coordinates of the specific boundary points; and a resampling step, which performs a first resampling process on the specific boundary points on the probability map and performs a second resampling process, different from the first resampling process, on other points on the probability map besides the specific boundary points, to obtain the segmentation result at the target resolution.

[0008] In addition, one of the technical solutions is a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the image processing method described above.

[0009] According to the above technical solution of this application, by finding boundary points that may be affected by interpolation at low resolution, and only performing high-cost post-processing calculations on some of the found points, the number of points that need to be calculated with high cost can be greatly reduced, thereby reducing the amount of computation of the entire resampling process and improving the post-processing speed and efficiency of the segmentation process. Attached Figure Description

[0010] Figure 1 This is a block diagram illustrating an example of the functional configuration of the image processing apparatus according to the first embodiment.

[0011] Figure 2 This is a schematic diagram of an example of a coarse distribution image obtained from a probability graph.

[0012] Figure 3 (a) is a schematic diagram of a mask image showing the coarse boundaries extracted from a coarse distribution image. Figure 3 (b) is for Figure 3 A schematic diagram of the image after linear resampling of the entire image in (a). Figure 3 (c) shows from Figure 3 A schematic diagram of the mask image of fine boundaries extracted from image (b).

[0013] Figure 4 This is a flowchart illustrating the boundary point search process in the first embodiment.

[0014] Figure 5 This is an example diagram used to illustrate the effect of the boundary point search process in the first embodiment.

[0015] Figure 6 (a) is a schematic diagram of the image after the entire coarse-distributed image has been resampled to nearest neighbor. Figure 5 (b) replaces the result after linear resampling of the points at the fine boundary with... Figure 5 A schematic diagram of the image obtained by taking the corresponding position in (a).

[0016] Figure 7 This is a flowchart illustrating the segmentation process performed by the image processing apparatus of the first embodiment.

[0017] Figure 8 This is a block diagram illustrating an example of the functional configuration of the image processing apparatus according to the second embodiment.

[0018] Figure 9 This is a flowchart illustrating the post-processing performed by the image processing apparatus of the second embodiment.

[0019] Explanation of reference numerals in the attached figures:

[0020] 100, 200 image processing units; 10 segmentation units; 20 boundary point search units; 30 positioning units; 40 resampling units; 50 downsampling units. Detailed Implementation

[0021] Hereinafter, preferred embodiments of the image processing apparatus and image processing method of the present invention will be described in detail with reference to the accompanying drawings.

[0022] The image processing apparatus of this invention comprises multiple functional modules. It can be installed as a program product including a computer program in a standalone computer or other device with a CPU (central processing unit) and memory, or it can be distributed across multiple devices, with a processor executing the various functional modules of the image processing apparatus stored in memory. Alternatively, it can be implemented in hardware as a circuit capable of executing the various functions of the image processing apparatus. The circuit implementing the image processing apparatus can transmit and receive data or acquire data via a network such as the Internet. Furthermore, the image processing apparatus of this invention can also be directly installed in a medical image acquisition device, such as a CT scanner or an MRI scanner.

[0023] Furthermore, in the following description, the image processing apparatus of the present invention is used for multi-organ segmentation processing of medical images including multiple organs, and particularly for post-processing of medical images after preliminary segmentation. However, the present invention is not limited to multi-organ segmentation, and can also be used for single-organ segmentation processing or other segmentation processing such as segmentation processing between different tissues. The present invention can be applied to any post-processing in the segmentation processing of images.

[0024] (First Implementation)

[0025] First, refer to Figures 1 to 6 The first embodiment of the present invention will be described.

[0026] Figure 1 This is a block diagram illustrating an example of the functional configuration of the image processing apparatus according to the first embodiment. For example... Figure 1 As shown, the image processing apparatus 100 of the first embodiment includes a segmentation unit 10, a boundary point search unit 20, a positioning unit 30, and a resampling unit 40.

[0027] The segmentation unit 10 receives medical images of the subject acquired by medical image acquisition devices such as CT scanners, ultrasound scanners, and MRI scanners. Using a trained segmentation model and other tools, it performs preliminary segmentation processing on the received medical images to obtain a probability map as the initial segmentation result. In this embodiment, an abdominal medical image including multiple organs is used as an example. Furthermore, assuming the resolution of the received medical image is higher than the inference resolution required by the segmentation model, the segmentation unit 10 also performs preprocessing on the received medical image, such as normalization and sampling to the inference resolution.

[0028] The preprocessing and segmentation based on the segmentation model performed by segmentation unit 10 can be performed using existing methods. Preprocessing refers to converting the image into the uniform format used during segmentation model training, typically including pixel value normalization and sampling to inference resolution. Furthermore, segmentation unit 10 can also utilize existing segmentation methods that do not use deep learning-based segmentation models to segment medical images, obtaining a probability map as the initial segmentation result. For example, segmentation can be performed using Bayesian networks, Markov networks, or other similar methods.

[0029] Since the target resolution of the segmented image, which is the desired segmentation result, is generally higher than the inference resolution during segmentation, post-processing is required to obtain a segmentation result with the target resolution. The target resolution refers to the resolution of the image that becomes the final segmentation result. Generally, the target resolution is the same as the resolution of the original medical image acquired by the medical image acquisition device before preprocessing, but it can also be arbitrarily set according to requirements. Post-processing refers to the process of converting the output of the segmentation model into the final required segmentation result, such as converting the output probability heatmap into a segmentation mask image, or performing connected component filtering to remove noise. Since there is a difference between the inference resolution and the target resolution, post-processing generally inevitably includes a resampling step, that is, sampling the inference result to the target resolution of the image.

[0030] The boundary point finding unit 20 extracts a set of specific boundary points on the image based on the probability map obtained by the segmentation unit 10 as the result of the initial segmentation process. Here, a boundary is defined as a set of points of different categories within a defined surrounding area. Specific boundary points refer to predefined boundary points that can represent the boundary. In other words, the boundary point finding unit 20 can determine a set of specific boundary points representing the boundaries between different regions in the image based on the initial segmentation process result. As a principle for determining specific boundary points, it is preferable to accurately depict the boundary with a smaller number of points. In this embodiment, the boundary point finding unit 20 obtains a fine boundary mask composed of the set of specific boundary points by filtering points in the image boundary region using the probability map.

[0031] Specifically, the boundary point finding unit 20 uses a classification tool (commonly the argmax (argument of the maximum) function or other tools for finding the maximum value index) to transform the probability map into a coarse distribution image. Figure 2 This is a schematic diagram of an example of a coarse distribution image obtained from a probability graph. For example... Figure 2As shown, by transforming the probability map into a coarse distribution image, the distribution of different regions corresponding to the probability map can be visually represented on the image. Different gray levels are used to represent different regions; of course, different colors could also be used. This embodiment uses an abdominal medical image as an example for explanation. Figure 2 The results of the initial two-dimensional segmentation are shown, including six different organs: liver, kidney, spleen, pancreas, erector spinae muscle, and lumbar muscle.

[0032] Next, the boundary point search unit 20 from Figure 2 The boundary mask within a specified range is extracted from the coarsely distributed image shown. The boundary is defined as the set of points of different categories within the specified range; here, 1 indicates boundary points and 0 indicates non-boundary points. This specified range can be arbitrarily set; here it can be set to 1 (including adjacent points and adjacent corner points), thus forming, for example... Figure 3 The boundary mask image is as shown in (a). The boundary is coarser here because it is defined on a coarse-distributed image with lower resolution. Figure 3 In (a), the boundary points of the boundary are linearly sampled to the target resolution to obtain... Figure 3 (b) shows the boundary mask image at the target resolution. It can be seen that... Figure 3 The boundary mask image shown in (b) has a relatively coarse boundary and includes a large number of points in the set. To further reduce the number of boundary points and accelerate post-processing, the boundary point search unit 20 can, for example, use a threshold of 1 to... Figure 3 The boundary of (b) is further filtered, removing those points smaller than a specified threshold from the boundary points after linear sampling (for example, setting it to 1 yields the best results, proving that the definition of setting the threshold to 1 can be referred to in the following section on " Figure 3 (Proof that) the set of specific boundary points of the fine boundary shown in (c) can cover all points that need interpolation calculations in post-processing, is obtained by using the boundary points after the boundary points as the set of specific boundary points. Figure 3 The fine boundary mask image shown in (c) is shown.

[0033] Figure 4 This is a flowchart illustrating the boundary point search process described above in the first embodiment. To determine specific boundary points, in step S401, the boundary point search unit 20 extracts coarse boundaries from the coarse distribution image corresponding to the probability map, and then performs linear resampling on the extracted coarse boundaries to the target resolution (step S402). Finally, a finer boundary is extracted using a threshold of 1, and the points included in this finer boundary are taken as specific boundary points. This limits the range of the boundary as much as possible, obtaining a minimal set of boundary points.

[0034] The results extracted through the above boundary point search and processing are as follows: Figure 3 The set of specific boundary points shown in (c) can cover all points that require interpolation calculations in post-processing. The reasons for this conclusion are explained below.

[0035] Each medical image is represented as a multidimensional matrix in the processor, but is discrete in physical space. For points that do not fall exactly on a grid point, interpolation is needed to calculate the value of that point. Depending on the interpolation method, the value of a target point is generally determined only by its neighboring grid points.

[0036] Segmentation requires classifying each point in an image, and its direct output is the probability that each point belongs to the corresponding category. The format is:

[0037]

[0038] Where n is the number of target categories, a i Let be the probability that the point belongs to class i, where 0 represents the background class. The final class of the point is the index corresponding to the highest probability, i.e.

[0039] c = argmax([a0, a1, ..., a... n ])

[0040] The problem that needs to be proven for the extraction of fine boundaries is: when adjacent points are of the same category C, all points falling within the adjacent points must also be of category C. These points do not require interpolation calculations, while the remaining points do require interpolation calculations. Furthermore, the fine boundaries extracted by the boundary point search process in this embodiment (such as...) Figure 3 (c) contains all the points that need to be calculated.

[0041] First consider as follows Figure 5 In the one-dimensional case shown in (a), let points P1 and P2 be adjacent points, and point P be the point to be interpolated. Then the probabilities of points P1 and P2 are respectively:

[0042] P1 prob =[a0,a1,…,a n ],P2 prob =[b0,b1,…,b n ]

[0043] Using the linear interpolation formula, we can derive:

[0044] P prob =[xb0+(1-x)a0,xb1+(1-x)a1,…,xb n +(1-x)a n ]

[0045] Where x is the normalized distance from point P to point P1. When x is 0, point P coincides with point P1, and when x is 1, point P coincides with point P2.

[0046] When points P1 and P2 belong to the same category, for example, let's say category C, then the following holds true:

[0047] a c =max([a0,a1,…,a…) n ]),b c =max([b0,b1,…,b n ]),

[0048] a c ≥a i ,b c ≥b i i∈[0,n],i≠c

[0049] xb c +(1-x)a c ≥xb i +(1-x)a i

[0050] argmax([xb0+(1-x)a0,xb1+(1-x)a1,…,xb n +(1-x)a n ])=c

[0051] At this point, the category of point P must also be C.

[0052] When points P1 and P2 have different categories, the category of point P is related to x and P1. prob P2 prob The values ​​are all correlated, so interpolation calculations are required.

[0053] Consider whether points P1 and P2 are Figure 4 In step S401, the boundary points in the coarse boundary (the boundary points in step S401 are, for example, defined as points of different categories within their 1-neighborhood). Figure 3 On (a), the value of boundary points is 1, and the value of non-boundary points is 0, with a total of four cases: (1, 1), (1, 0), (0, 1), and (0, 0).

[0054] According to the definition of a boundary point, when any one of points P1 and P2 is in... Figure 3 When the value of (a) is 0, point P is at Figure 3 The interpolation result on (b) will always be less than 1. The neighboring points of a point with a value of 0 have the same category, and since points P1 and P2 are adjacent, points P1 and P2 have the same category. Point P does not need to be interpolated, and the category of points P1 and P2 can be directly taken as the final result.

[0055] When points P1 and P2 are (1, 1), point P is in Figure 3The interpolation result on (b) must be 1, so in Figure 4 In step S403, when filtering, 1 is used as the threshold, and the points equal to 1 must contain all the points that need to be interpolated.

[0056] Secondly, consider as Figure 5 In the two-dimensional case shown in (b), let point P be the point to be interpolated, and points P1, P2, P3, and P4 be adjacent points, and x and y be the distances from point P to edges P1P3 and P1P2, respectively.

[0057] Similar to the one-dimensional case, point P does not require interpolation calculations only when points P1, P2, P3, and P4 all belong to the same category. Consider whether points P1, P2, P3, and P4 are boundary points. When the value of any of these points is 0, the adjacent points of the point with a value of 0 all belong to the same category. Since points P1, P2, P3, and P4 are adjacent points, they all belong to the same category, and point P does not require interpolation calculations.

[0058] On the other hand, when the values ​​of points P1, P2, P3, and P4 are all 1, point P is in Figure 3 The interpolation result on (b) must be 1, so in Figure 4 In step S403, when filtering, 1 is used as the threshold, and the points equal to 1 must include all points that are susceptible to interpolation and require interpolation calculation.

[0059] Similarly, the same principle applies to cases with more than three dimensions.

[0060] As can be seen above, the boundary point search unit 20 of this embodiment can extract the minimum set of boundary points while ensuring that all boundary points that need to be interpolated are included.

[0061] return Figure 1 As explained above, the positioning unit 30 maps the specific boundary points extracted by the boundary point finding unit 20 as described above back to the probability map that serves as the result of the primary segmentation process, thereby obtaining the coordinates of the specific boundary points on the medical image.

[0062] The resampling unit 40 performs higher precision linear resampling on the specific boundary points on the probability map based on the coordinates of the specific boundary points obtained by the positioning unit 30, and performs lower precision neighbor resampling on other points on the probability map except for the specific boundary points, thereby obtaining the segmentation result at the target resolution.

[0063] Specifically, resampling unit 40 uses the argmax function on medical images to transform the probability map into... Figure 2The image shown is a coarse distribution image. Here, when the boundary point finding unit 20 generates a coarse distribution image, the resampling unit 40 can also directly use the coarse distribution image generated by the boundary point finding unit 20.

[0064] Next, 40 pairs of resampling units Figure 2 The coarse-distributed image shown is resampled from nearest neighbor to obtain the following result: Figure 5 A mask image with coarse target resolution, as shown in (a). From Figure 5 As can be seen in (a), the edges of the organs in the image are jagged and relatively rough.

[0065] Next, the resampling unit 40 performs linear resampling to the target resolution interpolation only on these specific boundary points mapped on the probability map, based on their coordinates. Then, it executes the argmax function on the interpolated boundary points to obtain their classification, and finally backfills the linearly resampled boundary points to the target resolution based on their coordinates. Figure 5 In (a), we obtain the following: Figure 5 (b) shows the image as a high-precision segmentation result at the target resolution.

[0066] In addition, Figure 5 In the example, the resampling unit 40 first performs neighbor resampling on all points in the medical image, which may affect accuracy but is fast. Then, it performs higher-precision linear resampling on the extracted specific boundary points and then backfills them. This allows for segmentation results with similar accuracy to existing technologies in a shorter time and with better memory consumption. However, the invention is not limited to this. The resampling unit 40 can also perform resampling on specific boundary points and other points with different precision, and then combine the results of the resampling to achieve the same result. Figure 5 (b) shows the image as a high-precision segmentation result at the target resolution.

[0067] Figure 7 This is a flowchart illustrating the segmentation process performed by the image processing apparatus of the first embodiment.

[0068] After the segmentation process begins, the image processing device 100 is input with the medical image to be processed (step S701). First, in step S702, the receiving unit 10 performs preprocessing on the medical image, such as downsampling to inference resolution, and then uses the segmentation model to perform inference to obtain a probability map of the medical image (step S703).

[0069] Next, the boundary point search unit 20 classifies the data according to the probability map to obtain a coarse distribution image (step S704), and extracts a set of specific boundary points that meet the specified screening conditions from the coarse distribution image (step S705).

[0070] Next, in step S706, the localization unit 30 maps the extracted specific boundary points back to the probability map to obtain the coordinates of the specific boundary points. Furthermore, the resampling unit 40 performs neighbor resampling of the coarse distribution image to the target resolution, and performs linear resampling of the set of specific boundary points to the target resolution (step S707). The resampling results are then merged to obtain the segmentation result at the target resolution (step S708).

[0071] Furthermore, in the first embodiment, the boundary point search unit 20 corresponds to the "boundary point search unit", the positioning unit 30 corresponds to the "positioning unit", and the resampling unit 40 corresponds to the "resampling unit".

[0072] Using the same segmentation model (3D nnUNet, segmentation targets are kidney, liver, spleen, pancreas, gallbladder, erector spinae, and lumbar muscles) to infer probability maps from the same batch of medical images, and then applying the present invention and an existing high-precision method without acceleration (which performs independent linear resampling of the probability map of each category channel in the background and selects the highest probability as the category of that point) to obtain the final segmentation results, the segmentation accuracy and time and memory consumption of the two methods are compared.

[0073] The above experiments showed that the segmentation results of the present invention are almost identical to those of the prior art in terms of accuracy, and the present invention can achieve the same segmentation accuracy as the prior art.

[0074] On the other hand, the average speedup of this invention is more than 40% higher than that of the prior art, and the average peak post-processing memory usage is reduced by more than 60% compared to the prior art. This demonstrates that the present invention provides both speedup and memory optimization in most cases. Furthermore, when the number of image categories is large and the images are large, the resampling time and memory consumption in the prior art often exceed the inference process of the segmentation model, significantly impacting the efficiency of image segmentation. The larger the image size and the more target categories being segmented, the more significant the speedup effect of the present invention, and the better the average speedup and average peak post-processing memory usage compared to the prior art.

[0075] Therefore, according to the first embodiment, by finding boundary points that may be affected by interpolation at low resolution, and performing high-cost post-processing calculations only on some of the found points, the number of points requiring high-cost calculations can be greatly reduced, the amount of computation in the entire resampling process can be reduced, and the post-processing speed and efficiency of the segmentation process can be improved.

[0076] (Modified Example)

[0077] In the first embodiment, the medical image is initially segmented by the segmentation unit 10, and then the probability map, which is the result of the initial segmentation, is post-processed to obtain the segmentation result at the target resolution. However, the segmentation unit 10 can also be omitted, and the image processing apparatus 100 can obtain the segmentation probability map with a resolution different from the target resolution from an external segmentation model or apparatus, and then post-process the probability map to obtain the segmentation result at the target resolution.

[0078] Furthermore, in the first embodiment, the segmentation unit 10 performs segmentation processing at inference resolution using a segmentation model. However, other methods, such as non-model-based segmentation methods, can also be used for segmentation processing.

[0079] Furthermore, in the first embodiment, the resampling unit 40 performs linear resampling on specific boundary points on the probability map, and performs neighbor resampling on other points on the probability map besides the specific boundary points, thereby obtaining the segmentation result at the target resolution. However, the resampling method is not limited to this; any resampling process with higher accuracy for specific boundary points than for non-specific boundary points can be used, or different resampling methods can be used for resampling.

[0080] Furthermore, in the first embodiment, as a preferred embodiment, the boundary point finding unit 20 first extracts the coarse boundary range from the image, then resamples the coarse boundary range and uses a threshold to narrow it down to extract the minimum set of boundary points. However, the present invention is not limited to this; for example, the boundary point finding unit 20 may directly use... Figure 3 Using the boundary points within the coarse boundary range shown in (b) as specific boundary points, although such a boundary range is coarser and includes more boundary points that need to be processed, it can still reduce the amount of post-processing computation and achieve an acceleration effect compared with the existing technology.

[0081] In addition, the boundary point search unit 20 can also perform classification processing on the probability map, extract all boundary points from the classification results, and then select boundary points within a specified range from all boundary points as a set of specific boundary points. Other extraction methods can also be used to extract boundary points from the probability map as specific boundary points, as long as a set of specific boundary points can be extracted.

[0082] (Second Implementation)

[0083] The following reference Figures 8 to 9The second embodiment of the present invention will now be described. The image processing apparatus 200 of the second embodiment is based on the first embodiment. The main difference compared to the first embodiment is that, in the second embodiment, the image processing apparatus 200 does not have a segmentation unit 10, but instead has a downsampling unit 50. The following description will primarily focus on the differences, and identical or similar configurations will be labeled with the same reference numerals while omitting repetitive descriptions.

[0084] Figure 8 This is a block diagram illustrating an example of the functional configuration of the image processing apparatus according to the second embodiment. For example... Figure 8 As shown, the image processing apparatus 200 of the second embodiment includes a boundary point finding unit 20, a positioning unit 30, a resampling unit 40, and a downsampling unit 50.

[0085] After receiving the probability map as the result of the initial segmentation process from an external input, the downsampling unit 50 downsamples the received probability map to obtain a new probability map with a resolution lower than the target resolution (hereinafter referred to as the downsampled probability map). The downsampling magnitude can be set to, for example, half the resolution of the received probability map, and then compared with the target resolution. If the downsampled resolution is lower than the target resolution, the process proceeds to the next step. If the downsampled resolution is higher than the target resolution, the process of downsampling by half is performed again until the downsampled resolution is lower than the target resolution.

[0086] Alternatively, the downsampling unit 50 may first compare the resolution (segmentation resolution) of the received probability map with the target resolution, and perform downsampling only when the segmentation resolution is the same as or higher than the target resolution, so that the resolution after downsampling is lower than the target resolution. If the segmentation resolution is lower than the target resolution, downsampling is not performed and other post-processing is performed directly.

[0087] The downsampling unit 50 performs downsampling processing to obtain a downsampled probability map with a resolution lower than the target resolution. The image processing apparatus 200 performs the same post-processing as in the first method on this downsampled probability map to obtain a segmentation result at the target resolution.

[0088] That is, the boundary point finding unit 20 extracts a set of specific boundary points on the image based on the downsampled probability map. The localization unit 30 maps the specific boundary points extracted by the boundary point finding unit 20 back to the downsampled probability map, thereby obtaining the coordinates of the specific boundary points on the medical image. The resampling unit 40 performs a higher-precision linear resampling process on the specific boundary points on the downsampled probability map based on the coordinates of the specific boundary points obtained by the localization unit 30, while performing a lower-precision neighbor resampling process on other points on the downsampled probability map besides the specific boundary points, thereby obtaining the segmentation result at the target resolution.

[0089] Figure 9 This is a flowchart illustrating the post-processing performed by the image processing apparatus of the second embodiment.

[0090] After the segmentation process begins, the image processing device 200 receives a segmented probability map of the medical image (step S901). First, in step S902, the downsampling unit 50 downsamples the received probability map to obtain a new probability map with a resolution lower than the target resolution.

[0091] Next, the boundary point search unit 20 classifies the probability map after downsampling to obtain a coarse distribution image (step S903), and extracts a set of specific boundary points that meet the specified screening conditions from the coarse distribution image (step S904).

[0092] Next, in step S905, the localization unit 30 maps the extracted specific boundary points back to the probability map to obtain the coordinates of the specific boundary points. Furthermore, the resampling unit 40 performs neighbor resampling of the coarse distribution image to the target resolution, performs linear resampling of the set of specific boundary points to the target resolution (step S906), and merges the resampling results to obtain the segmentation processing result at the target resolution (step S907).

[0093] Furthermore, in the second embodiment, the downsampling unit 50 corresponds to the "downsampling section".

[0094] In existing technologies, if the inference resolution is the same as the target resolution, classification processing such as argmax is directly performed to obtain the classification result. When there are many segmentation categories and the image is large, argmax processing is also very time-consuming. This implementation simulates low resolution through downsampling processing, which can significantly reduce the number of points that need to be processed in post-processing.

[0095] (Modified Example)

[0096] In the second embodiment, the downsampling unit 50 performs downsampling at a fixed amplitude (e.g., half the amplitude) until the resolution after downsampling is lower than the target resolution. However, in order to prevent over-downsampling from causing lost isolated points in the original image, a resolution threshold can be set for the downsampling unit 50, and downsampling can be performed only when the resolution is higher than the resolution threshold, or the downsampling result can be canceled when the resolution after downsampling is lower than the resolution threshold.

[0097] In addition, the downsampling amplitude of the downsampling unit 50 can be set arbitrarily.

[0098] The constituent elements of each device in the above-described embodiments are functional concepts and do not necessarily need to be physically configured as shown in the illustrations. That is, the specific form of the distributed / integrated arrangement of each device is not limited to the content shown in the illustrations, and all or part of them can be configured in any unit, functionally or physically, according to various loads and usage conditions. Furthermore, all or any part of the processing functions performed in each device can be implemented by a CPU, a program executed by the CPU, and a program product including the program, or it can be implemented as hardware based on wiring logic.

[0099] Furthermore, the image processing apparatus and image processing method described in the above embodiments can be implemented by executing a pre-prepared program by a computer such as a personal computer or workstation. This program can be distributed via a network such as the Internet, or distributed as a program product. Additionally, the program can be recorded on a computer-readable non-transitory recording medium such as a hard disk, floppy disk (FD), CD-ROM, MO, or DVD, and then read from the recording medium by a computer for execution.

[0100] Several embodiments of the present invention have been described, but these embodiments are provided as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in a wide variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the invention as described in the claims and its equivalents.

Claims

1. An image processing apparatus for processing a primary segmentation result obtained by segmenting an image at a segmentation resolution, characterized in that, have: The boundary point finding unit extracts a set of specific boundary points on the image based on the probability map, which is the result of the primary segmentation process. The positioning unit maps the extracted specific boundary points back to the probability map to obtain the coordinates of the specific boundary points; and The resampling unit performs a first resampling process on the specific boundary point on the probability map, and performs a second resampling process, different from the first resampling process, on other points on the probability map besides the specific boundary point, to obtain the segmentation result at the target resolution.

2. The image processing apparatus as claimed in claim 1, characterized in that, The segmentation resolution is lower than the target resolution.

3. The image processing apparatus as claimed in claim 1, characterized in that, The boundary point search unit performs classification processing on the probability map and extracts all boundary points from the classification results as the set of the specific boundary points.

4. The image processing apparatus as claimed in claim 1, characterized in that, The boundary point search unit performs classification processing on the probability map, extracts all boundary points from the classification results, and selects boundary points within a specified range from all boundary points as the set of specific boundary points.

5. The image processing apparatus as claimed in claim 1, characterized in that, The boundary point finding unit performs classification processing on the probability map, extracts all boundary points from the classification processing results, performs linear sampling on all boundary points to the target resolution, and takes the boundary points after removing boundary points less than a specified threshold from the linearly sampled boundary points as the set of the specific boundary points.

6. The image processing apparatus as claimed in claim 1, characterized in that, The accuracy of the first resampling process is higher than that of the second resampling process.

7. The image processing apparatus as claimed in claim 1, characterized in that, The first resampling process includes linear resampling, and the second resampling process includes neighbor resampling.

8. The image processing apparatus as claimed in claim 7, characterized in that, The resampling unit performs classification processing on the probability map to obtain a coarse distribution image, performs neighbor resampling on the obtained coarse distribution image to obtain a first mask image at the target resolution, performs linear resampling on the specific boundary points in the probability map and then performs classification processing to obtain the classification result of the specific boundary points, and combines the classification result with the first mask image to generate a second mask image as the segmentation processing result at the target resolution.

9. The image processing apparatus as claimed in claim 1, characterized in that, It also includes a downsampling unit that downsamples the probability map to obtain a new probability map with a resolution lower than the target resolution. The boundary point finding unit extracts a set of specific boundary points on the image based on the new probability map.

10. The image processing apparatus as claimed in claim 1, characterized in that, The segmentation resolution is the same as or higher than the target resolution.

11. An image processing method, wherein the method processes a primary segmentation result obtained by segmenting an image at a segmentation resolution, characterized in that, include: The boundary point search step extracts a set of specific boundary points within a specified range on the image based on the probability map, which is the result of the primary segmentation process. The localization step involves mapping the extracted specific boundary points back to the probability map to obtain the coordinates of the specific boundary points. as well as The resampling step involves performing a first resampling process on the specific boundary point on the probability map, and performing a second resampling process, different from the first resampling process, on other points on the probability map besides the specific boundary point, to obtain the segmentation result at the target resolution.

12. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the image processing method of claim 11.

Images