Image segmentation device and image segmentation method

Abstract

To provide an image segmentation device and an image segmentation method that improve accuracy of image segmentation.SOLUTION: An image segmentation device 1 according to an embodiment includes: variable visual field function calculation means 10 that calculates a variable visual field function capable of adaptively generating a visual field range of a size corresponding to a plurality of segmentation targets included in an image; patch generation means 20 that generates a patch of the size corresponding to the plurality of segmentation targets by using the variable visual field function; training means 30 that trains a segmentation model by using the generated patch; and inference means 40 that infers the image by using the trained segmentation model to obtain a segmentation result for a plurality of segmentation targets.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The embodiments disclosed in this specification and the drawings relate to an image segmentation device and an image segmentation method. [Background technology]

[0002] In medical image analysis, human organs and lesions often exhibit multi-scale characteristics. Specifically, the size of the objects has a wide distribution. For example, the pulmonary trachea has about 24 levels of branching from the first-level bronchi to the final-level alveoli. The scale distribution of lung nodules can range from 0 to 30 mm or even larger. The size of kidney tumors can range from less than 3 cm in diameter to about 20 to 30 cm. How to improve the segmentation effect of different scale objects is an important issue in medical image segmentation processing.

[0003] Medical image segmentation can be divided into two methods: fully automatic and semi-automatic. Fully automatic medical image segmentation is a technology that inputs a specified image and then automatically obtains segmentation results (tumors, organs, tissues, etc.) using a direct model. In the task of fully automatic medical image segmentation, the segmentation objects always have large differences in shape, scale, and position, and it is difficult to obtain accurate segmentation results for all segmentation objects that have large differences in shape, scale, and position.

[0004] On the other hand, semi-automatic medical image segmentation is more commonly adopted, for example, a technique in which a user manually specifies a segmentation position or range first, and then the specified position or range is segmented by a model. In the task of semi-automatic medical image segmentation, specifying the segmentation range is a crucial point, and specifying a too large field of view (FOV) range will result in insufficient segmentation accuracy for small objects, resulting in a waste of computational resources. Conversely, specifying a too small FOV will tend to result in incomplete coverage of the objects. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] China Patent Application Publication No. 109598728 [Non-patent literature]

[0006] [Non-Patent Document 1] "A 3D Coarse-to-Fine Framework for Volumetric Medical Image Segmentation," 2018 International Conference on 3D Vision (3DV), 2018, pp. 682-690, doi:10.1109 / 3DV.2018.00083. [Non-Patent Document 2] "nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation" Nat Methods 18, 203-211 (2021). (https: / / doi.org / 10.1038 / s41592-020-01008-z) [Non-Patent Document 3] "L-SNet: from Region Localization to Scale Invariant Medical Image Segmentation" Jiahao Xie, Sheng Zhang, Jianwei Lu, Ye Luo(https: / / arxiv.org / abs / 2102.05971) Summary of the Invention [Problem to be solved by the invention]

[0007] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to improve the accuracy of image segmentation. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]

[0008] An image segmentation device according to an embodiment includes a variable field of view function calculation unit that calculates a variable field of view function capable of adaptively generating a field of view range of a corresponding size for a plurality of segmentation targets contained in an image, a patch generation unit that uses the variable field of view function to generate patches of a corresponding size for the plurality of segmentation targets, and an inference unit that infers the image using a segmentation model trained using the patches to obtain segmentation results for the plurality of segmentation targets. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of an image segmentation device according to an embodiment. [Diagram 2] FIG. 2 is a block diagram showing an example of the configuration of a medical image processing system according to the embodiment. [Diagram 3] FIG. 3 is a comparison diagram for explaining features of the processing executed by the image segmentation device according to the embodiment. [Figure 4] FIG. 4 is a flowchart showing the image segmentation process executed by the image segmentation device according to the first embodiment. [Figure 5A] FIG. 5A is a flowchart showing a process for calculating a variable view field function by the image segmentation device according to the first embodiment. [Figure 5B] FIG. 5B is a schematic diagram showing statistics used when the image segmentation device according to the first embodiment calculates a variable view field function. [Figure 6A] FIG. 6A is a schematic diagram showing a first mode of processing for the image segmentation device according to the first embodiment to calculate a variable field of view function. [Figure 6B] FIG. 6B is a schematic diagram showing a second method of processing for calculating a variable field of view function by the image segmentation device according to the first embodiment. [Figure 7] FIG. 7 is a schematic diagram showing a process of generating patches by the image segmentation device according to the first embodiment. [Figure 8] FIG. 8 is a schematic diagram showing an inference process in the image segmentation processing according to the first embodiment. [Figure 9] FIG. 9 is a flowchart showing an image segmentation process executed by the image segmentation device according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] In this embodiment, an image segmentation device and method with adaptive FOV for segmenting multi-scale objects is described. For example, the image segmentation device according to the embodiment includes an adaptive FOV calculation unit for calculating and fitting adaptive FOV functions for object features of different scales, a patch generation unit for generating patches with different FOVs for objects of different scales, a model training unit for training a model in an adaptive manner for objects of different scales using patches with different FOVs, and a model inference unit for obtaining an initial segmentation result by a coarse model, calculating an optimal FOV for the initial segmentation result, and then further inferring with the optimal FOV using a fine model to obtain a final inference result.

[0011] Specifically, according to one aspect of an embodiment, an image segmentation device is provided that performs segmentation on multiple segmentation targets contained in an image, comprising: a variable field of view function calculation means for calculating a variable field of view function capable of adaptively generating a field of view range of a corresponding size for the multiple segmentation targets; a patch generation means for generating patches of a corresponding size for the multiple segmentation targets using the variable field of view function calculated by the variable field of view function calculation means; and an inference means for inferring the image using a segmentation model trained using the patches generated by the patch generation means to obtain segmentation results for the multiple segmentation targets.

[0012] According to another aspect of the embodiment, an image segmentation method is provided for performing segmentation on multiple segmentation objects contained in an image, comprising: a variable field of view function calculation step for calculating a variable field of view function capable of adaptively generating a field of view range of a corresponding size for the multiple segmentation objects; a patch generation step for generating patches of a corresponding size for the multiple segmentation objects using the variable field of view function calculated in the variable field of view function calculation step; and an inference step for inferring the image using a segmentation model trained using the patches generated in the patch generation step to obtain segmentation results for the multiple segmentation objects.

[0013] According to the embodiment, a variable field of view function is calculated for segmentation objects of different sizes, and the generated variable field of view function is used to generate patches having sizes corresponding to the scales of each segmentation object, and then image segmentation is performed. Therefore, according to the embodiment, it is suitable for segmentation tasks in which the scales and shapes of the segmentation objects are significantly different, and it is possible to solve the technical problem that the segmentation accuracy of small objects is insufficient and the segmentation accuracy of large objects is incomplete, and it is possible to obtain a highly accurate segmentation result.

[0014] It should be noted that the embodiment can be applied to semi-automatic and fully automatic segmentation algorithms, and the variable field function calculation means according to the embodiment can be applied independently to other detection and segmentation models.

[0015] Hereinafter, an image segmentation device, an image segmentation method, and a storage medium according to the embodiment will be described in detail with reference to the accompanying drawings. Note that the image segmentation device, the image segmentation method, and the storage medium according to the embodiment are not limited to the embodiment shown below. In the following description, similar components are given common reference numerals, and duplicated descriptions are omitted.

[0016] First, an overview of the segmentation device according to the embodiment will be described. The image segmentation device according to the embodiment may exist in the form of a medical image diagnostic device such as an ultrasound diagnostic device, a CT imaging device, or an MRI imaging device, or may exist independently in the form of a workstation or the like.

[0017] FIG. 1 is a block diagram showing an example of a configuration of an image segmentation device according to an embodiment. The image segmentation device 1 according to the embodiment performs segmentation on a segmentation target included in an image using a deep learning neural network. As shown in FIG. 1, the image segmentation device 1 mainly includes a variable field of view function calculation means 10, a patch generation means 20, a training means 30, and an inference means 40. The variable field of view function calculation means 10 receives an image to be processed, and calculates a variable field of view function having a statistical amount obtained by statistically calculating shape features of a plurality of segmentation targets of different sizes as a variable, for adaptively generating a field of view range of a size corresponding to the plurality of segmentation targets. The patch generation means 20 generates patches having sizes corresponding to the scales of the segmentation targets, using the variable field of view function calculated by the variable field of view function calculation means 10, for the segmentation target. The training means 30 trains a segmentation model using the patches generated by the patch generation means 20. The inference means 40 uses the segmentation model trained by the training means 30 to infer the image and obtain a final segmentation result for the segmentation target.

[0018] The image segmentation device 1 according to the embodiment may be installed in an image segmentation device such as an ultrasound diagnostic device. In this case, the image segmentation device 1 further includes a control unit, an ultrasound probe, a display, an input / output interface, and a device main body, which are not shown in the figure, and the variable field of view function calculation means 10, the patch generation means 20, the training means 30, and the inference means 40 are installed in the control unit and are communicatively connected to the ultrasound probe, the display, the input / output interface, and the device main body. The configurations and functions of the control unit, the ultrasound probe, the display, the input / output interface, and the device main body are well known to those skilled in the art, so detailed descriptions thereof will be omitted. Note that, although an example in which the image segmentation device 1 is installed in an ultrasound diagnostic device has been described, the image segmentation device 1 may be installed in other medical image diagnostic devices such as a CT imaging device or an MRI imaging device in a similar manner.

[0019] An example of the configuration of the image segmentation device 1 shown in Fig. 1 is shown in Fig. 2. For example, each of the means shown in Fig. 1 is realized by a processing circuit 100 in Fig. 2.

[0020] Specifically, Fig. 2 illustrates a medical image processing system including an image segmentation device 1. The medical image processing system in Fig. 2 includes the image segmentation device 1, a medical image diagnostic device 2, and an image storage device 3, which are communicatively connected via a network NW.

[0021] The medical image diagnostic device 2 is a device that collects medical images from a subject. As described above, the type of the medical image diagnostic device 2 is not particularly limited, and any type of modality device, such as a CT imaging device or an MRI imaging device, can be used. In addition, the medical image processing system may include multiple types of medical image diagnostic devices 2.

[0022] The image storage device 3 is a device that stores medical images collected by the medical image diagnostic device 2. In the embodiment, the medical images are described as including data collected from a subject by the medical image diagnostic device 2 and various data generated from the collected data. For example, in the case of a CT imaging device, raw data is collected from a subject by executing a CT scan, a reconstructed image is reconstructed from the raw data, various image processing is performed on the reconstructed image to generate a display image, and the display image is displayed on a display. In the following description, the raw data, the reconstructed image, the display image, etc. are not particularly distinguished from each other, and are simply described as "medical images". The image storage device 3 is, for example, a server of a PACS (Picture Archiving and Communication System).

[0023] 2, the image segmentation device 1 includes a processing circuit 100 and a memory 200. The configuration of the image segmentation device 1 can be adjusted as appropriate, and may include, for example, a display, an input / output interface, and the like.

[0024] The memory 200 is realized by, for example, a semiconductor memory element such as a random access memory (RAM), a flash memory, a hard disk, an optical disk, etc. For example, the memory 200 stores medical images collected by the medical image diagnostic apparatus 2, programs for enabling circuits included in the image segmentation apparatus 1 to realize their functions, etc. The memory 200 may be realized by a group of servers (cloud) connected to the image segmentation apparatus 1 via a network NW.

[0025] The processing circuit 100 includes a variable field of view function calculation function 110, a patch generation function 120, a training function 130, and an inference function 140. For example, the processing circuit 100 functions as the variable field of view function calculation function 110 by reading a program corresponding to the variable field of view function calculation function 110 from the memory 200 and executing it. In the same manner, the processing circuit 100 functions as the patch generation function 120, the training function 130, and the inference function 140.

[0026] The variable field function calculation function 110 realized by the processing circuit 100 is an example of the variable field function calculation means 10 shown in Fig. 1 and is also an example of the variable field function calculation section. The patch generation function 120 is an example of the patch generation means 20 and is also an example of the patch generation section. The training function 130 is an example of the training means 30 and is also an example of the training section. The inference function 140 is an example of the inference means 40 and is also an example of the inference section.

[0027] In the image segmentation device 1 shown in Fig. 2, each processing function is stored in the memory 200 in the form of a program executable by a computer. The processing circuit 100 is a processor that realizes a function corresponding to each program by reading and executing the program from the memory 200. In other words, the processing circuit 100 in a state in which a program has been read has a function corresponding to the read program.

[0028] 2, the variable visual field function calculation function 110, the patch generation function 120, the training function 130, and the inference function 140 are realized by a single processing circuit 100, but the processing circuit 100 may be configured by combining multiple independent processors, and each processor may execute a program to realize the functions. Also, each processing function of the processing circuit 100 may be realized by being appropriately distributed or integrated into a single or multiple processing circuits.

[0029] The processing circuit 100 may also realize functions by using a processor of an external device connected via a network NW. For example, the processing circuit 100 reads out and executes a program corresponding to each function from the memory 200, and realizes each function shown in FIG. 2 by using a server group (cloud) connected to the image segmentation device 1 via the network NW as a computational resource.

[0030] FIG. 3 is a comparison diagram for explaining features of the processing executed by the image segmentation device according to the embodiment.

[0031] The features of the processing according to this embodiment will be described in comparison with the flow of the conventional technology in Fig. 3. The left side of Fig. 3 is a flowchart of image processing in the conventional technology, and the right side of Fig. 3 is a flowchart of processing executed by the image segmentation device according to this embodiment.

[0032] As shown in FIG. 3, among the processes executed by the image segmentation device according to this embodiment, steps S300 and S400 in the training process and steps S700 and S800 in the inference process are characteristic steps of this embodiment. In the prior art, training of a fine model generates training patches by randomly selecting and trimming, similar to training of a coarse model. In contrast, in this embodiment, step S300 of calculating an adaptive FOV function (variable field of view function) is added to training of the fine model. Furthermore, in this embodiment, in step S400, a patch having a size according to the scale of the segmentation target is generated for training using the adaptive FOV function calculated in step S300. In the inference process, the segmentation processing according to this embodiment also adds step S700 to the fine model inference, and in this step, a patch of an appropriate size is obtained based on the coarse inference result using the variable field of view function, and further, the fine model is inferred using the patch of the appropriate size. It should be noted that the processes of steps S700 and S800 in the inference process are not essential and can be omitted as appropriate. Details will be described later.

[0033] By the processing in the above feature step, the image segmentation device according to this embodiment calculates a variable field of view function for multi-scale segmentation objects in an image, with the statistics of the shape features of multiple segmentation objects as variables, and generates patches of corresponding sizes using the generated variable field of view function, and then performs image segmentation. Therefore, this embodiment is suitable for segmentation tasks in which the scale and shape of the segmentation objects are significantly different. According to this embodiment, the technical problem that the segmentation accuracy of small objects is insufficient and the segmentation of large objects is incomplete can be solved, and a highly accurate segmentation result can be obtained.

[0034] The process executed by the image segmentation device according to the present embodiment has been outlined above. Next, the present embodiment will be described in detail using semi-automatic segmentation and fully automatic segmentation as examples.

[0035] (First embodiment) In the first embodiment, semi-automatic segmentation will be described. Fig. 4 is a flowchart showing the first embodiment of the image segmentation process executed by the image segmentation device according to the present embodiment. The image segmentation process is divided into two main parts, a training process and an inference process. First, the processing of the training process will be described with reference to the drawings.

[0036] In the training process, first, in step S10, the image segmentation device 1 receives an image to be segmented for training and a label image of an object contained in the image.

[0037] Next, the image segmentation device 1 trains the coarse model through steps S100 and S200.

[0038] Specifically, in step S100, the image segmentation device 1 first selects a foreground / background region to be cropped at a certain ratio, and then crops the selected region to be cropped at a fixed resolution to obtain a patch image and a patch label. Next, the image segmentation device performs preprocessing including operations such as normalization to generate training patches.

[0039] Next, in step S200, the image segmentation device 1 trains a coarse segmentation model with the Patch images and Patch labels obtained in step S100. The coarse segmentation model can be realized by, for example, a 3D U-Net model.

[0040] On the other hand, the image segmentation device 1 according to this embodiment trains a fine model through steps S300 to S500.

[0041] Specifically, in step S300, the image segmentation device 1 calculates, by the variable field of view function calculation means 10, a variable field of view function, which is a function whose variables are statistics obtained by statistically analyzing the shape features of multiple segmentation objects at different scales, and which adaptively generates field of view ranges of corresponding sizes for multiple segmentation objects at different scales.

[0042] Step S300 will be described in detail below with reference to FIGS. 5A, 5B, 6A, and 6B.

[0043] 5A and 5B are schematic diagrams for explaining the process of calculating an adaptive FOV function. Fig. 5A is a flowchart showing the process of calculating a variable field of view function by the image segmentation device according to the first embodiment. Fig. 5B is a schematic diagram showing statistics used when the image segmentation device according to the first embodiment calculates a variable field of view function.

[0044] 5A, step S300 may include steps S301 and S302. The variable visual field function calculation means 10 calculates the variable visual field function through steps S301 and S302.

[0045] Specifically, the variable visual field function calculation means 10 first performs statistical calculations on shape features of each segmentation object in the training set image in step S301 to obtain statistics. The shape features here are features related to the shape of each segmentation object, and preferably scale features related to the scale of each segmentation object. Specifically, the scale features can include the lengths of the bounding box of the segmentation object in each direction in a three-dimensional space, the volume size of the segmentation object, the radius of the segmentation object, and the major and minor axes. FIG. 5B shows a distribution of the scale features of each object as an example of statistics. Specifically, the left diagram of FIG. 5B shows a distribution example of len_z, len_y, and len_x, which are the lengths of the bounding box of each object in the z-axis, y-axis, and x-axis directions. The right diagram of FIG. 5B shows a distribution example of the volume of each object.

[0046] The scale feature of the segmentation target is an example of a shape feature. The length in each direction in the three-dimensional space of the bounding box of the segmentation target, the volume of the segmentation target, the radius of the segmentation target, the major and minor axes, etc. are examples of scale features. The variable field of view function of this embodiment is generated from statistics of the shape features of the segmentation target, and uses the statistics as variables. In generating the variable field of view function, shape features related to the shape of each segmentation target are used, and data distribution statistics as shown in FIG. 5B are performed on the shape features to obtain the required statistics.

[0047] Details of the shape feature statistics in step S301 will become apparent from the process of calculating the adaptive FOV function, which will be described later with reference to FIGS. 6A and 6B.

[0048] Returning now to the explanation of FIG. 5A.

[0049] After step S301, in step S302, the variable visual field function calculation means 10 calculates a variable visual field function based on the statistics of the shape features obtained in step S301.

[0050] The method by which the variable field function calculation means 10 calculates the variable field function in step S302 can include at least two methods: a method of fitting the variable field function directly from the statistics of the shape features of the segmentation target in the dataset, and a method of obtaining the variable field function by learning the statistics of the shape features.

[0051] Hereinafter, the process of step S302 in the above two methods will be described in detail with reference to FIGS. 6A and 6B.

[0052] FIG. 6A is a schematic diagram of a first mode of processing for calculating a variable field of view function by the image segmentation device according to the first embodiment.

[0053] In an example of a first method for the image segmentation device according to the first embodiment to calculate a variable field of view function shown in Fig. 6A, in step S302, the variable field of view function calculation means 10 designs a variable field of view function by fitting, in which the statistics of the shape features of each object included in the data set obtained in step S301 are used as variables and the FOV size is used as a function value. The parameters of this variable field of view function are set according to the statistics.

[0054] The variable viewing function calculation means 10 preferably uses a monotonically increasing linear function as the variable viewing function. That is, the variable viewing function calculation means 10 preferably designs the variable viewing function so that the FOV size increases as the statistics of the shape feature of the object increases. More preferably, the variable viewing function calculation means 10 uses a piecewise linear function as the variable viewing function, as shown in Fig. 6A, in which the value of the function does not change when the variable is at a minimum or maximum, and the value of the function monotonically increases when the variable is between a minimum value and a maximum value.

[0055] In the first embodiment shown in Fig. 6A, the average value "mean(len_z, len_y, len_x)" of len_z, len_y, and len_x, which are the lengths in the z-axis, y-axis, and x-axis directions of the bounding box of each object, is used as a statistic used to generate the piecewise linear function. That is, the piecewise linear function shown in Fig. 6A as an example is a function FOV in which the average value "mean(len_z, len_y, len_x)" of len_z, len_y, and len_x of each object is a variable, and the size of the field of view range is the function value. Here, the parameters of the function FOV are set according to statistics so that the size of the field of view range is constant near both ends of the horizontal axis, i.e., when the average value of len_z, len_y, and len_x, “mean(len_z, len_y, len_x)”, is smaller than the minimum value or larger than the maximum value, and so that the size of the field of view range increases monotonically and linearly as the average value of len_z, len_y, and len_x, “mean(len_z, len_y, len_x)”, increases in the middle part of the horizontal axis, i.e., when the average value of len_z, len_y, and len_x, “mean(len_z, len_y, len_x)”, is between the minimum and maximum values.

[0056] The piecewise linear function FOV having the above characteristics can be expressed by the following formula (1).

[0057]

number

[0058] Here, FOV is the size of the field of view, len_z, len_y, and len_x are the lengths in the z, y, and x directions in the three-dimensional space of the object to be segmented, respectively, and parameters a, b, c, and d are adjustable hyperparameters related to the statistics of the scale features (in this example, “mean(len_z, len_y, len_x)”).

[0059] In formula (1), a and b respectively represent the minimum and maximum values ​​of the statistical variable at the two inflection points of the piecewise linear function, c represents the intermediate value of the statistical variable, where b>c>a, and the parameter d is for ensuring an appropriate space between the field of view FOV and the segmentation target. The parameter d is also referred to as the space adjustment parameter.

[0060] The principle by which the piecewise linear function exemplified by equation (1) can adaptively generate visual field ranges of corresponding sizes for segmentation targets of different scales will be described in detail below.

[0061] As shown in FIG. 6A, according to formula (1), when the variables “mean(len_z, len_y, len_x)>b”, i.e., when the average value of len_z, len_y, and len_x is greater than the maximum value b, the result of finding “min(b,mean(len_z,len_y,len_x))” in formula (1), i.e., the minimum value of b and “mean(len_z,len_y,len_x)”, is given by the following formula (2).

[0062]

number

[0063] In this case, since "b>a", the result of finding the maximum value for "max(a,min(b,mean(len_z,len_y,len_x)))" in equation (1), i.e., a and "min(b,mean(len_z,len_y,len_x))=b", is the following equation (3).

[0064]

number

[0065] In other words, when "mean(len_z, len_y, len_x)>b", that is, when the average value of len_z, len_y, and len_x is greater than the maximum value b, formula (1) becomes formula (4) below.

[0066]

Number

[0067] On the one hand, when "mean(len_z, len_y, len_x) < a", that is, when the average value of len_z, len_y, and len_x is smaller than the minimum value a, since "b > a", it necessarily follows that "mean(len_z, len_y, len_x) < b". Therefore, "min(b, mean(len_z, len_y, len_x))" in Equation (1) becomes the following Equation (5).

[0068]

Number

[0069] Also, since "mean(len_z, len_y, len_x) < a", it becomes the following Equation (6).

[0070]

Number

[0071] That is, when "mean(len_z, len_y, len_x) < a", that is, when the average value of "len_z", len_y, and len_x is smaller than the minimum value a, Equation (1) becomes the following Equation (7).

[0072]

Number

[0073] When "mean(len_z, len_y, len_x)" is between a and b, that is, when the average value of len_z, len_y, and len_x is between the minimum value a and the maximum value b, since "mean(len_z, len_y, len_x) ≤ b", it becomes the following Equation (8).

[0074]

number

[0075] In addition, since "mean(len_z, len_y, len_x)≧a", the following formula (9) holds.

[0076]

number

[0077] In other words, when "a≦mean(len_z, len_y, len_x)≦b", that is, when the average values ​​of len_z, len_y, and len_x are between the minimum value a and the maximum value b, equation (1) becomes the following equation (10).

[0078]

number

[0079] Therefore, equation (1) can be simplified to a piecewise function such as equation (11) below.

[0080]

number

[0081] In other words, the variable field of view function FOV exemplified in equation (1) is a piecewise linear function with variables len_z, len_y, and the average value of len_x, “mean(len_z, len_y, len_x)”, and when the variable “mean(len_z, len_y, len_x)” is smaller than the minimum value a or larger than the maximum value b, the size of the field of view range is fixed to “a / c×d” and “b / c×d”, respectively, and when the variable “mean(len_z, len_y, len_x)” is between the minimum value a and the maximum value b, the size of the field of view range increases linearly as “mean(len_z, len_y, len_x)” increases.

[0082] Next, a method for setting the hyperparameters a, b, c, and d in the first embodiment will be described in detail.

[0083] In the first embodiment shown in FIG. 6A, the minimum value a is, for example, the length value at the 10% quantile, i.e., the first decile, of the length distribution of all segmentation objects in the training set; the maximum value b is, for example, the length value at the 90% quantile, i.e., the 9th decile, of the length distribution of all segmentation objects in the training set; the intermediate value c is, for example, the length value at the median, of the length distribution of all segmentation objects in the training set; and the spatial adjustment parameter d is, in accordance with the intermediate value c being the median, the product of the median resolution IR of the image and the matrix size "Patch_size" input to the deep learning model, i.e., "d=IR×Patch_size".

[0084] For example, the length distribution in the training dataset is 100 consecutive integer lengths from 1mm to 100mm, IR is 1mm / pixel, and Patch_size is 96 pixels. In this case, if a=10mm, b=90mm, and c=50mm, the above analysis shows that when the length of the object to be segmented is small (1 to 10mm), FOV=a / c×d=a / c×(IR×Patch_size)=10mm / 50mm×(1mm / pixel) When the length of the object to be segmented is large (90 to 100 mm), "FOV=b / c×d=b / c×(IR×Patch_size)=90mm / 50mm×(1mm / pixel×96 pixels)=172.8mm", and when the length of the object to be segmented is "10 to 90 mm", the FOV is "19.2mm to 172.8mm", which is a value that changes in proportion to the length of the object to be segmented. As a result, the adaptive FOV function of this embodiment becomes a piecewise linear function as shown in the lower center of Figure 6A, where when the scale of the object to be segmented is small, the corresponding FOV is also small (lower left side of Figure 6A), and when the scale of the object to be segmented is large, the corresponding FOV is also large (lower right side of Figure 6A), maintaining an appropriate space between the object to be segmented and the corresponding FOV. By generating such an adaptive FOV function, it is possible to obtain an FOV of an appropriate size according to the scale of the object to be segmented.

[0085] The specific settings of the hyperparameters a, b, c, and d above are merely examples, and the embodiment is not limited thereto.

[0086] For example, in the above example, a takes the length value at the 10% quantile of the length distribution of all segmentation objects in the training set, b takes the length value at the 90% quantile of the length distribution of all segmentation objects in the training set, and c takes the length value at the median of the length distribution of all segmentation objects in the training set; however, it is clear that a and b, which respectively represent the minimum and maximum values ​​of the abscissa at two inflection points of the piecewise linear function, can also take other values; for example, a can take the length value at the 15% quantile, the 20% quantile, etc. of the length distribution of all segmentation objects in the training set, b can take the length value at the 85% quantile, the 80% quantile, etc. of the length distribution of all segmentation objects in the training set; similarly, the parameter c can take other values ​​other than the median, for example reflecting the average value of the length distribution of all segmentation objects in the training set or the average value of the entire scale feature. In the above example, d is the product of the median resolution IR of the image (e.g., 1 mm / pixel) and the matrix size Patch_size (e.g., 96 pixels) input to the deep learning model. The parameter d is for ensuring an appropriate space between the field of view and the segmentation target. In the above example, the median resolution IR of the image is set to "1 mm / pixel" and the matrix size Patch_size input to the deep learning model is set to "96 pixels". However, the embodiment is not limited to this, and the median resolution IR of the image and the matrix size Patch_size input to the deep learning model may be other values ​​as necessary. In the first embodiment, the scale feature of the target to be segmented is the length of the bounding box, and the parameter d is set to "d=IR×Patch_size" accordingly. However, when the FOV function is set using a scale feature other than the length of the bounding box as a variable, the parameter d may take other corresponding values. The hyperparameters a, b, c, and d can be applied to this embodiment as long as they are adjustable hyperparameters related to the statistics of the scale feature.

[0087] In the above example of the first embodiment, the average value "mean(len_z, len_y, len_x)" of the lengths of the z-axis, y-axis, and x-axis directions of the bounding box of each object is used as a statistic, and the variable visual field function is designed based on the average value. However, the embodiment is not limited to this, and the average value may be calculated by taking into account the radius of each object in addition to the lengths of the z-axis, y-axis, and x-axis directions of the bounding box of each object. In other words, the variable visual field function may be designed using the average value "mean(len_z, len_y, len_x, r)" of the lengths and radii of the z-axis, y-axis, and x-axis directions of the bounding box of each object, that is, len_z, len_y, len_x, and r, as a statistic. In this case, the piecewise linear function of the variable visual field function can be expressed by the following formula (12).

[0088]

number

[0089] In equation (2), FOV, len_z, len_y, len_x, a, b, c, and d are the same as in the embodiment described above for equation (1), and r is the radius of the segmentation object.

[0090] In addition to the average value, the median, mode, quartile, etc. can be used as the statistical quantity depending on the situation. Similarly, the parameters of the variable visual field function and the functional form itself can be modified in various ways, and any linear increasing function designed by fitting with the statistical quantity of the shape feature of the object to be segmented as a variable can be applied to this embodiment.

[0091] The above describes the first method of processing by which the image segmentation device of the first embodiment calculates a variable field function. Next, we will describe the second method of processing by which the image segmentation device of the first embodiment calculates a variable field function.

[0092] FIG. 6B is a schematic diagram showing a second method of processing for calculating a variable field of view function by the image segmentation device according to the first embodiment.

[0093] As shown in FIG. 6B, when the image segmentation device according to the first embodiment calculates the variable field of view function, the second method is to obtain the variable field of view function by automatic fitting using a neural network.

[0094] For example, in step S302, the variable field of view function calculation means 10 obtains an FOV function by using the statistics of the shape features obtained in step S301 as input to a neural network, setting the optimal FOV as a prediction target, and optimizing the difference between the predicted value and the optimal FOV as a loss function.

[0095] In the second method, the statistics of the shape features that are input to the neural network can be the average values ​​of len_z, len_y, and len_x, or the average values ​​of len_z, len_y, len_x, and r, as in the first method, and the loss function loss that is input to the neural network in the second method can be L1 loss or L2 loss.

[0096] Returning now to the description of the flow in Fig. 4. After step S300, the process proceeds to step S400, where the image segmentation device 1 uses the variable field of view function calculated in step S300 to generate training patches having a size according to the scale of the object to be segmented.

[0097] Details of step S400 will be described below with reference to Fig. 7. Fig. 7 is a schematic diagram showing the process of generating a patch of an appropriate size by the image segmentation device according to the first embodiment.

[0098] In the example shown in FIG. 7, step S400 includes steps S401 to S403. The training means 30 generates training patches of an appropriate size through steps S401 to S403. Specifically, the training means 30 first randomly selects n objects to be segmented in the training set in step 401. In FIG. 7, segmentation target 1, segmentation target 2, and segmentation target n are illustrated as the n objects to be segmented. As illustrated in FIG. 7, segmentation target 1, segmentation target 2, and segmentation target n are three segmentation targets having clearly different scales. Note that, although only three segmentation targets are illustrated in FIG. 7, the training means 30 can also select four or more segmentation targets.

[0099] Next, in step S402, the training means 30 uses the variable field of view function calculated in step S300 for segmentation target 1, segmentation target 2, ..., segmentation target n, and substitutes the shape features of each segmentation target as variables into the variable field of view function to obtain field of view ranges FOV1, FOV2, ..., FOVn corresponding to each segmentation target. The details and effects of obtaining an appropriate size field of view range by the variable field of view function using the shape features of each segmentation target as variables in step S402 are described in detail above with reference to Figures 6A and 6B, so that description thereof will be omitted here.

[0100] Next, in step S403, the training means 30 generates patch 1, patch 2, patch n for each of segmentation target 1, segmentation target 2, ..., segmentation target n using the field of view ranges FOV1, FOV2, ..., FOVn of appropriate sizes calculated in step S402, and uses them for training. The process of generating patches in step S403 can adopt any method known in the art, and for example, by trimming the field of view ranges FOV1, FOV2, ..., FOVn as target sizes with the center of gravity of the segmentation target as the center, a training patch having a size according to the scale of the segmentation target can be obtained. The details of the generation of training patches can be realized by various methods in the prior art, so further detailed explanation will be omitted here.

[0101] Returning to the description of the flow in Fig. 4, after step S400, the process proceeds to step S500, where the image segmentation device 1 trains a fine segmentation model using training patches of an appropriate size according to the scale of the object to be segmented obtained in step S400 and the label image of the corresponding object. In the first embodiment, it is preferable to use 3D U-Net as the fine segmentation model.

[0102] It should be noted that the deep learning model and neural network training can be implemented in a variety of ways in the prior art, and therefore a detailed description thereof will be omitted here.

[0103] The training process in the image segmentation process according to the first embodiment has been described above. Next, an inference process in the image segmentation process will be described.

[0104] In the inference process of the image segmentation processing according to the first embodiment, the inference means 40 uses steps S600 to S800 shown in FIG. 4 to infer an image using the segmentation model trained by the training means 30, and obtains a final segmentation result.

[0105] In a preferred embodiment, the inference means 40 may include a first segmentation means and a second segmentation means. The first segmentation means infers an image using a coarse model (first segmentation model) in step S600 to obtain a coarse inference result (first segmentation result). The second segmentation means generates a first segmentation result patch having an appropriate size using a variable field function calculated by the variable field function calculation means based on the coarse inference result (first segmentation result) in step S700, and further infers the first segmentation result patch using a fine segmentation model (second segmentation model) in step S800 to obtain a final segmentation result. The first segmentation means and the second segmentation means are also referred to as a first segmentation unit and a second segmentation unit, respectively.

[0106] The inference process of the image segmentation processing according to the first embodiment will be described in detail below with reference to Fig. 8. Fig. 8 is a schematic diagram showing detailed processing of the inference process in the image segmentation processing.

[0107] In the inference process shown in Fig. 8, the inference means 40 first performs inference of a coarse model in step S600 to obtain a coarse inference result. The coarse inference process can be performed by inputting, for example, an original image and a seed point to the coarse inference model. Specifically, for example, a user specifies a seed point of an object to be segmented, and a patch of a fixed size is trimmed around the seed point and input to the coarse segmentation model, and the coarse segmentation model makes a prediction to obtain a coarse inference result.

[0108] The inference of the coarse model can be achieved in various ways in the prior art, and therefore will not be described in further detail here.

[0109] Next, in step S700, the inference means 40 calculates a field of view range FOV of an appropriate size for each rough inference result obtained in step S600 using a variable field of view function, and generates a patch of an appropriate size. The process of step S700 is similar to the process of step S400 in the training process, particularly steps S402 to S403, and therefore a detailed description thereof will be omitted here.

[0110] Next, in step S800, the inference means 40 inputs the appropriate size patch obtained in step S700 into a fine segmentation model, which predicts to obtain the final segmentation result.

[0111] As described above, the image segmentation device according to this embodiment first calculates a variable field of view function for a multi-scale segmentation target in an image in a training process, with the statistics of shape features of multiple segmentation targets as variables. In addition, the image segmentation device according to this embodiment generates patches of a corresponding size for the multi-scale segmentation target in an inference process using the generated variable field of view function, and then segments the image using the model trained in the training process. Therefore, this embodiment can adaptively perform segmentation on the target to be multi-scale segmented.

[0112] As mentioned above, in the task of semi-automatic medical image segmentation, specifying too large an FOV will result in insufficient segmentation accuracy for small objects, resulting in a waste of computational resources, while specifying too small an FOV will tend to result in incomplete coverage of objects.

[0113] In the prior art, some countermeasures have been taken for image segmentation to solve the above problems. A specific example of the countermeasure is the sliding window technique, which divides the entire image into fixed-size patches in a sliding window format, performs inference, and integrates the inference results of each patch to obtain segmentation results for all objects. Another countermeasure is hierarchical segmentation, i.e., adopting models with different segmentation accuracy from coarse-to-fine. In hierarchical segmentation, different models are usually trained using images with two different resolutions. The coarse model generally has a low resolution, a large FOV, and is good at segmenting large objects. The fine model generally has a high resolution, a small FOV, and is good at segmenting small objects.

[0114] As described above, regardless of which method is adopted, in conventional technology, when segmenting an image, the same processing method is often used for all objects without taking into account the scale differences of the objects to be segmented, resulting in a problem that the FOV is insufficient for large segmentation objects, resulting in incomplete segmentation, and the FOV is too large for small segmentation objects, resulting in insufficient segmentation accuracy.

[0115] On the other hand, in conventional medical imaging technologies, such as related technologies such as X-ray diagnosis and endoscopic imaging, the size of the imaging area can be changed by zooming, which changes the shooting distance by moving the bed or imaging device. This allows doctors to select an appropriate observation field by enlarging or reducing the image even in their daily interpretation of images.

[0116] Therefore, in processes such as medical image segmentation based on machine learning, there is a demand for technology that can adaptively select an FOV based on different segmentation target scales and automatically enlarge small objects and reduce large objects in the observation field of view, like a doctor reading an image, to obtain highly accurate segmentation results.

[0117] In contrast, according to the present embodiment, it is possible to adaptively perform segmentation on an object to be segmented at a multi-scale. That is, according to the present embodiment, it is possible to solve the technical problem of insufficient segmentation accuracy for small objects and incomplete segmentation for large objects, and to obtain a highly accurate segmentation result.

[0118] Although the inference process according to the first embodiment has been described above, the embodiment is not limited to this, and the inference process of the image segmentation processing according to this embodiment can be modified in various ways.

[0119] For example, after step S700, a judgment step may be added to judge whether the FOV calculation result in step S700 and the FOV of the coarse model inference result in step S600 are close to each other. If the FOV calculation result in step S700 and the FOV of the coarse model inference result in step S600 are close to each other, the fine model segmentation in step S800 can be skipped and the coarse model segmentation result in step S600 can be used as the final segmentation result as it is. In this case, the calculation result based on the adaptive FOV method of this embodiment is used as a judgment criterion in the judgment step, so that it is guaranteed to obtain a highly accurate segmentation result as well as to obtain beneficial effects such as improved processing speed and saving of computational resources.

[0120] In addition, the FOV calculation result in step S700 may not be the optimal result in some special cases, such as when the shape, scale, or contrast of the segmentation target is special. In this case, if the segmentation result in step S800 is used as the final segmentation result, good segmentation accuracy may not be obtained. Therefore, this embodiment may further provide a mode that supports a selection algorithm. That is, after step S800, a selection step may be added to select a better segmentation result from the coarse model inference result in step S600 and the fine model inference result in step S800 as the final segmentation result. This selection step may be performed, for example, by setting a predetermined threshold value in advance for a technical index indicating the result of the inference process and comparing the index in the coarse model inference result and the index in the fine model inference with the threshold value, or by a user of the image segmentation device 1 manually selecting according to a predetermined criterion.

[0121] The above-described determination of whether the FOV is close or not and the criteria for that determination, as well as the algorithm for selecting the segmentation result and the criteria for that selection, can all be realized in various ways in the prior art and are not the gist of this embodiment, so detailed explanations will be omitted here.

[0122] According to the first embodiment, the variable field of view function calculation means does not simply expand the FOV of the coarse model outward by a certain size as in the prior art, but calculates an adaptive FOV function by fitting or learning based on statistics of the shape features of all objects in the entire data set, and generates a corresponding FOV based on the adaptive FOV function, so that the FOV accuracy of this embodiment does not depend on the accuracy of the coarse model. Even if the accuracy of the coarse model segmentation is low or fails, the FOV function obtained according to this embodiment can operate to obtain an appropriate FOV. Note that the shape feature used in this embodiment is not unique, and may be a single feature or multiple features. For example, the shape feature used in this embodiment may be a scale feature such as the length in each direction in the three-dimensional space of the bounding box of the segmentation object, the volume of the segmentation object, the radius of the segmentation object, and the long and short diameters of the segmentation object, or other shape features that can characterize the scale or shape of the object.

[0123] According to the first embodiment described above, it is possible to adaptively perform segmentation on multi-scale segmentation targets, thereby resolving the technical issues of insufficient segmentation accuracy for small targets and incomplete segmentation for large targets, and obtaining highly accurate segmentation results.

[0124] Based on the technical solution of the first embodiment described above, the inventors conducted a test on the segmentation of lung late stage tumor medical images and evaluated the segmentation results. When Dice Similarity Coefficient (DSC) is used as the evaluation criterion, the DSC of the segmentation result of the conventional technology is "0.706", while the DSC of the segmentation result of the first embodiment is "0.774", and the segmentation accuracy of this embodiment is obviously superior to that of the conventional technology.

[0125] Second embodiment The above describes the first semi-automatic embodiment in which an original image and seed points are input into an inference model in step S600 to perform inference, but the embodiment is not limited to this and can also be applied to, for example, fully automatic segmentation.

[0126] The second embodiment is a case where the embodiment is applied to fully automatic segmentation, and the second embodiment will be described in detail below with reference to FIG.

[0127] In the description of the second embodiment, differences from the first embodiment will be mainly described. Note that in the description of the second embodiment, the same components as those in the first embodiment will be denoted by the same reference numerals and the description thereof will be omitted.

[0128] FIG. 9 is a flowchart showing an image segmentation process executed by the image segmentation device according to the second embodiment.

[0129] 9, in the inference process of the second embodiment, unlike the process of step S600 of the first embodiment, in the process of step S600', coarse model inference is performed fully automatically in a sliding window format. That is, in the inference process of the second embodiment, the entire original image is traversed with a sliding window to obtain patches, inference is performed on each patch to obtain an inference result for each patch, and the inference results for each patch are integrated to obtain a coarse inference result for the entire image.

[0130] The other steps of the image segmentation process according to the second embodiment are similar to those of the first embodiment, so a detailed description thereof will be omitted here.

[0131] According to the second embodiment, the embodiment can be applied to fully automatic segmentation, and adaptive segmentation can be performed on multi-scale segmentation targets, thereby resolving the technical issues of insufficient segmentation accuracy for small targets and incomplete segmentation for large targets, and achieving highly accurate segmentation results similar to those of the first embodiment.

[0132] The inventors have conducted a test on the segmentation of renal tumor medical images based on the technical solution of the second embodiment described above, and evaluated the segmentation results. When DSC is used as the evaluation criterion, the DSCs of the segmentation results of the single coarse segmentation and the stepwise segmentation from coarse to fine segmentation in the conventional technology are "0.825" and "0.855", respectively, while the DSC of the segmentation result according to the second embodiment is "0.867". In addition, for segmentation objects of different sizes, the coarse-to-fine stepwise segmentation of the conventional technology has DSCs of 0.768 and 0.838 for renal tumor radii of 20 mm or less and 40 mm or less, respectively, whereas the DSCs of the segmentation results of the second embodiment are 0.811 and 0.860, respectively, indicating that the present embodiment has superior segmentation accuracy for each segmentation method and each scale of the segmentation object compared to the conventional technology.

[0133] (Other embodiments) In the inference process of the first and second embodiments described above, the processing based on the coarse model and the fine model in steps S600 (S600') to S800 is a preferred embodiment, but the processing in steps S700 and S800 is not essential; that is, in step S600 (S600'), an image may be inferred using a model trained in the training process of the embodiment, and the obtained segmentation result may be used as the final inference result as it is.

[0134] Although the embodiment infers an image using a model trained by the training means 30 has been described above, the coarse segmentation model in the inference process of the embodiment can be applied to any existing segmentation model that can obtain scale information of the object to be segmented, such as an object detection model, a model based on a conventional threshold segmentation method, etc. The coarse model segmentation may be manual segmentation. The coarse segmentation model of the embodiment may be any one of a deep learning segmentation model, a landmark model, an object detection model, and a graph cut segmentation model.

[0135] Although the above embodiment has been described taking segmentation as an example, it goes without saying that the embodiment can also be applied to other image processing such as detection other than segmentation.

[0136] The embodiment is applicable to semi-automatic and fully automatic segmentation algorithms, and the variable field function calculation means 10 of the embodiment can be applied independently to other existing detection and segmentation models.

[0137] The image processing, segmentation, deep learning model, and neural network training and inference described in the above embodiments can all be realized using various methods in the prior art, so detailed explanations are omitted here.

[0138] The embodiment may be realized as the above-described image segmentation device 1, or may be realized as an image segmentation method, a program, or a medium storing an image segmentation program.

[0139] The image segmentation device 1 according to the present application may be incorporated in a medical image diagnostic device 2, or the image segmentation device 1 may execute processing independently. In such a case, the image segmentation device 1 includes a processing circuit 100 that executes processing similar to each step described above, and a memory 200 that stores programs corresponding to each function and various information, as shown in FIG. 2. The processing circuit 100 acquires two-dimensional or three-dimensional medical image data from a medical image diagnostic device 2 such as an ultrasound diagnostic device or an image storage device 3 via a network NW, and executes the above-mentioned processing using the acquired medical image data. Here, the processing circuit 100 is a processor that realizes functions corresponding to each program by reading and executing programs from the memory.

[0140] The term "processor" used in the above description means a circuit such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or an Application Specific Integrated Circuit (ASIC), a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA)). The processor realizes its function by reading and executing a program stored in a memory. Instead of storing a program in a memory, the processor may be configured to directly incorporate the program in its circuit. In this case, the processor realizes its function by reading and executing the program incorporated in the circuit. Each processor in this embodiment is not limited to being configured as a single circuit for each processor, and may be configured as a single processor by combining multiple independent circuits to realize its function.

[0141] Note that the components of each device illustrated in the above description of the embodiment are functional concepts, and are not necessarily physically configured as illustrated. In other words, the specific form of distribution and integration of each device is not limited to that illustrated, and all or a part of the devices can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions. Furthermore, all or any part of the processing functions performed by each device can be realized by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

[0142] The processing method described in the above embodiment can be realized by executing a processing program prepared in advance on a computer such as a personal computer or a workstation. This processing program can be distributed via a network such as the Internet. This processing program can also be recorded on a non-transitory recording medium readable by a computer, such as a hard disk, a flexible disk (FD), a CD-ROM, an MO, a DVD, a USB memory, or a flash memory such as an SD card memory, and can be executed by being read from the non-transitory recording medium by a computer.

[0143] According to at least one of the embodiments described above, the accuracy of image segmentation can be improved.

[0144] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents described in the claims, as well as in the scope and spirit of the invention. [Explanation of symbols]

[0145] 1: Image segmentation device 10: Variable visual field function calculation means 20: Patch generation method 30: Training methods 40: Reasoning means 100: Processing circuit 110: Variable visual field function calculation function 120: Patch generation function 130: Training function 140: Inference function 200:Memory 2: Medical imaging diagnostic equipment 3: Image Archive

Claims

1. a variable viewing function calculation unit that calculates a variable viewing function that can adaptively generate viewing ranges of sizes corresponding to a plurality of segmentation targets included in an image; a patch generator for generating patches of corresponding sizes for the plurality of segmentation objects using the variable field of view function; an inference unit for inferring the image using a segmentation model trained using the patches to obtain a segmentation result of the plurality of segmentation targets; An image segmentation apparatus comprising:

2. 2. The image segmentation device of claim 1, wherein the inference unit includes a first segmentation unit that segments the image with a first segmentation model to obtain a first segmentation result, and a second segmentation unit that generates a first segmentation result patch using the variable field of view function based on the first segmentation result, and segments the first segmentation result patch with a second segmentation model to obtain the segmentation result.

3. The image segmentation device according to claim 1 , wherein the variable field of view function has as its variable a statistical quantity obtained by statistically analyzing shape features of the plurality of segmentation objects of different sizes.

4. The image segmentation device according to claim 3 , wherein the shape features are scale features related to a scale of the segmentation object.

5. The image segmentation device according to claim 4 , wherein the scale features include at least one of the lengths in each direction in three-dimensional space of a bounding box of the segmentation object, the volume of the segmentation object, and the major and minor axes of the segmentation object.

6. The image segmentation device according to claim 5 , wherein the variable field of view function calculation unit sets a monotonically increasing linear function set based on the scale feature as the variable field of view function.

7. The variable visual field function calculation unit sets a piecewise linear function represented by the following formula (1) as the variable visual field function, In formula (1), FOV is the field of view; len_z, len_y, and len_x are the lengths in the z-direction, y-direction, and x-direction of the segmentation target in three-dimensional space, respectively; The image segmentation apparatus of claim 6 , wherein a, b, c, and d are tunable hyper-parameters for the statistics of the scale features. [0010]

8. The image segmentation device of claim 3, wherein the variable field of view function calculation unit calculates the variable field of view function by automatic fitting of a neural network that has the shape features of the multiple segmentation targets as input and the field of view range as output, and the variable field of view function calculation unit calculates the variable field of view function by adjusting a loss function of the neural network to obtain an optimal field of view range.

9. The image segmentation apparatus of claim 2 , wherein the first segmentation model is a coarse segmentation model and the second segmentation model is a fine segmentation model.

10. The image segmentation device of claim 9 , wherein the coarse segmentation model is one of a deep learning segmentation model, a landmark model, an object detection model, and a graph cut segmentation model.

11. Calculating a variable field of view function capable of adaptively generating field of view ranges of sizes corresponding to a plurality of segmentation targets included in the image; generating patches of corresponding size for the plurality of segmentation objects using the variable field of view function; Inferring the image using a segmentation model trained using the patches to obtain segmentation results for the plurality of segmentation objects. The image segmentation method includes:

Images