Three-dimensional interface adjustment apparatus, adjustment method, computer program product, and storage medium

Abstract

A three-dimensional interface adjustment device for adjusting an interface to be adjusted to a specified target position, comprising: an acquisition unit that acquires a three-dimensional medical image including a segmentation result of a substructure and a segmentation result of other structures in the vicinity of the substructure; a selection unit that selects the interface to be adjusted on the substructure; a specification unit that specifies a target position to which the interface to be adjusted is to be moved on the other structures; a calculation unit that forms a model surface by modeling the interface to be adjusted, and calculates a moving direction and distance of a point set on the model surface of the interface to be adjusted based on the target position; and an update unit that updates the interface to be adjusted based on the moving direction and distance of the point set on the model surface of the interface to be adjusted, and updates adjacent substructures using a change volume of the substructure resulting from the interface update.

Description

Technical Field

[0001] This invention relates to a three-dimensional interface adjustment device, adjustment method, computer program product, and storage medium. Background Technology

[0002] Clinically, for surgical planning and diagnosis, it is necessary to segment substructures such as liver segments, lung segments, and brain regions. There are known methods for segmentation using automatic or semi-automatic algorithms. However, in practice, there is a need for further adjustments to the results of these algorithms. For example, if the segmentation result itself is not accurate enough, it needs to be modified; or sometimes it is necessary to adjust the surgical scope and assess risks based on structural changes such as those involving blood vessels. Therefore, a method is desired that can accurately adjust the boundaries of the segmentation results.

[0003] Previously, methods were proposed to modify boundaries by dragging a vertex in a 2D image to a target location. However, updating a slice requires correction of multiple slices and smoothing between them. If a 3D update is to be achieved by dragging on a single slice, changes in other slices become uncontrollable, leading to repeated processing. Generally, doctors prefer to operate on a 3D plane that can represent three-dimensional shapes.

[0004] However, while it's possible to drag vertices to target locations to correct surfaces in 3D images, there are difficulties and limitations in locating the drag point and controlling the range of influence generated by the drag. In practice, it's necessary to find the draggable point on the 3D structure and complete the drag operation. However, 3D structures are overlapping, making it difficult to accurately locate the draggable position and the destination. Furthermore, effectively controlling the range of influence of the drag is itself challenging.

[0005] In addition, there is a method of surface fitting by clicking near the surface, but it cannot control the degree and range of change of the surface fitting very well.

[0006] Previous techniques have proposed using different strategies to automatically determine the range of changes and perform surface editing. For example, the user identifies a point of interest on the surface model, at least one point adjacent to that point is designated as the expected vertex, ranking values ​​are generated based on different purposes or strategies, and the expected vertices are identified based on their distance from the point of interest and the ranking function of each vertex. The user then moves the expected vertices to update the 3D surface model. While these techniques can pinpoint the draggable location, they also suffer from difficulties in locating the drag destination within overlapping 3D images, as well as issues with controlling the drag's influence range. Furthermore, these techniques cannot update adjacent substructures. Summary of the Invention

[0007] This invention provides a three-dimensional interface adjustment device, adjustment method, computer program product, and storage medium that automatically updates the substructure by specifying the target position to which the substructure is to be moved, rather than changing the shape of the substructure by dragging vertices on the substructure.

[0008] Technical solution one is a three-dimensional interface adjustment device for adjusting an interface to a specified target position. It comprises: an acquisition unit that acquires a three-dimensional medical image containing segmentation results of substructures and segmentation results of other structures near the substructures; a selection unit that selects the interface to be adjusted located on the substructures; a designation unit that designates a target position to which the interface to be adjusted is to be moved on the other structures; a calculation unit that models the interface to be adjusted to form a model surface, and calculates the movement direction and distance of a set of points on the model surface of the interface to be adjusted based on the target position; and an update unit that updates the interface to be adjusted based on the movement direction and distance of the set of points on the model surface of the interface to be adjusted, and updates adjacent substructures using the changes in the substructures caused by the interface update.

[0009] Technical solution two is a three-dimensional interface adjustment device for adjusting an interface to a specified target position. It comprises: an acquisition unit that acquires two-dimensional medical images from multiple perspectives containing segmentation results of substructures and a three-dimensional medical image containing segmentation results of substructures; a selection unit that selects the interface to be adjusted located on the substructure in the three-dimensional medical image; a designation unit that designates the desired target position in each of the multiple perspectives of the two-dimensional medical image; a calculation unit that models the interface to be adjusted to form a model surface, and calculates the movement direction and distance of the point set on the model surface of the interface to be adjusted based on a set of points on the multiple target positions in three dimensions; and an update unit that updates the interface to be adjusted and the substructure based on the movement direction and distance of the point set on the model surface of the interface to be adjusted.

[0010] Technical Solution 3 is a 3D interface adjustment method for adjusting an interface to a specified target position. The method includes the following steps: an acquisition step, acquiring a 3D medical image containing segmentation results of substructures and other structures near the substructures; a selection step, selecting the interface to be adjusted located on the substructures; a specification step, specifying the target position to which the interface to be adjusted should be moved on the other structures; a calculation step, modeling the interface to be adjusted to form a model surface, and calculating the movement direction and distance of the point set on the model surface of the interface to be adjusted based on the target position; and an update step, updating the interface to be adjusted based on the movement direction and distance of the point set on the model surface of the interface to be adjusted, and updating adjacent substructures using the changes in the substructures caused by the interface update.

[0011] Technical Solution 4 is a three-dimensional interface adjustment method for adjusting an interface to a specified target position, comprising the following steps: an acquisition step, acquiring two-dimensional medical images and three-dimensional medical images containing segmentation results of substructures from multiple perspectives; a selection step, selecting the interface to be adjusted located on the substructure in the three-dimensional medical image; a specification step, specifying the target position to be moved to in each of the multiple perspective two-dimensional medical images; a calculation step, modeling the interface to be adjusted to form a model surface, and calculating the movement direction and distance of the point set on the model surface of the interface to be adjusted based on the multiple target positions in three dimensions; and an update step, updating the interface to be adjusted and updating the substructure based on the movement direction and distance of the point set on the model surface of the interface to be adjusted.

[0012] Technical solution five is a computer program product comprising a program for adjusting an interface to a specified target position, wherein the program causes a computer to perform the following steps: an acquisition step, acquiring a three-dimensional medical image containing segmentation results of substructures and segmentation results of other structures near the substructures; a selection step, selecting the interface to be adjusted located on the substructures; a specification step, specifying the target position to which the interface to be adjusted is to be moved on the other structures; a calculation step, modeling the interface to be adjusted to form a model surface, and calculating the movement direction and distance of the set of points on the model surface of the interface to be adjusted according to the target position; and an update step, updating the interface to be adjusted based on the movement direction and distance of the set of points on the model surface of the interface to be adjusted, and updating adjacent substructures using the changes in the substructures caused by the interface update.

[0013] Technical solution six is ​​a computer program product comprising a program for adjusting an interface to a specified target position, wherein the program causes a computer to perform the following steps: an acquisition step, acquiring two-dimensional medical images and three-dimensional medical images containing segmentation results of substructures from multiple perspectives; a selection step, selecting the interface to be adjusted located on the substructure in the three-dimensional medical image; a specification step, specifying the target position to be moved to in each of the multiple perspectives of the two-dimensional medical images; a calculation step, modeling the interface to be adjusted to form a model surface, and calculating the movement direction and distance of the point set on the model surface of the interface to be adjusted based on the multiple point sets of the target positions in three dimensions; and an update step, updating the interface to be adjusted and updating the substructure based on the movement direction and distance of the point set on the model surface of the interface to be adjusted.

[0014] Technical solution seven is a storage medium storing a program that causes a computer to perform the following steps: an acquisition step, acquiring a three-dimensional medical image containing segmentation results of substructures and segmentation results of other structures near the substructures; a selection step, selecting the interface to be adjusted located on the substructures; a specification step, specifying a target position to which the interface to be adjusted should be moved on the other structures; a calculation step, modeling the interface to be adjusted to form a model surface, and calculating the movement direction and distance of the set of points on the model surface of the interface to be adjusted according to the target position; and an update step, updating the interface to be adjusted based on the movement direction and distance of the set of points on the model surface of the interface to be adjusted, and updating adjacent substructures using the changes in the substructures caused by the interface update.

[0015] Technical solution eight is a storage medium storing a program that causes a computer to perform the following steps: an acquisition step, acquiring two-dimensional medical images and three-dimensional medical images containing segmentation results of substructures from multiple perspectives; a selection step, selecting the interface to be adjusted located on the substructure in the three-dimensional medical image; a specification step, specifying the target position to be moved to in the two-dimensional medical images from multiple perspectives respectively; a calculation step, modeling the interface to be adjusted to form a model surface, and calculating the movement direction and distance of the point set on the model surface of the interface to be adjusted based on the multiple point sets of the target positions in three dimensions; and an update step, updating the interface to be adjusted and updating the substructure based on the movement direction and distance of the point set on the model surface of the interface to be adjusted.

[0016] Technical effect

[0017] The technical solution of this invention can generate a new interface simply by specifying the target position to which the interface to be adjusted is to be moved, and can automatically update the substructure by utilizing the changes in the substructure generated by the interface update. Compared with the methods of changing the interface through complex combination operations in the prior art, it has the advantage of simple operation.

[0018] Furthermore, the above-mentioned technical solution of the present invention does not require locating and dragging vertices on a three-dimensional surface and accurately dragging the vertices, which can improve the accuracy of interface adjustment.

[0019] Furthermore, the above-mentioned technical solutions of the present invention can correspond to the automatic updating of a single substructure or the automatic updating of multiple adjacent substructures, thus having a wider range of applications. Attached Figure Description

[0020] Figure 1 This is a block diagram showing the three-dimensional interface adjustment device of Embodiment 1.

[0021] Figure 2 This is a flowchart illustrating the three-dimensional interface adjustment method of Implementation Method 1.

[0022] Figure 3A It is a three-dimensional medical image containing the segmentation results of multiple lung segments of the right lung.

[0023] Figure 3B It was a choice Figure 3A The interface between the S1 and S3 segments of the right lung is used as the interface to be adjusted.

[0024] Figure 4 This indicates that a portion of the interface between adjacent lung segments is selected as the interface to be adjusted.

[0025] Figure 5 This is a diagram illustrating the calculation of the shortest distance to a target point specified on a vein.

[0026] Figure 6 This is a schematic diagram illustrating the updating of adjacent substructures based on the plane to be adjusted before the update and the new interface after the update.

[0027] Figure 7A and Figure 7B These represent cases where multiple target points or portions of a vein segment are specified as the target locations to be moved to.

[0028] Figure 8 It is a flowchart representing the updating of substructures based on multiple target points.

[0029] Figure 9This indicates that the entire interface of the right lung S3 segment, which is connected to the right lung S1 and S2 segments respectively, is used as the interface to be adjusted.

[0030] Figure 10 This is a block diagram showing the three-dimensional interface adjustment device of embodiment 3.

[0031] Figure 11 This is a flowchart illustrating the three-dimensional interface adjustment method of implementation method 3.

[0032] Figure 12 This is a schematic diagram illustrating the three-dimensional interface adjustment method of implementation method 3. Detailed Implementation

[0033] Hereinafter, embodiments will be described with reference to the accompanying drawings. The embodiments described below do not limit the invention. Furthermore, in the following description, constituent elements having substantially the same function and structure will be labeled with the same reference numerals, and will be repeated only where necessary. Additionally, even identical components may be shown at different scales and from different perspectives in the various drawings.

[0034] (Implementation Method 1)

[0035] The three-dimensional interface adjustment device 10 of this embodiment is as follows: Figure 1 As shown, it has an acquisition unit 11, a selection unit 12, a designation unit 13, a calculation unit 14, an update unit 15, and a display control unit 16.

[0036] The acquisition unit 11 acquires a three-dimensional medical image containing segmentation results of substructures and segmentation results of other structures near the substructures. Examples of substructures include lung segments, liver segments, and brain regions; examples of other structures near the substructures include the trachea, arteries, and veins. However, the substructures and other structures referred to in this invention are not limited to the examples listed above. Furthermore, the three-dimensional medical image acquired by the acquisition unit 11 can originate from a storage unit (not shown) within the three-dimensional interface adjustment device 10 itself, or from an external input interface.

[0037] Under the control of a control signal, the selection unit 12 selects the interface between adjacent substructures as the interface to be adjusted in subsequent operations. For example, under the control of the control signal, the selection unit 12 selects the interface between segments S1 and S3 of the right lung as the interface to be adjusted. The control signal may be a signal input to the selection unit 12 from the outside via an input interface, or it may be a signal generated by a control unit (not shown) that is present in the three-dimensional interface adjustment device 10 itself. In addition, the control signal may be generated by the user's click operation on the substructure to be edited.

[0038] Under the control of a control signal, the designation unit 13 designates the target position to which the interface to be adjusted should be moved on other structures near the substructure. For example, under the control of the control signal, the designation unit 13 designates at least a portion of the intersegmental vein V1b as the target position to be moved. The control signal may be a signal input to the designation unit 13 from the outside via an input interface, or it may be a signal generated based on a control unit (not shown) inherent in the three-dimensional interface adjustment device 10 itself. Furthermore, the control signal may be generated, for example, by a user's click operation on the target position.

[0039] The calculation unit 14 models the interface to be adjusted to form a model surface, and calculates the movement direction and distance of the set of points on the model surface of the interface to be adjusted according to the specified target position. Modeling refers to constructing an object using points and lines, and the model surface is a surface obtained by modeling the interface to be adjusted using points and lines. The modeling includes meshing and polygonization. For ease of explanation, the following description uses meshing of the interface to be adjusted as an example. The calculation unit 14 first meshes the interface selected by the selection unit 12, i.e., the interface to be adjusted, and then generates a new interface in different ways according to the number of points at the target position specified by the designation unit 13. Specifically, whether there is one or multiple points corresponds to different interface generation methods. When there is one point, based on the specified target point, the point on the grid surface of the interface to be adjusted with the shortest distance to the target point is calculated. Then, using the distance, direction, and weight function of the points on the grid surface of the interface to be adjusted to the target point, the movement direction and distance of the point set on the grid surface of the interface to be adjusted are calculated. When there are multiple points, multiple points are first fitted to generate a smooth surface, and then the smooth surface is meshed to obtain a new grid surface. Finally, the grid surface of the interface to be adjusted is mapped to the new grid surface to obtain the movement direction and distance of the point set on the grid surface of the interface to be adjusted. The interface generation methods in different cases will be explained in detail later with reference to the attached diagram.

[0040] The update unit 15 updates the interface to be adjusted based on the movement direction and distance of the point set on the grid surface of the interface to be adjusted, and updates adjacent substructures using the changes in substructures caused by the interface update. The interface update and substructure update implemented by the update unit will be described in detail later with reference to the accompanying drawings.

[0041] The display control unit 16 causes a display unit (not shown) to display a three-dimensional medical image acquired by the acquisition unit 11, which includes the segmentation results of the substructure and the segmentation results of other structures near the substructure. Furthermore, the display control unit 16 adjusts the display angle of the substructure in the three-dimensional medical image to ensure that the interface to be adjusted on the substructure and the target position to which the interface to be adjusted are moved do not overlap. After the target position is specified, the display unit automatically displays the updated substructure. When observing a three-dimensional structure, if there is overlap between the structure to be adjusted and the target position to be specified, operation becomes difficult and results are not satisfactory. Therefore, it is desirable to operate when the interface to be adjusted and the target position to be specified can be clearly distinguished. Under the control of the display control unit 16, obstructive segmentation structures can be hidden, displaying only the substructure to be adjusted and the target position to be moved, and the viewing angle can be adjusted to clearly distinguish the molecular structure and the target position. Under the control of the display control unit 16, the user can easily specify the target position to which the substructure should be moved on other structures near the substructure (e.g., intersegmental veins).

[0042] The following reference Figures 2-8 This will illustrate the method for adjusting the three-dimensional interface in Implementation Method 1.

[0043] Figure 2 The document illustrates the general steps for adjusting a 3D interface.

[0044] First, in step S101, the acquisition unit 11 acquires a three-dimensional medical image containing the segmentation results of the substructures and the segmentation results of other structures near the substructures. In the following description, lung segments S1, S2, and S3 in the right lung are used as substructures, and the intersegmental vein V1b is used as other structures near the substructures. In practice, it is desirable that the intersegmental vein V1b is located between lung segments. Therefore, when the obtained intersegmental interface of the lung segment and the intersegmental vein V1b are not on the same plane, it is desirable to adjust the intersegmental interface to make them coplanar.

[0045] In step S102, the interface to be adjusted is selected on the substructure. In this embodiment, the inter-segment interface is selected as the interface to be adjusted.

[0046] Figure 3A The diagram illustrates lung segments S1, S2, and S3 obtained in step S101. Lung segment S3 has interfaces with lung segments S1 and S2, and lung segment S1 also has an interface with lung segment S2. By selecting the lung segment to be edited, such as lung segments S3 and S1, the interface between lung segments S3 and S1 can be automatically selected, and this interface can be used as the plane to be adjusted. Figure 3B As shown, after selecting the plane to be adjusted, it can be displayed differently from other planes. Alternatively, it can be displayed as follows: Figure 4 As shown, instead of using the entire interface between lung segments S3 and S1 as the plane to be adjusted, a portion of the interface is selected. Furthermore, the lung segment S3 shown in Figure 3 is the same lung segment as the lung segment S3 in subsequent views; the difference in morphology is solely due to the different display angles. The display angle in Figure 3 is used to easily illustrate the positional relationships between lung segments, while the display angles in subsequent views are used to easily illustrate the positional relationships between lung segments and intersegmental veins.

[0047] Next, in step S103, the target position to which the interface to be adjusted is to be moved is specified on other structures. In this embodiment, the target position refers to the location of the point specified by the user, specifically the coordinates of one or more points specified by the user on the intersegmental vein V1b. Since the user can specify one or more points, the target position in this embodiment can also be one or more. The interface adjustment method varies depending on the number of points specified by the user on the intersegmental vein V1b.

[0048] First, let's explain the case where there is only one specified point.

[0049] Figure 5 The diagram illustrates how a target point A is specified on the intersegmental vein V1b, and the shortest distance point B on the plane to be adjusted is determined based on that target point A.

[0050] In step S104, the plane to be adjusted is first meshed, that is, the plane to be adjusted is converted into a mesh surface composed of points, lines, and surfaces. Then, using the algorithm of exploring the shortest distance, the point B on the mesh surface of the plane to be adjusted that is closest to the target point A is found. Figure 5 The representation of the grid surface is omitted, but it can be understood that point B is a point on the grid surface of the plane to be adjusted. This shortest distance algorithm includes distance map calculation and scanning retrieval, but is not limited to these methods.

[0051] After finding point B, we further calculate the distance and direction of point B's movement relative to point A.

[0052] Next, while ensuring consistency in direction, the movement distance of other points on the plane to be adjusted is calculated based on the weight function of each point on the plane to be adjusted. The weight function can be a Gaussian function, but is not limited to it.

[0053] In step S105, based on the calculated movement distance and direction of each point on the plane to be adjusted, the interface to be adjusted is updated to a new interface that has passed the target position. Based on the interface changes of the interface to be adjusted and the new interface, a change body of a substructure can be obtained. By using this change body of a substructure, one substructure and another substructure can be updated.

[0054] Figure 6 The diagram shows a schematic of how the interface of segment S3 is adjusted upwards, resulting in an increase in the volume of segment S3 and a decrease in the volume of its adjacent segment S1.

[0055] like Figure 6 As shown, the interface of segment S3 is updated to the target position above, which increases the volume of segment S3 relative to the original structure, namely the S3 increase area (ΔS3) shown in the figure. In contrast, the volume of segment S1 corresponding to the S3 increase area (ΔS3) is reduced.

[0056] Therefore, based on the updated interface, the variant ΔS3 of segment S3 can be obtained. By utilizing this variant ΔS3, the substructures corresponding to segments S1 and S3 can be automatically updated.

[0057] Next, we will explain the case where multiple points are specified. This case includes both specifying multiple points and specifying a segment structure. The reason for this is that when specifying a segment structure, the segment structure can be regarded as a set of multiple points.

[0058] Figure 7A and Figure 7B These represent cases where multiple target points or a portion of a vein segment structure are specified on an intersegmental vein as the target location to be moved to.

[0059] Figure 8 This is a schematic diagram illustrating the process of updating a substructure based on multiple target points. Specifically, it is a schematic diagram of generating a target mesh surface based on multiple target points and updating the substructure based on the mapping relationship between the mesh surface of the interface to be adjusted and the target mesh surface.

[0060] In step S104', a smooth surface is first created using multiple specified three-dimensional coordinate points, and then the smooth surface is converted into a mesh surface composed of points, lines, and surfaces. The mesh surface corresponding to the smooth surface is referred to as the target mesh surface.

[0061] Next, in step S105', the interface to be adjusted is meshed. The mesh surface of the interface to be adjusted can also be called the original mesh surface. Then, the mapping relationship between the original mesh surface and the target mesh surface is calculated, and based on the above mapping relationship, the moving distance and direction of the point set on the mesh surface (original mesh surface) of the interface to be adjusted are obtained. The above mapping relationship can be calculated using algorithms such as the least squares method of three-dimensional point sets, but is not limited to this.

[0062] Next, in step S106', based on the movement direction and distance of the point set on the grid surface (original grid surface) of the interface to be adjusted, the original grid surface is updated onto the target grid surface, and the adjacent substructures are updated using the changes in the substructures generated by the interface update. This method of updating adjacent substructures using changes in the substructures is similar to... Figure 6 The method shown is the same, so detailed explanations are omitted here.

[0063] The above-described method for adjusting the 3D interface allows for accurate adjustment of the 3D interface with simple operation, and enables adjacent substructures to update automatically.

[0064] (Implementation Method 2)

[0065] The above describes the case where only the interface between one substructure and another substructure (e.g., the S1 and S3 segments of the right lung) is selected as the interface to be adjusted. However, it is also possible to select the interface between one substructure and two other substructures (e.g., the S3 segment of the right lung and the S2 and S1 segments) as the interface to be adjusted. In this case, after the interface to be adjusted is updated, it is necessary to determine the changes in each of the other two substructures. The difference between Embodiment 2 and Embodiment 1 lies only in how to calculate the changes in each of the other two substructures; the following explanation focuses on the differences, omitting explanations of the identical parts.

[0066] The following section uses the S3, S2, and S1 segments of the right lung as examples to illustrate how to calculate the change.

[0067] like Figure 9 As shown, the three lung segments have the following positional relationship: Segment S3 has interfaces that connect with segments S2 and S1 respectively, and segment S1 connects with S2 and connects with segment S3 respectively.

[0068] Suppose we select the interfaces in segment S3 that connect to segments S2 and S1 respectively as the interfaces to be adjusted, and the specified target position is located above the interfaces to be adjusted (i.e., in the direction that increases the volume of segment S3). After the interfaces to be adjusted are updated, the volume of segment S3 increases, while the volumes of segments S1 and S2 decrease. We can directly calculate the changes in segments S1 and S2 based on the overlap between the increased area of ​​segment S3 and the original segments S1 and S2, and then update the substructures of segments S1 and S2.

[0069] Next, assume that the interfaces in segment S3 that connect to segments S2 and S1 are selected as the interfaces to be adjusted, and the specified target position is located below the interfaces to be adjusted (that is, in the direction that reduces the volume of segment S3). At this point, after the interfaces to be adjusted are updated, the volume of segment S3 decreases, while the volumes of segments S1 and S2 increase. The decrease in the volume of segment S3 is the sum of the increases in the volume of segment S1 and the increases in the volume of segment S2.

[0070] To calculate the changes in segments S1 and S2, we can use the set of points at the intersection to determine the changes. For example, based on the attribution information of each point in the interface to be adjusted before the update (i.e., whether it belongs to segment S1 or S2), we can track the attribution information of each point in the updated interface, thereby determining the intersection of segments S1 and S2 in the updated interface. Based on the attribution information of the point set at the intersection, we can distinguish the change in volume of segment S3 into segments S1 and S2, and calculate the changes in the adjacent segments S1 and S2. By using the changes in segment S3 and the changes in segments S1 and S2, we can update segments S3, S1, and S2.

[0071] Furthermore, to calculate the changes in segments S1 and S2, the volume change of segment S3 can be differentiated into segments S1 and S2 by calculating the extensions of the joint surfaces of original segments S1 and S2. Thus, the changes in the interconnected segments S1 and S2 can be determined. By utilizing the changes in segment S3 and the changes in segments S1 and S2, segments S3, S1, and S2 can be updated.

[0072] The above examples illustrate two ways to update a substructure in addition to the other two substructures, but this is not the only approach. Any other method can be used as long as the same effect can be achieved.

[0073] Furthermore, the above update method can also be applied to situations where one substructure is connected to three other substructures. For example, in the right lung, segment S6 is connected to segments S8, S9, and S10. The interface between segment S6 and segments S8, S9, and S10 is selected as the interface to be adjusted. In this case, after the interface to be adjusted is updated, using the same change calculation method as described above, the changes of one substructure and the other three substructures can be calculated separately, thereby updating segments S6, S8, S9, and S10.

[0074] The same effect as in Embodiment 1 can also be achieved by adjusting the three-dimensional interface using the method described in Embodiment 2.

[0075] (Implementation Method 3)

[0076] Implementation 1 describes a method for modifying the interface between two adjacent substructures and further updating those two adjacent substructures. Implementation 3 describes the case where the substructure has only a single construction; a specific example is the case where the entire lung-divided structure is used as a single structure, but this is not the only possibility. According to Implementation 3, even if the substructure has only a single construction, its boundary surface can be updated and the substructure can be further updated.

[0077] It is known that when displaying lung segmentation results, the lung area should be shown in dark. However, sometimes when lesion areas are present, these lesion areas are highlighted. Ideally, the segmentation algorithm should include lesion areas within the lung segmentation result. Therefore, there is a need to adjust the segmentation results.

[0078] In the past, as a method of adjustment, the boundaries of the lung regions could be adjusted separately for multiple slices. However, this would result in too many slices, take too long, and involve a large workload.

[0079] To address the aforementioned issues, this embodiment proposes a method for generating a three-dimensional target mesh surface based on specified boundary points on multiple two-dimensional planes, and updating the substructure by updating the mesh surface of the interface to be adjusted to the target mesh surface.

[0080] The three-dimensional interface adjustment device 20 of embodiment 3 is as follows Figure 9 As shown, it has an acquisition unit 21, a selection unit 22, a designation unit 23, a calculation unit 24, an update unit 25, and a display control unit 26.

[0081] The three-dimensional interface adjustment device 20 of Embodiment 3 is similar to the three-dimensional interface adjustment device 10 of Embodiment 1 in that the acquisition unit acquires a three-dimensional medical image with segmentation results, the selection unit selects the interface to be adjusted on the three-dimensional interface of the substructure, the designation unit designates the target position to which the interface to be adjusted is to be moved, the calculation unit calculates the moving distance and direction of each point on the interface to be adjusted based on the target position, and the update unit updates the substructure.

[0082] The difference between the three-dimensional interface adjustment device 20 of Embodiment 3 and the three-dimensional interface adjustment device 10 of Embodiment 1 is that the acquisition unit acquires two-dimensional medical images from multiple perspectives in addition to acquiring three-dimensional medical images, the designation unit designates target points in the multiple two-dimensional medical images, the calculation unit calculates the target mesh surface based on the set of points in three dimensions of the multiple designated target points, the updating unit updates only the substructure of this single structure, and the display control unit causes the display unit to display the acquired three-dimensional medical images and multiple two-dimensional medical images, and automatically displays the updated substructure after the target position is designated.

[0083] Figure 11 This is a flowchart illustrating the three-dimensional interface adjustment method of implementation method 3. Figure 12 This is a schematic diagram illustrating the three-dimensional interface adjustment method of Embodiment 3. Below, refer to... Figure 11 and Figure 12 To illustrate the three-dimensional interface adjustment method of implementation method 3.

[0084] In step S201, two-dimensional medical images with multiple perspectives containing lung segmentation results and three-dimensional medical images containing lung segmentation results are obtained. Figure 12 The top section displays two 2D lung segmentation images in the axial plane and one 2D lung segmentation image in the sagittal plane. The two axial plane images are obtained from different axial planes when viewed from the axial plane. The bottom left corner displays the 3D lung segmentation image corresponding to the three 2D lung segmentation images mentioned above.

[0085] Next, in step S202, at least a portion of the lung interface is selected from the three-dimensional lung segmentation image as the interface to be adjusted. (Illustrations omitted here.)

[0086] Next, in step S203, the desired lung boundary adjustment positions are specified in the two-dimensional lung segmentation images from multiple perspectives. As mentioned earlier, lesion areas of the lung may be displayed as light-colored areas in the lung segmentation images, resulting in a reduction in the lung segmentation boundary (the boundary of the dark-colored portion) obtained by the segmentation algorithm compared to the actual lung area; the lesion area is not included in the overall lung. To complete the lung segmentation results, in step S203, multiple points are specified as target positions by clicking on the desired lung boundary adjustment positions in each two-dimensional lung segmentation image.

[0087] Next, in step S204, the calculation unit meshes the interface to be adjusted selected in step S202 to form a mesh surface composed of points, lines, and surfaces. This mesh surface formed by meshing the interface to be adjusted is referred to as the original mesh surface. Next, based on the multiple points specified in step S203, the calculation unit generates a three-dimensional set of points and obtains a smooth surface passing through the aforementioned set of points through fitting. This smooth surface is then meshed to form another mesh surface composed of points, lines, and surfaces. This mesh surface of the smooth surface generated through fitting is referred to as the target mesh surface. The original mesh surface and the target mesh surface are mapped to calculate the movement direction and distance of the point set on the original mesh surface.

[0088] Next, in step S205, based on the calculated movement direction and distance of the point set of the interface to be adjusted, the interface to be adjusted is updated on the target mesh surface, and the three-dimensional lung segmentation structure is updated.

[0089] According to implementation method 3, the three-dimensional interface can also be accurately adjusted with a simple operation method, and the substructures in the segmentation results can be automatically updated.

[0090] (Other implementation methods)

[0091] The above embodiments describe the three-dimensional interface adjustment device and the three-dimensional interface adjustment method. However, it can be understood that as long as the various functions of the three-dimensional interface adjustment method can be realized, the above functions can also be realized by hardware or by a combination of hardware and software.

[0092] For example, when the 3D interface adjustment method is implemented through a processing circuit, i.e., a processor, the various processing functions of the processor are stored in a storage circuit as programs executable by a computer. The processor reads and executes each program from the storage circuit to implement the function corresponding to each program. Alternatively, instead of storing the program in the storage circuit, the program can be directly loaded into the processor's circuitry. In this case, the processor reads and executes the program loaded into the circuitry to implement the function. Furthermore, the processor in the above-described embodiments is not limited to being a single circuit; multiple independent circuits can be combined to form a single processor to implement its function.

[0093] The term "processor" as used in the above description may mean, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or an Application Specific Integrated Circuit (ASIC), a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA)).

[0094] The program executed by the processor can be provided pre-loaded into ROM (Read Only Memory) or storage circuitry. Alternatively, the program can be provided as a file recorded on a computer-readable storage medium such as CD (Compact Disc)-ROM, FD (Flexible Disk), CD-R (Recordable), or DVD (Digital Versatile Disc).

[0095] Alternatively, the program can be stored on a computer connected to a network such as the Internet and provided or distributed by downloading it over the network.

[0096] While several embodiments of the invention have been described, these embodiments are provided by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other ways, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the scope of the invention described in the claims and its equivalents.

Claims

1. A three-dimensional interface adjustment device for adjusting an interface to a specified target position, comprising: The acquisition section acquires three-dimensional medical images containing segmentation results of substructures and segmentation results of other structures near the substructures; In the selection section, select the interface to be adjusted located on the substructure; The designated section specifies the target position to which the interface to be adjusted should be moved on the other structures. The computing unit models the interface to be adjusted to form a model surface, and calculates the movement direction and distance of the set of points on the model surface of the interface to be adjusted based on the target position; and The updating unit updates the interface to be adjusted based on the movement direction and distance of the point set on the model surface of the interface to be adjusted, and updates the adjacent substructures by utilizing the changes in the substructures caused by the interface update.

2. The three-dimensional interface adjustment device as described in claim 1, The model surface is obtained by modeling the interface to be adjusted using points and lines.

3. The three-dimensional interface adjustment device as described in claim 1, The interface to be adjusted is either the entirety of the interface between adjacent substructures or a part of the interface.

4. The three-dimensional interface adjustment device as described in claim 2, The target location is a target point specified in the segmentation results of the other structures. The calculation unit calculates the point on the model surface of the interface to be adjusted that is closest to the target point, and calculates the distance and direction from that point to the target point. Based on the distance and direction and the weight function of each point on the model surface, the unit calculates the movement distance and direction of other points on the model surface.

5. The three-dimensional interface adjustment device as described in claim 4, The weighting function is a Gaussian function.

6. The three-dimensional interface adjustment device as described in claim 2, The target location is one of multiple target points or a portion of a region specified in the segmentation results of the other structures. The calculation unit generates a smooth surface based on multiple specified target points or a portion of the region. The model surface after modeling the smooth surface is used as the target model surface. The unit calculates the mapping relationship between the model surface of the interface to be adjusted and the target model surface, thereby obtaining the movement direction and distance of the set of points on the model surface of the interface to be adjusted.

7. The three-dimensional interface adjustment device as described in claim 1, The substructure is one of the lung segment, liver segment, and brain region, and other structures near the substructure are one of the trachea, arteries, and veins.

8. The three-dimensional interface adjustment device as described in claim 3, When the selected interface to be adjusted is a substructure between another substructure. The updating unit updates the interface to obtain a variant of the substructure, and updates the substructure and the other substructure by utilizing the variant of the substructure.

9. The three-dimensional interface adjustment device as described in claim 3, When the selected interface to be adjusted is the interface between one substructure and two other substructures, the other two substructures are connected to each other, and the target position is located in the direction that reduces the volume of the one substructure, The updating unit updates the interface to obtain the variation of the substructure, and based on the information of the substructure to which each point on the interface to be adjusted belongs before the update and the information of the substructure to which each point on the updated interface belongs, it calculates the variation of each of the other two connected substructures. By using the variation of the substructure and the variation of each of the other two substructures, the substructure and the other two substructures are updated.

10. The three-dimensional interface adjustment device as described in claim 3, When the selected interface to be adjusted is the interface between one substructure and two other substructures, the other two substructures are connected to each other, and the target position is located in the direction that reduces the volume of the one substructure, The calculation unit calculates the extension surface of the joint surface based on the orientation of the joint surface between the other two interconnected substructures. The updating unit updates the interface to obtain the variation of the substructure. Based on the variation of the substructure and the extension surface, it calculates the variation of each of the other two substructures that are connected to each other. By using the variation of the substructure and the variation of each of the other two substructures, it updates the substructure and the other two substructures.

11. The three-dimensional interface adjustment device as described in claim 1, It also includes a display control unit, which displays a three-dimensional medical image containing the segmentation results of the substructures and the segmentation results of other structures near the substructures. In the three-dimensional medical image, the display angle of the substructure is adjusted in such a way that the interface to be adjusted on the substructure and the target position to which the interface to be adjusted is to be moved do not overlap. After the target position is specified, the display unit automatically displays the updated substructure.

12. A three-dimensional interface adjustment device for adjusting an interface to be adjusted to a specified target position, comprising: The acquisition section acquires two-dimensional medical images from multiple perspectives containing segmentation results of substructures and three-dimensional medical images containing segmentation results of substructures. The selection unit selects the interface to be adjusted located on the substructure in the three-dimensional medical image; The designated part specifies the target position to be moved to in the two-dimensional medical images from the multiple perspectives; The computing unit models the interface to be adjusted to form a model surface, and calculates the movement direction and distance of the point set on the model surface of the interface to be adjusted based on the multiple target positions in three dimensions. as well as The updating unit updates the interface to be adjusted and its substructures based on the movement direction and distance of the point set on the model surface of the interface to be adjusted.

13. The three-dimensional interface adjustment device as described in claim 12, The model surface is obtained by modeling the interface to be adjusted using points and lines.

14. The three-dimensional interface adjustment device as described in claim 12, wherein, The interface to be adjusted is either the entirety of the outer periphery of a single-structured three-dimensional substructure or a portion thereof.

15. The three-dimensional interface adjustment device as described in claim 13, wherein, The calculation unit transforms the target positions in the two-dimensional medical images from multiple perspectives specified by the designating unit into a set of points representing the target positions in three dimensions. Based on the set of points representing the target positions in three dimensions, a smooth surface is generated. The model surface after modeling the smooth surface is used as the target model surface. The mapping relationship between the model surface of the interface to be adjusted and the target model surface is calculated, thereby obtaining the movement direction and distance of the set of points on the model surface of the interface to be adjusted.

16. The three-dimensional interface adjustment device as described in claim 12, wherein, It also includes a display control unit, which enables the display unit to display two-dimensional medical images with multiple perspectives containing segmentation results of substructures and three-dimensional medical images containing segmentation results of substructures. Once the target location is specified, the display unit automatically displays the updated substructure.

17. A three-dimensional interface adjustment method for adjusting an interface to a specified target position, comprising the following steps: The acquisition step involves obtaining a three-dimensional medical image containing the segmentation results of the substructures and the segmentation results of other structures near the substructures; Select the step, and select the interface to be adjusted located on the substructure; The specified steps specify the target position to which the interface to be adjusted should be moved on the other structures; The calculation steps involve modeling the interface to be adjusted to form a model surface, and calculating the movement direction and distance of the set of points on the model surface of the interface to be adjusted based on the target position. as well as The update step involves updating the interface to be adjusted based on the movement direction and distance of the point set on the model surface of the interface to be adjusted, and updating adjacent substructures using the changes in substructures caused by the interface update.

18. A three-dimensional interface adjustment method for adjusting an interface to a specified target position, comprising the following steps: The acquisition steps involve obtaining two-dimensional medical images from multiple perspectives containing segmentation results of substructures and three-dimensional medical images containing segmentation results of substructures. In the selection step, select the interface to be adjusted located on the substructure in the three-dimensional medical image; The specified steps involve specifying the target location to be moved to in the two-dimensional medical images from the multiple perspectives; The calculation steps involve modeling the interface to be adjusted to form a model surface, and calculating the movement direction and distance of the point set on the model surface of the interface to be adjusted based on the set of points of multiple target positions in three dimensions. as well as The update step involves updating the interface to be adjusted and its substructures based on the movement direction and distance of the point set on the model surface of the interface to be adjusted.

19. A computer program product comprising a program for adjusting an interface to a specified target position, wherein, The program causes the computer to perform the following steps: The acquisition step involves obtaining a three-dimensional medical image containing the segmentation results of the substructures and the segmentation results of other structures near the substructures; Select the step, and select the interface to be adjusted located on the substructure; The specified steps specify the target position to which the interface to be adjusted should be moved on the other structures; The calculation steps involve modeling the interface to be adjusted to form a model surface, and calculating the movement direction and distance of the set of points on the model surface of the interface to be adjusted based on the target position. as well as The update step involves updating the interface to be adjusted based on the movement direction and distance of the point set on the model surface of the interface to be adjusted, and updating adjacent substructures using the changes in substructures caused by the interface update.

20. A computer program product comprising a program for adjusting an interface to a specified target position, wherein, The program causes the computer to perform the following steps: The acquisition steps involve obtaining two-dimensional medical images from multiple perspectives containing segmentation results of substructures and three-dimensional medical images containing segmentation results of substructures. In the selection step, select the interface to be adjusted located on the substructure in the three-dimensional medical image; The specified steps involve specifying the target location to be moved to in the two-dimensional medical images from the multiple perspectives; The calculation steps involve modeling the interface to be adjusted to form a model surface, and calculating the movement direction and distance of the point set on the model surface of the interface to be adjusted based on the set of points of multiple target positions in three dimensions. as well as The update step involves updating the interface to be adjusted and its substructures based on the movement direction and distance of the point set on the model surface of the interface to be adjusted.

21. A storage medium storing a program that causes a computer to perform the following steps: The acquisition step involves obtaining a three-dimensional medical image containing the segmentation results of the substructures and the segmentation results of other structures near the substructures; Select the step, and select the interface to be adjusted located on the substructure; The specified steps specify the target position to which the interface to be adjusted should be moved on the other structures; The calculation steps involve modeling the interface to be adjusted to form a model surface, and calculating the movement direction and distance of the set of points on the model surface of the interface to be adjusted based on the target position. as well as The update step involves updating the interface to be adjusted based on the movement direction and distance of the point set on the model surface of the interface to be adjusted, and updating adjacent substructures using the changes in substructures caused by the interface update.

22. A storage medium storing a program that causes a computer to perform the following steps: The acquisition steps involve obtaining two-dimensional medical images from multiple perspectives containing segmentation results of substructures and three-dimensional medical images containing segmentation results of substructures. In the selection step, select the interface to be adjusted located on the substructure in the three-dimensional medical image; The specified steps involve specifying the target location to be moved to in the two-dimensional medical images from the multiple perspectives; The calculation steps involve modeling the interface to be adjusted to form a model surface, and calculating the movement direction and distance of the point set on the model surface of the interface to be adjusted based on the set of points of multiple target positions in three dimensions. as well as The update step involves updating the interface to be adjusted and its substructures based on the movement direction and distance of the point set on the model surface of the interface to be adjusted.

Images