Efficient rib surface visualization by automation

Abstract

The rib surface visualization based on the axial view is extended. Data representing the three-dimensional diagnostic image is received. The ribs are segmented, the rib centerlines are detected and labeled. The fitting surfaces corresponding to the rib centerlines are calculated separately for each rib, and the out-of-plane rib portions are determined. The visualization of the ribs is generated individually or as rib pairs. The extended visualization is generated for the out-of-plane rib portions and can be executed via a fusion plan or a surface plan. The fusion plan includes projecting the out-of-plane rib portions onto the visualization so as to be fused with the in-plane rib portions. The surface plan includes fitting a polynomial surface such that all points within the point set are close to the surface. By this approach, individual ribs (pairs) can be easily inspected with a single axial view.

Description

Technical Field

[0001] The present invention relates to the field of medical images, and more particularly to a method and apparatus for rib surface visualization by automatic expansion.

Background Art

[0002] Reading image scans, more specifically scans in the trauma or emergency department, is a time-critical task that requires high attention to avoid missing important findings. In many cases, these settings are based on whole-body scans that result in large amounts of image data being generated, and the large amounts of generated image data must be thoroughly examined. The ribs are a particularly important structure to be evaluated, for example, with respect to trauma. These are repetitive structures that take a significant amount of time to examine (usually 12 rib pairs and 24 vertebrae).

[0003] Computed tomography (CT) scans have become the modality of choice for assessing a patient's overall condition for applications such as trauma or emergency settings. Healthcare providers, such as radiologists, often rely on the inner image data in the form of CT scans for diagnostic purposes, such as detecting rib fractures. During reading, each of the 24 ribs needs to be individually followed while scrolling through the image slices.

[0004] Looking at three-dimensional (3D) CT scans of the chest or the whole body slice by slice is often a time-consuming process, especially when the target anatomical structure spans multiple slices (e.g., tracing 24 individual ribs). As a result, there is a possibility of missing rib abnormalities.

[0005] The evaluation of the ribs requires a significant amount of reading time since the ribs are generally traced one by one through the volume of image data. Furthermore, medical conditions such as fractures (especially buckling) are very subtle and can easily be missed.

[0006] Several approaches have been proposed to simplify the visualization and evaluation of a patient's anatomical and critical structures, and in particular, approaches targeting the visualization of the rib cage have been proposed. While offering several advantages, current approaches come with distinct fundamental limitations.

[0007] One known visualization scheme is the "file view" or "fishbone view", which is based on segmenting the ribs (e.g., using a deep convolutional neural network), followed by using a centerline extractor, and then labeling the rib pairs within the field of view. Each rib is sampled along its trace, normalized, and straightened in a way that enables visualization of each rib and all ribs in a straightened format (curved planar format). With this view, medical experts such as radiologists can accurately examine the rib centerlines in a normalized view (reformatted view) where all ribs are straightened and placed at unique positions on the examination canvas.

[0008] However, this type of view has several drawbacks. One drawback in this type of view is that the nature of processing each rib independently leads to discontinuities between the ribs and can lead to image artifacts from adjacent ribs that appear in the rib shape.

[0009] Another visualization scheme is the visceral cavity view. In this view, a segmentation algorithm (e.g., using a model-based approach) is applied to the segments inside the rib cage with respect to the deformation of a cylindrical manifold. Once segmentation is performed, the manifold can be unwrapped and a maximum intensity projection (MIP) close to the surface proximity can be calculated. With this view, the user can examine the entire rib cage with respect to a continuous visualization on the examination canvas.

[0010] One of the drawbacks of this type of view is that the relative rib lengths are not maintained. In other words, the property of unwrapping the cylinder manifold does not allow for visualization of the correct rib lengths. For example, the first rib may appear too long relative to the other ribs. Additionally, the nature of MIP visualization may not allow for detection of microfractures. For example, MIP visualization can render small rib fractures invisible and undetectable in the generated view. Another drawback is that this view adds significant unrealistic distortion (wavy ribs) that limits the clinical reliability in the generated visceral cavity view.

[0011] Therefore, there is a need for an efficient rib surface visualization that overcomes the drawbacks associated with these views. SUMMARY OF THE INVENTION PROBLEMS TO BE SOLVED BY THE INVENTION

[0012] An object of the present invention is to provide a technique for an automatically expanding view that overcomes the drawbacks of existing visualization schemes. This technique can be applied to several imaging systems, including CT, CT arms, single photon emission computed tomography CT (SPECT-CT), magnetic resonance CT (MR-CT), positron emission tomography CT (PET-CT), and magnetic resonance imaging (MRI) systems. MEANS FOR SOLVING THE PROBLEMS

[0013] According to a first aspect of the present invention, a method for expanding visualization of a rib surface is provided. The method includes receiving data representing a three-dimensional diagnostic image, wherein the three-dimensional diagnostic image includes one or more ribs of a subject, segmenting the one or more ribs according to the received data representing the three-dimensional diagnostic image, detecting and labeling a rib centerline from the rib segmentation, for each rib, separately calculating a fitting surface corresponding to the rib centerline determining an out-of-plane rib portion for each rib; generating a visualization of the rib(s), either individually or as rib portions; generating an extended visualization for the out-of-plane rib portion and.

[0014] In a second aspect of the present invention, an imaging device is provided. The imaging device comprises a memory configured to store computer-executable instructions; at least one processor, which executes the computer-executable instructions to cause the imaging device to receive data representing a three-dimensional diagnostic image, the three-dimensional diagnostic image including one or more ribs of a subject; segmenting the one or more ribs according to the received data representing the three-dimensional diagnostic image; detecting and labeling a rib centerline from the rib segmentation; separately calculating a fitting surface corresponding to the rib centerline for each rib; determining an out-of-plane rib portion for each rib; generating a visualization of the rib(s), either individually or as rib portions; generating an extended visualization for the out-of-plane rib portion and at least one processor configured to cause the above to be executed and.

[0015] In a third aspect of the present invention, a non-transitory computer-readable medium storing instructions for causing a processing circuit to execute a process is provided. The process comprises receiving data representing a three-dimensional diagnostic image, the three-dimensional diagnostic image including one or more ribs of a subject; segmenting the one or more ribs according to the received data representing the three-dimensional diagnostic image; Detecting and labeling the rib centerline from the rib segmentation step; Calculating the fitting surface corresponding to the rib centerline for each rib separately; Determining the off-plane rib portion for each rib; Generating a visualization of the rib either individually or as a rib portion; Generating an extended visualization for the off-plane rib portion and having.

[0016] In a preferred embodiment, the step of generating an extended visualization for the off-plane rib portion includes calculating the distance to the fitting surface for points on each rib centerline, and the rib portion outside the fitting surface is projected onto the visualization of the rib so as to be fused with the in-plane rib portion. The off-plane rib portion may be highlighted to identify rib portions that are not part of the original visualization of the rib. One aspect of the present invention provides a step of switching between the original visualization of the rib and the extended visualization of the rib, and highlighting the radius of influence in the extended visualization of the rib.

[0017] In a preferred embodiment, the step of generating an extended visualization for the off-plane rib portion includes establishing a set of points including all points on the rib centerline and cross points between the fitting surface and the data boundary, and fitting a polynomial surface, so that all points within the set of points will be close to the surface. In one aspect of the present invention, the rendering direction is set to be the same as the fitting surface, the surface is rendered, and the rendering result is projected onto the fitting surface. The rendering mode can be selected from the group consisting of multi-planar reformation (MPR), maximum intensity projection (MIP), minimum intensity projection (MinIP), average intensity projection (AIP), and volume rendering (VR).

[0018] In one aspect of the present invention, the steps of segmenting the rib, detecting the rib centerline, and labeling are performed via machine learning or deep learning techniques.

Brief Description of the Drawings

[0019]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

DETAILED DESCRIPTION OF THE INVENTION

[0020] FIG. 1 shows an imaging system 100 such as a computed tomography (CT) imaging system. The imaging system 100 generally includes a stationary gantry 102 and a rotating gantry 104. The rotating gantry 104 is rotatably supported by the fixed gantry 102 and rotates around the inspection area 106 around the vertical axis or z-axis with respect to the fixed gantry 102.

[0021] A patient support 112, such as an examination table, supports an object or examination object, such as a human patient, in the examination region 106. The support 112 is configured to move the object or examination object for loading, scanning, and unloading. A radiation source 108, such as an x-ray tube, is rotatably supported by a rotating gantry 104. The radiation source 108 rotates with the rotating gantry 104 and emits radiation that traverses the examination region 106.

[0022] A radiation sensitive detector array 110 defines a range of an angular arc on the opposite side of the radiation source 108 across the examination region 106. The detector array 110 includes one or more rows of detectors that extend along the z-axis direction, detect radiation traversing the examination region 106, and generate projection data indicative thereof.

[0023] A general-purpose computing system or computer functions as an operator console 114 and includes an input device 116, such as a mouse, keyboard, and / or the like, and an output device 120, such as a display monitor, film m, etc. The console 114 enables an operator to control the operation of the system 100. This includes control of the image processing device 118. The imaging processing device 118 receives data representing a three-dimensional (3D) diagnostic image generated by the imaging system 100, separately calculates a fitting surface corresponding to the center line of each rib, generates a visualization of the rib, and can generate an extended visualization when the rib portion is outside the determined fitting surface. A user can switch between the original visualization of a given rib and the extended visualization of the rib.

[0024] It should be understood that the processing of the image processing device 118 can be implemented through a microprocessor that executes computer-readable instructions encoded or embedded on a computer-readable storage medium, such as physical memory and other non-transitory media. Additionally or alternatively, the microprocessor can execute computer-readable instructions carried by a carrier wave, signal, and other transitory media.

[0025] Figure 2 shows various views of the rib cage by the visceral cavity view 202 and the "fillet view" or "fishbone view" 204. These views enable medical professionals such as radiologists to accurately examine the rib centerlines in an unfolded view (reformed view) where all the ribs are straightened and placed at unique positions on the examination canvas. It can be seen that these views enable an efficient evaluation of the overall rib cage, but a new view that needs to be well-known to radiologists, shows significant distortion, and can even hide medical conditions such as fractures, is introduced.

[0026] The proposed new visualization plan is based on axial view plane adjustment. The axial plane is the field of view that most radiologists use to mainly interpret relevant imaging examinations. Usually, most of the ribs are in a single plane. To utilize this characteristic, a reformatted view corresponding to the optimal view plane is shown to the user. The inclined axial view can be easily examined by radiologists. However, parts of the ribs may not be located in the plane and may thus be overlooked.

[0027] Therefore, the new automatic expansion plan automatically detects out-of-plane parts and generates an expanded view that enables the examination of out-of-plane parts. Two different implementations of the expanded view are provided, with the affected parts highlighted, enabling the user to identify rib parts that are not part of the original visualization of the ribs.

[0028] One expanded view is provided by performing a fusion of parts that do not exist in the plane. Another expansion is provided by generating a curved surface close to the inclined axial plane and the points of the rib centerline.

[0029] For each expanded view, segmentation of the ribs is performed, the rib centerline is extracted, and a fitting surface corresponding to the rib centerline is calculated separately for each fitting surface.

[0030] Figure 3 shows the points of the centerlines of the ribs on a single plane. From the segmentation, several different representations can be derived. Rib centerline visualization is a common representation derived from the segmentation. The segmentation and / or derived representations can be labeled according to anatomical labels (i.e., L1 to 12, R1 to 12). For segmentation, labeling, and centerline extraction, several well-established methods exist and have been successful in demonstrating sufficient robustness. An example of such a technique is to apply a fully convolutional neural network (FCNN) to generate a probability map for detecting the first rib pair, the 12th rib pair, and the set of all intermediate ribs. In a second stage, a centerline extraction algorithm is applied to this multi-label probability map. Finally, the separate detection of the first and 12th ribs enables the derivation of individual rib labels by sorting and counting the detected centerlines. The rib centerline can be obtained using an interpolation scheme such as a spline.

[0031] Typically, most of the points of the centerlines of the ribs are approximately within a single plane. The plane can be determined through a covariance analysis of the coordinates of the center points where the eigenvector corresponding to the minimum eigenvalue of the covariance matrix corresponds to the plane normal vector. Since ribs, especially those close to the sternum at the vertebral connection, often exhibit significant flexion, a weighted covariance matrix focused on the central part of the rib is beneficial to ensure that the calculated plane passes through the majority of the points.

[0032] The different weightings of the centerline points for the plane fitting calculation are shown at 302. The lighter parts have lower weights to ensure a focus on the majority of the rib. The plane passing through the rib, shown as a mesh, is shown at 304.

[0033] For the fusion plan, using the calculated plane fit, the out-of-plane parts can be automatically determined for each centerline by calculating the distance d to the found plane. All points greater than a specified threshold T are considered out-of-plane. Based on the determined distances, two main scenarios can be defined.

[0034] The first scenario is shown in Figure 4. In the first scenario, all points are near or very near the aircraft. By visualizing the axial view of the markings corresponding to the determined surface fit, the ribs can be well evaluated. Figure 4 shows the visualization of rib pairs 402 and 404 based on the inclined axial view corresponding to the mating surface.

[0035] The second scenario is shown in Figure 5. The second scenario is the case where a point is out of plane. In such a case, certain portions of the ribs cannot be evaluated in the axial direction of the markings derived from the surface fit. In these cases, the user starts navigation to display the out-of-plane portion. However, the original in-plane portion disappears. This increases the complexity and reading time of the image slices. Therefore, in such scenarios where not all portions are in-plane, an "extended view" is implemented. The in-plane portions are displayed as described above, while the out-of-plane portions are projected onto the in-plane. More precisely, since the ribs are bright structures, maximum intensity projection can be performed locally. By smoothly transitioning this projection, sharp edges in the image are prevented.

[0036] The outer rib portion of the multi-planar reformation (MPR) is automatically determined at 502. A larger variable region along the ribs is determined at 504, and the depth change of the MPR plane 506 is smoothly varied to ensure a smoothly obtained image. The fusion is limited only to the regions where the ribs are locally out of plane. Therefore, most of the image is not affected. The fused portions may be highlighted, for example, via a color overlay or a color circle, to identify for the user one or more regions that are not parts of the original image.

[0037] Figure 6 shows an enlarged view of the out-of-plane rib portion and the out-of-plane portion. The MPR where the rib portion is slightly out-of-plane and completely out-of-plane is indicated by 602. The overlooked fracture is shown in the enlarged view of 604. The highlighting can be extended over the entire area that may be considered to indicate the radius of influence. The index of the variable area is shown at 606. This may be provided as warning information to the clinician or radiologist, and the highlighting may be toggled on and off by the clinician.

[0038] Another example of the MPR view and the extended view is shown in Figure 7. The MPR view is indicated by 702, and the extended view is indicated by 704. The area at 706 of the extended view 704 shows the out-of-plane portion of the rib that is missing in the MPR view 702.

[0039] In the case of surface planning, understanding that there are an infinite number of surfaces that can cross the points of the center line, when a set of surface equations is given, the closest surface may not be close to the conforming surface (tilted axis surface). To ensure that the surface is close to its plane, it is necessary to extend the point set using points located on the plane. This limits any potential irregularities of the surface so that the image content becomes more natural.

[0040] A polynomial is a way to construct and solve a surface, and the surface portion could be easily controlled through the higher-order part. The accuracy of the surface can be improved by adding the higher-order part, and the bending is highly restricted by adding a penalty to the higher-order part. The degree of the polynomial can be determined by limiting the average distance between the point set and the calculated surface.

[0041] Figure 8 shows surface fitting. The conforming surface where the point part is locally deviated from the plane is indicated by 802. The enlarged view by surface fitting is shown at 804. The fitted polynomial surface is close to the original conforming surface, and the distance between the point and the surface is reduced.

[0042] FIG. 9 illustrates an enlarged view with surface adaptation. At 902, an MPR is shown where a rib portion is locally out-of-plane. The enlarged view is shown at 904. The extended view shows the out-of-plane portion in region 906 that is missing in MPR 902.

[0043] Multiple different rendering modes, such as maximum intensity projection (MIP), minimum intensity projection (MinIP), average intensity projection (AIP), and volume rendering (VR), can be applied to verify that the complete images share the same thickness.

[0044] For both the fusion approach and the surface adaptation of the curved surface approach, it is calculated separately for each rib in order to efficiently navigate the entire rib cage. As shown in FIG. 4, the left and right ribs may be arranged side by side for display. The overview image can be shown for all 12 rib pairs within one stacked overview. Interaction can be enabled to click from one rib pair to the next. For each rib, individual navigation is possible (preferably, axis-parallel and limited to the rib).

[0045] FIG. 10 is a flowchart representing a method for expanding rib surface visualization according to some embodiments. At 1002, data representing a three-dimensional (3D) diagnostic image including ribs of a subject, such as a human patient, is received.

[0046] At 1004, the ribs are segmented according to the received data representing the 3D diagnostic image. The segmentation of the ribs can be performed by applying an automatic machine learning approach such as manual, semi-automatic, or fully convolutional neural network (FCNN) to generate a probability map for detecting a set of the first rib pair, the twelfth rib pair, and all intermediate ribs. Next, a centerline extraction algorithm is applied to this multi-label probability map. The rib centerline can be obtained using cubic spline interpolation. Finally, the separate detection of the first and twelfth ribs enables the derivation of individual rib labels by simply sorting and counting the detected centerlines.

[0047] The center line of the rib is detected and labeled from rib segmentation at 1006. The rib segmentation may be performed via machine learning or deep learning techniques.

[0048] At 1008, a fitting surface corresponding to the rib center line is calculated separately for each rib. The out-of-plane rib portion is determined for each rib at 1010. At 1012, a visualization of the rib is generated. The visualization at 1012 can be generated individually for each rib or as a rib pair. At 1014, an extended visualization for the out-of-plane rib portion is generated.

[0049] FIG. 11 is a flowchart depicting a method for extending rib surface visualization via a fusion plan according to some embodiments. At 1102, a distance is calculated to fit a plane to the points of each rib center line. That is, for each center line, a distance d to the determined plane is calculated. If all points are in-plane or close to the plane, the rib can be evaluated by visualizing an inclination axis view corresponding to the determined plane. This is shown at 402 in FIG. 4.

[0050] If a particular point is outside the plane, the associated out-of-plane portion of the rib cannot be evaluated with the inclination axis derived from the fitting surface. Using the calculated fitting surface, the out-of-plane rib portion can be automatically determined. The automatically determined out-of-plane rib portion is projected onto the visualization of the rib and is seen as being fused with the in-plane portion of the rib at 1104, as shown at 402 in FIG. 4. The fusion is limited to the region where the rib is locally out-of-plane so that most of the image region including the region where the rib is in-plane is not affected.

[0051] At 1106, the out-of-plane rib portion can be highlighted in the extended view to identify rib portions that are not part of the original visualization. At 1108, a radius of influence can be highlighted in the extended visualization of the rib. The user can switch between the original visualization of the rib and the extended visualization of the rib at 1110.

[0052] Figure 12 is a flowchart showing a method for extending rib surface visualization via a surface plan according to some embodiments. At 1202, a set of points is established that includes all the centerline points of the ribs and the cross points between the fitting surface and the data boundary.

[0053] Given a family of surface types, considering that there are an infinite number of surfaces that can cross the centerline points, the closest surface may not be close to the fitting surface. To ensure that the surface is close to a plane, the set of points may be extended with points located on the plane. Polynomials are an effective way to construct surfaces. At 1204, a polynomial surface is fitted such that all points within the set of points are close to the surface.

[0054] The degree of the polynomial is determined by limiting the average distance between the set of points and the calculated surface. The accuracy of the surface can be improved by adding higher-order terms. At 1206, the rendering direction is set to be the same as that of the fitting surface. FIG. 9 shows MPR rendering and an extended view according to the surface plan. Different rendering modes including MIP, MinIP, AIP, and VR may also be applied. At 1208, the rendered result is projected onto the fitting surface.

[0055] Although the present invention has been illustrated and described in detail in the drawings and the foregoing description, such illustration and description should be considered illustrative or exemplary and not restrictive, and the present invention is not limited to the disclosed embodiments.

[0056] Other variations to the disclosed embodiments can be understood and achieved by those skilled in the art when implementing the claimed invention from a study of the drawings, the disclosure, and the appended claims.

[0057] In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite articles "a" or "an" do not exclude a plurality.

[0058] A single processor, apparatus, or other unit can fulfill the functions of several items listed in the claims. The mere fact that specific means are described in mutually different dependent claims does not indicate that a combination of these means cannot be used advantageously.

[0059] Operations such as acquisition, determination, acquisition, output, provision, storage, or storage, calculation, simulation, reception, warning, and stop can be implemented as program code means of a computer program and / or as dedicated hardware.

[0060] The computer program can be stored and / or distributed on a suitable medium, such as an optical storage medium or a solid-state medium, supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless electrical communication systems.

Claims

1. A method for supplementing a rib cage surface image, wherein the method is A step of receiving data representing a three-dimensional diagnostic image, wherein the three-dimensional diagnostic image includes one or more ribs of a subject. The steps include: segmenting one or more ribs based on the received data representing the three-dimensional diagnostic image via machine learning or deep learning techniques; A step of detecting and labeling the rib centerline from the segmented rib representation, wherein the rib centerline represents a line in the axial direction of the rib that passes through the center of the cross-section of the rib. A step of separately calculating a fitting surface corresponding to the rib centerline for each rib, wherein each fitting surface is separately calculated as a single plane that includes most of the multiple points on the rib centerline of each rib; Based on the received data representing the three-dimensional diagnostic image, the step of determining three-dimensional data for each rib that shows the portion of the rib located outside the plane of the fitting surface, The steps include generating the aforementioned rib cage surface display image, The steps include generating a rib surface display image complemented by the rib portion located outside the plane of the fitting surface by projecting the rib portion located outside the plane of the fitting surface onto the rib surface display image so that it appears as if it is fused with the rib portion located within the plane of the fitting surface, and A method having

2. The method according to claim 1, wherein the step of generating a rib plane display image complemented by rib portions located outside the plane of the fitting surface comprises the step of calculating the distance of each rib centerline point to the fitting surface.

3. The method according to claim 2, wherein the rib portions outside the surface of the fitting surface are highlighted to identify the rib portions that are not part of the rib surface display image.

4. The method according to claim 2, further comprising the step of switching between the rib surface display image and a rib surface display image supplemented with rib portions located outside the plane of the fitting surface.

5. The method according to claim 2, further comprising the step of highlighting the complemented portion in a rib surface display image complemented by a rib portion located outside the plane of the matching surface.

6. The method according to claim 1, wherein the step of detecting and labeling the rib midline is performed via machine learning or deep learning techniques.

7. An imaging device, Memory configured to store computer executable instructions, At least one processor, which executes the computer executable instructions to the imaging device, A step of receiving data representing a three-dimensional diagnostic image, wherein the three-dimensional diagnostic image includes one or more ribs of a subject. A step of segmenting one or more ribs based on the received data representing the three-dimensional diagnostic image via machine learning or deep learning techniques, A step of detecting and labeling the rib centerline from the segmented rib representation, wherein the rib centerline represents a line in the axial direction of the rib that passes through the center of the cross-section of the rib. A step of separately calculating a fitting surface corresponding to the rib centerline for each rib, wherein each fitting surface is separately calculated as a single plane that includes most of the multiple points on the rib centerline of each rib; Based on the received data representing the three-dimensional diagnostic image, the step of determining three-dimensional data for each rib that shows the portion of the rib located outside the plane of the fitting surface, The steps include generating the aforementioned rib cage surface display image, The steps include generating a rib surface display image complemented by the rib portion located outside the plane of the fitting surface by projecting the rib portion located outside the plane of the fitting surface onto the rib surface display image so that it appears as if it is fused with the rib portion located within the plane of the fitting surface, and A processor configured to perform the following: An imaging device having

8. The imaging apparatus according to claim 7, wherein the processor is configured to perform the step of generating a rib plane display image complemented by rib portions located outside the plane of the fitting surface by calculating the distance of each rib centerline point to the fitting surface.

9. The imaging apparatus according to claim 8, wherein the processor is configured to highlight rib portions located outside the plane of the matching surface to identify rib portions that are not part of the rib surface display image.

10. The imaging apparatus according to claim 8, wherein the processor is further configured to switch between the rib surface display image and a rib surface display image supplemented with rib portions located outside the plane of the fitting surface, based on user input.

11. The imaging apparatus according to claim 8, wherein the processor is further configured to highlight the complemented portion in a rib surface display image complemented by a rib portion located outside the plane of the fitting surface, based on user input.

12. The imaging apparatus according to claim 7, wherein the step of detecting and labeling the rib centerline is performed via machine learning or deep learning techniques.

13. A non-temporary computer-readable medium storing instructions for causing a processing circuit to execute a process, wherein the process is: A step of receiving data representing a three-dimensional diagnostic image, wherein the three-dimensional diagnostic image includes one or more ribs of a subject. A step of segmenting one or more ribs based on the received data representing the three-dimensional diagnostic image via machine learning or deep learning techniques, A step of detecting and labeling the rib centerline from the segmented rib representation, wherein the rib centerline represents a line in the axial direction of the rib that passes through the center of the cross-section of the rib. A step of separately calculating a fitting surface corresponding to the rib centerline for each rib, wherein each fitting surface is separately calculated as a single plane that includes most of the multiple points on the rib centerline of each rib; Based on the received data representing the three-dimensional diagnostic image, the step of determining three-dimensional data for each rib that shows the portion of the rib located outside the plane of the fitting surface, The steps include generating the aforementioned rib cage surface display image, The steps include generating a rib surface display image complemented by the rib portion located outside the plane of the fitting surface by projecting the rib portion located outside the plane of the fitting surface onto the rib surface display image so that it appears as if it is fused with the rib portion located within the plane of the fitting surface, and A non-temporary computer-readable medium having [a certain characteristic].

Images