Preoperative planning method and device, readable storage medium and electronic equipment

Abstract

Embodiments of the present application provide a preoperative planning method and device, a readable storage medium and an electronic device. The specific implementation scheme is as follows: a target lung three-dimensional model is obtained, and lung segment segmentation is performed on the target lung three-dimensional model to obtain a lung segment segmentation result; a lesion area in the target lung three-dimensional model is determined; based on the lesion area and the lung segment segmentation result, a target lung segment is determined; and preoperative planning is performed based on the target lung segment. Based on the present scheme, preoperative planning can be automatically performed, and the accuracy of the preoperative planning is high, thereby reducing the dependence of preoperative planning on manpower.

Description

Technical Field

[0001] This application relates to the field of medical data processing technology, and more specifically, to a preoperative planning method, apparatus, readable storage medium, and electronic device. Background Technology

[0002] Lung segments are defined by anatomical division of the lungs, and this division is of great significance in the medical field, such as for preoperative planning of lung surgeries.

[0003] Preoperative planning is required for lung segmentectomy. Currently, preoperative planning mainly relies on manual methods, and there is an urgent need for an intelligent solution to reduce the dependence on manual methods.

[0004] Furthermore, preoperative planning relies on lung segmentation. The morphology of pulmonary blood vessels varies in healthy individuals, and these variations can affect lung segmentation. Therefore, accurate lung segmentation has become a crucial technical challenge. Summary of the Invention

[0005] This application provides a preoperative planning method, apparatus, readable storage medium, and electronic device, aiming to at least address one of the aforementioned technical deficiencies. The technical solution adopted in this application is as follows:

[0006] In a first aspect, embodiments of this application provide a preoperative planning method, the method comprising:

[0007] Obtain a three-dimensional model of the target lung and perform lung segmentation on the three-dimensional model of the target lung to obtain the lung segmentation results;

[0008] Identify the lesion region in the three-dimensional model of the target lung;

[0009] The target lung segment was determined based on the lesion area and lung segmentation results;

[0010] Preoperative planning is based on the target lung segment.

[0011] As an alternative approach, the target lung segment is determined based on the lesion region and lung segmentation results, including:

[0012] The lesion area is expanded based on a preset expansion strategy to obtain the expanded area.

[0013] The lung segments that intersect with the outward expansion region in the lung segmentation results are identified as the target lung segments.

[0014] As an optional approach, the lesion area is expanded based on a preset expansion strategy to obtain an expanded area, including:

[0015] The outer contour of the lesion area is moved outward by a preset distance to obtain the expanded area;

[0016] Take the region of the circumscribed sphere of the expanded region as the outer expansion region.

[0017] As an alternative approach, preoperative planning based on the target lung segment includes:

[0018] The surgical path is determined based on the root point of the artery within the target lung segment and the nearest surface point to the root point.

[0019] As an alternative approach, the surgical path is determined based on the root point of the artery within the target lung segment and the nearest surface point, including:

[0020] Determine the line connecting the root point of the artery within the target lung segment to the nearest surface point.

[0021] If the line does not pass through the rib area, the line is identified as the surgical path;

[0022] In response to the line passing through the rib region, the surface point is translated so that the line connecting the translated surface point and the root point does not pass through the rib region, and the line connecting the translated surface point and the root point is determined as the surgical path.

[0023] As an optional approach, lung segmentation is performed on the three-dimensional model of the target lung to obtain lung segmentation results, including:

[0024] A pre-trained lung segmentation model is used to segment the target lung into three-dimensional models, and the lung segmentation results are obtained.

[0025] As an alternative approach, the lung segmentation model is trained in the following manner:

[0026] Obtain training samples, which include the three-dimensional lung model of the sample and the ground truth of the vascular segmentation results in the three-dimensional lung model of the sample;

[0027] The lung segmentation model is used to segment and predict the points of the vascular part in the three-dimensional model of the lung sample, and the prediction results of the vascular segmentation are obtained.

[0028] Based on the predicted results of vascular segmentation and the true values ​​of vascular segmentation results, the loss is determined, and the lung segmentation model is trained based on the loss.

[0029] As an alternative approach, the blood vessels include the trachea, and the above methods also include:

[0030] Based on the preset first tracheal segmentation model, the points of the tracheal part in the three-dimensional model of the lungs of the sample are segmented and predicted to obtain multiple tracheal segments.

[0031] Based on the preset second tracheal segmentation model, each tracheal segment is segmented and predicted to obtain tracheal subsegments, which are then used as the true values ​​of the tracheal segmentation results.

[0032] As an alternative approach, the vessels also include arteries, and the methods described above also include:

[0033] Based on the preset first artery segmentation model, the points of the artery part in the three-dimensional model of the lung sample are segmented into instances to obtain multiple first connected components.

[0034] Based on the preset second artery segmentation model, each first connected component is segmented into instances to obtain the second connected component.

[0035] Based on the positional relationship between the tracheal subsegment and the second connected domain, the corresponding arterial subsegment is determined, and the arterial subsegment is used as the true value of the arterial segmentation result.

[0036] As an alternative approach, based on the positional relationship between the tracheal subsegment and the second connected domain, the corresponding arterial subsegment is determined, including:

[0037] The expanded region is obtained by extending the edge of the second connected domain outward by a preset distance.

[0038] The target tracheal subsegment is determined from the tracheal subsegments based on the number of points in the expanded region for each tracheal subsegment.

[0039] Based on the pre-defined correspondence between tracheal subsegments and arterial subsegments, the target arterial subsegment corresponding to the target tracheal subsegment is determined.

[0040] The target arterial subsegment is taken as the arterial subsegment corresponding to the second connected domain.

[0041] As an alternative approach, the blood vessels include veins, and the above methods also include:

[0042] Based on a pre-defined intersegmental vein recognition model, points in the vein portion of the sample lung 3D model are predicted to obtain multiple intersegmental veins, which are then used as the ground truth for the intersegmental vein segmentation results.

[0043] As an optional approach, the loss is determined based on the piecewise prediction results and the true values ​​of the piecewise results, including at least one of the following:

[0044] Based on the predicted results of tracheal segmentation and the true value of tracheal segmentation, the tracheal segmentation loss is determined.

[0045] Based on the predicted results of arterial segments and the true values ​​of arterial segments, the arterial segment loss is determined;

[0046] Based on the predicted results of intersegmental veins and the true value of intersegmental veins, segmental loss of intersegmental veins is determined, wherein the true value of intersegmental veins is the same as the true value of the lung segments on both sides of the intersegmental veins.

[0047] Secondly, embodiments of this application provide a preoperative planning device, the device comprising:

[0048] The lung segmentation module is used to acquire a three-dimensional model of the target lung and perform lung segmentation on the three-dimensional model of the target lung to obtain the lung segmentation results;

[0049] The lesion region determination module is used to determine the lesion region in the target lung 3D model;

[0050] The target lung segment determination module is used to determine the target lung segment based on the lesion area and lung segmentation results;

[0051] The preoperative planning module is used for preoperative planning based on the target lung segment.

[0052] Thirdly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the preoperative planning method described above.

[0053] Fourthly, embodiments of this application provide an electronic device, which includes:

[0054] One or more processors; and

[0055] A memory associated with one or more of the aforementioned processors, the memory being used to store program instructions that, when read and executed by the one or more of the aforementioned processors, perform the steps of the aforementioned preoperative planning method.

[0056] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the preoperative planning method described above.

[0057] The beneficial effects of the technical solutions provided in this application are:

[0058] The solution provided in this application involves acquiring a three-dimensional model of the target lung, segmenting the target lung into lung segments to obtain the segmentation results, identifying the lesion region in the target lung's three-dimensional model, determining the target lung segment based on the lesion region and the lung segmentation results, and performing preoperative planning based on the target lung segment. This solution enables automatic preoperative planning with high accuracy, thereby reducing reliance on manual intervention in preoperative planning.

[0059] Furthermore, by obtaining the ground truth values ​​of the sample lung 3D model and the vascular segmentation results within the sample lung 3D model as training samples, the lung segmentation model is used to predict the segmentation of points in the vascular portion of the sample lung 3D model, obtaining the predicted vascular segmentation results. Based on the predicted vascular segmentation results and the ground truth values ​​of the vascular segmentation results, the loss is determined, and the lung segmentation model is trained based on the loss. The lung segmentation model trained using this method can learn the information of vascular segmentation, thus improving the accuracy of lung segmentation results when performing lung segmentation based on the lung segmentation model. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0061] Figure 1 This is a system architecture diagram applicable to the embodiments of this application;

[0062] Figure 2 A flowchart illustrating the preoperative planning method provided in this application embodiment;

[0063] Figure 3 This is a schematic diagram of the lung segmentation results of the target lung three-dimensional model in the embodiments of this application;

[0064] Figure 4 A schematic diagram of the interface for 3D reconstruction;

[0065] Figure 5 This is a schematic diagram of the interface for preoperative planning.

[0066] Figure 6 A schematic diagram of the tracheal subsegments identified in the target lung 3D model;

[0067] Figure 7 A schematic diagram of the tracheal and arterial subsegments identified in the target lung 3D model;

[0068] Figure 8 A schematic diagram of the surgical planning system is provided for the embodiments of this application;

[0069] Figure 9 This is a schematic diagram of the preoperative planning device provided in the embodiments of this application;

[0070] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0071] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0072] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0073] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0074] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0075] To facilitate understanding of this application, the system architecture on which this application is based will first be described. For example... Figure 1 The diagram shows an exemplary system architecture that can be applied to embodiments of this application, such as... Figure 1 As shown, the system architecture may include: a terminal, a network, and a server.

[0076] A network is a medium used to provide a communication link between a server and a terminal. Networks can include various connection types, such as wired and wireless communication links or fiber optic cables, etc.

[0077] The terminal may include, but is not limited to, mobile phones, tablets, and smart wearable devices. Smart wearable devices may include smartwatches, smart glasses, smart bracelets, VR (Virtual Reality) devices, AR (Augmented Reality) devices, mixed reality devices (i.e., devices that support both virtual and augmented reality), and so on. In this embodiment, the user equipment typically has information display functionality, such as a display screen. In addition, it may also have operation command input functionality, such as inputting operation commands via touchscreen, keyboard, or voice.

[0078] The server can be a dedicated server, a server cluster, or a cloud server. A cloud server, also known as a cloud computing server or cloud host, is a hosting product within the cloud computing service system, designed to address the shortcomings of traditional physical hosts and Virtual Private Servers (VPS) services, such as high management difficulty and weak service scalability.

[0079] In one embodiment, the preoperative planning method provided in this application can be executed by a terminal or a server.

[0080] It should be understood that Figure 1 The number of servers and terminals shown is merely illustrative. Depending on implementation needs, there can be any number of servers and terminals.

[0081] Figure 2 This illustration shows a flowchart of a preoperative planning method provided in an embodiment of this application. The method can be performed by… Figure 1 This is executed on the terminal or server in the system shown. For example... Figure 2 As shown, this method mainly includes:

[0082] Step S210: Obtain a three-dimensional model of the target lung and perform lung segmentation on the three-dimensional model of the target lung to obtain the lung segmentation results;

[0083] Step S220: Identify the lesion region in the three-dimensional model of the target lung;

[0084] Step S230: Based on the lesion area and lung segmentation results, determine the target lung segment;

[0085] Step S240: Perform preoperative planning based on the target lung segment.

[0086] The target lung 3D model can be reconstructed based on medical images of the patient's lungs. Analyzing the target lung 3D model enables preoperative planning for the patient.

[0087] As an example, a three-dimensional reconstruction of the patient's chest tissue can be performed to obtain a three-dimensional model of the chest, and then a three-dimensional model of the lungs can be segmented from the three-dimensional model of the chest, that is, the target three-dimensional model of the lungs.

[0088] By segmenting the target lung into segments using a 3D model, lung segmentation results can be obtained. As an example, Figure 3 This is a schematic diagram of the lung segmentation results of the three-dimensional model of the target lung in this embodiment of the application.

[0089] The lesion area refers to the region in the three-dimensional model of the target lung where lesions such as nodules, pneumonia, and tumors are located.

[0090] As an example, lesion areas can be detected from a three-dimensional model of the target lung based on a pre-defined lesion detection model.

[0091] The lung segmentation result refers to the lung segment in the three-dimensional model of the target lung. Based on the lesion area and the lung segmentation result, the target lung segment can be determined. The target lung segment is the lung segment to be removed during the lesion resection surgery, thus enabling preoperative planning based on the target lung segment.

[0092] The method provided in this application involves acquiring a three-dimensional model of the target lung, segmenting the target lung into lung segments to obtain the segmentation results, identifying the lesion region in the target lung's three-dimensional model, determining the target lung segment based on the lesion region and the lung segmentation results, and performing preoperative planning based on the target lung segment. This method can automatically perform preoperative planning with high accuracy, thereby reducing the reliance on manual intervention in preoperative planning.

[0093] In one optional embodiment of this application, determining the target lung segment based on the lesion region and lung segmentation results includes:

[0094] The lesion area is expanded based on a preset expansion strategy to obtain the expanded area.

[0095] The lung segments that intersect with the outward expansion region in the lung segmentation results are identified as the target lung segments.

[0096] In this embodiment of the application, the lesion area can be expanded outward to obtain an expanded area, and the lung segment that intersects with the expanded area is taken as the target lung segment.

[0097] As an example, after identifying the target lung segment, a surgical paradigm can be determined based on it, which may involve either resecting the target lung segment or resecting a lung lobe. Specifically, a threshold can be set for the proportion of the lung segment to be resected in each lung lobe, i.e., a maximum value can be set for the ratio of the lung segment to be resected to the total number of lung segments in the lobe. When the ratio of the target lung segment to the total number of lung segments in the lobe is greater than this threshold, lobectomy is required; when the ratio of the target lung segment to the total number of lung segments in the lobe is not greater than this threshold, resection of the target lung segment is required.

[0098] In one optional embodiment of this application, the lesion area is expanded based on a preset expansion strategy to obtain an expanded area, including:

[0099] The outer contour of the lesion area is moved outward by a preset distance to obtain the expanded area;

[0100] Take the region of the circumscribed sphere of the expanded region as the outer expansion region.

[0101] In this embodiment, the preset expansion strategy can be as follows: first, the outer contour of the lesion area is expanded outward to obtain the expanded area, and then the area of ​​the outer sphere of the expanded area is taken as the expansion area.

[0102] As an example, the preset distance can be the distance of 3 voxels.

[0103] In one optional embodiment of this application, preoperative planning based on the target lung segment includes:

[0104] The surgical path is determined based on the root point of the artery within the target lung segment and the closest surface point to the root point in the 3D chest model.

[0105] Surgical path refers to the access path during surgery.

[0106] In this embodiment, after determining the target lung segment, the root point of the artery within the target lung segment can be obtained. The nearest point on the body surface of the chest model to this root point can then be determined, thereby establishing the surgical path based on the artery root point and the body surface point. For example, the body surface point can be used as the starting point of the surgical path, and the artery root point can be used as the ending point, thus determining the surgical path.

[0107] In one optional embodiment of this application, the surgical path is determined based on the root point of the artery within the target lung segment and the surface point in the three-dimensional chest model closest to the root point, including:

[0108] Determine the line connecting the root point of the artery within the target lung segment to the nearest surface point.

[0109] If the connection does not pass through the rib area in the 3D chest model, the connection will be identified as the surgical path.

[0110] In response to the line passing through the rib region in the 3D chest model, the surface point is translated so that the line connecting the translated surface point and the root point does not pass through the rib region in the 3D chest model, and the line connecting the translated surface point and the root point is determined as the surgical path.

[0111] In this embodiment, a line can be drawn between the root point of the artery and the nearest point on the body surface. This line may pass through the rib area in the three-dimensional model of the chest. In this case, the body surface point needs to be translated to avoid the rib area. Then, the line between the translated body surface point and the root point can be used as the surgical path.

[0112] As an example, the body surface point can be translated in a direction away from the area of ​​the ribs being traversed.

[0113] When the line connecting the root of an artery to the nearest point on the body surface does not pass through the rib area, this line can be used directly as the surgical path.

[0114] As an example, this application embodiment may provide a 3D reconstruction interface and a preoperative planning interface to assist in the implementation of preoperative planning.

[0115] Figure 4 This is a schematic diagram of the interface for 3D reconstruction.

[0116] like Figure 4 As shown, this interface displays a 3D model of the target lung in a window and provides a list of lesions. Users can select lesions and selectively display them. Each anatomical structure in the target lung 3D model is independent and can be individually displayed (either shown or not).

[0117] After generating a three-dimensional model of the target lung, preoperative planning can be performed based on the model, including lesion selection, surgical paradigm, and surgical approach.

[0118] Figure 5 This is a schematic diagram of the interface for preoperative planning.

[0119] like Figure 5 As shown, this interface provides a 3D visual representation of each surgical step, allowing users to select and edit them to create a preoperative planning document. The preoperative planning document contains key information such as the final determined surgical approach and pathway.

[0120] As an example, intraoperative navigation can be provided based on the preoperative planning file after it has been generated.

[0121] In one optional embodiment of this application, the above method further includes:

[0122] After generating the preoperative planning file, in response to the command to transfer the preoperative planning file, the preoperative planning file is transferred to the intraoperative navigation system using a preset transfer method, which includes any of the following:

[0123] The preoperative planning documents are transferred to a removable storage device so that the removable storage device can be connected to the intraoperative navigation system, thereby transferring the preoperative planning documents to the intraoperative navigation system;

[0124] Based on a wired or wireless connection with the intraoperative navigation system, the preoperative planning documents are transferred to the intraoperative navigation system.

[0125] In this embodiment, the navigation function of the intraoperative navigation system relies on the preoperative planning file. Therefore, it is necessary to transfer the preoperative planning file from the image comparison, follow-up, and management system to the intraoperative laparoscopic navigation system using a preset transfer method before surgery. The preset transfer method may specifically include transfer based on a removable storage device, transfer based on a wired network connection, or transfer based on a wireless network connection.

[0126] As an example, the intraoperative navigation system is a standalone system that can be brought into the operating room. Preoperative planning files can be transferred to a removable storage device, such as a USB flash disk, and then the removable storage device can be connected to the intraoperative navigation system, allowing the preoperative planning files on the removable storage device to be transferred to the intraoperative navigation system. Preoperative planning files can also be sent to the intraoperative navigation system via wired or wireless connections.

[0127] In one optional embodiment of this application, lung segmentation is performed on the three-dimensional model of the target lung to obtain lung segmentation results, including:

[0128] A pre-trained lung segmentation model is used to segment the target lung into three-dimensional models, and the lung segmentation results are obtained.

[0129] The lung segmentation model can be used to predict lung segments from a 3D lung model, thereby identifying the lung segments. The 3D lung model can include a backbone network and a detection head. The backbone network extracts features from each point in the 3D lung model, and the detection head predicts lung segments based on the features extracted by the backbone network, thus obtaining the lung segmentation result. Lung segment prediction is essentially a classification task, using each lung segment as a category, that is, predicting the lung segment to which a point in the 3D lung model belongs.

[0130] As an example, sparse convolution can be used for feature extraction in the backbone network of a 3D lung model.

[0131] In one optional embodiment of this application, the lung segmentation model is trained in the following manner:

[0132] Obtain training samples, which include the three-dimensional lung model of the sample and the ground truth of the vascular segmentation results in the three-dimensional lung model of the sample;

[0133] The lung segmentation model is used to segment and predict the points of the vascular part in the three-dimensional model of the lung sample, and the prediction results of the vascular segmentation are obtained.

[0134] Based on the predicted results of vascular segmentation and the true values ​​of vascular segmentation results, the loss is determined, and the lung segmentation model is trained based on the loss.

[0135] In this context, "vascular system" refers to the collective term for arteries, veins, and trachea. It is understood that, in the embodiments of this application, "vascular system" refers to the arteries, veins, and trachea of ​​the lungs.

[0136] The true value of the vascular segmentation result in the three-dimensional model of the lung sample can be obtained by pre-annotating the vascular segments in the three-dimensional model of the lung sample. It is understood that this annotation process can be done manually or by automatically segmenting the blood vessels.

[0137] During training, a lung segmentation model can be used to predict the segments of the vascular portion in the 3D model of the sample lung, obtaining the predicted vascular segmentation results. Then, based on the predicted vascular segmentation results and the ground truth values, a loss is calculated, and the lung segmentation model is trained based on this loss. Through this training process, the lung segmentation model can learn information about vascular segments. Since lung segments are generally divided based on the distribution of blood vessels, enabling the lung segmentation model to learn this information improves its accuracy in identifying lung segments.

[0138] The lung segmentation model trained using this method can learn information about vascular segments, thereby improving the accuracy of lung segmentation results when performing lung segmentation based on the lung segmentation model.

[0139] When labeling the true value of the vessel segmentation results using an automatic segmentation method for vessels, the automatic segmentation of vessels may include at least one of the following: automatic segmentation of the trachea, automatic segmentation of the arteries, and automatic segmentation of the intersegmental veins.

[0140] In one optional embodiment of this application, the blood vessels include the trachea, and the above method further includes:

[0141] Based on the preset first tracheal segmentation model, the points of the tracheal part in the three-dimensional model of the lungs of the sample are segmented and predicted to obtain multiple tracheal segments.

[0142] Based on the preset second tracheal segmentation model, each tracheal segment is segmented and predicted to obtain tracheal subsegments, which are then used as the true values ​​of the tracheal segmentation results.

[0143] In this embodiment of the application, when automatically segmenting the trachea, the points of the trachea part in the three-dimensional model of the lung sample can be segmented and predicted based on the first trachea segmentation model, that is, the points of the trachea part are classified, with 18 trachea segments as categories, thereby identifying the trachea segments.

[0144] In this embodiment of the application, after the tracheal segment is identified, the second tracheal segmentation model can be used to perform segmentation prediction for each tracheal segment to obtain tracheal subsegments. That is, the points in each tracheal segment are classified, and the tracheal subsegments are used as the categories, so that at least one tracheal subsegment is identified in each tracheal segment. Then, the identified tracheal subsegments can be used as the true value of the tracheal segmentation result.

[0145] As an example, the first tracheal segment model and the second tracheal segment model can be a Residual U-shaped Network (ResUnet) or other types of classification models.

[0146] As an example, Figure 6 This is a schematic diagram of the tracheal subsegments identified in the three-dimensional model of the target lung.

[0147] like Figure 6 As shown, points in the trachea portion of the three-dimensional lung model can be classified into different tracheal subsegments.

[0148] In one optional embodiment of this application, the vascular system further includes an artery, and the above method further includes:

[0149] Based on the preset first artery segmentation model, the points of the artery part in the three-dimensional model of the lung sample are segmented into instances to obtain multiple first connected components.

[0150] Based on the preset second artery segmentation model, each first connected component is segmented into instances to obtain the second connected component.

[0151] Based on the positional relationship between the tracheal subsegment and the second connected domain, the corresponding arterial subsegment is determined, and the arterial subsegment is used as the true value of the arterial segmentation result.

[0152] In this embodiment, when segmenting arteries, due to the numerous branches within an artery, directly predicting arterial segmentation may not yield accurate results. This application employs a method of identifying connected components in a three-dimensional lung model and matching these components with tracheal subsegments, which effectively provides arterial segmentation results.

[0153] In this embodiment, a first artery segmentation model can be used to perform 3D instance segmentation of points in the arterial portion of the sample lung 3D model, resulting in multiple first connected components. Then, a second artery segmentation model is used to perform 3D instance segmentation on each first connected component, resulting in second connected components. These second connected components represent the arterial subsegments in the sample lung 3D model, and need to be matched to the corresponding arterial subsegments.

[0154] The arteries and trachea in the lungs run alongside each other, so that the tracheal subsegments and arterial subsegments within the same lung subsegment correspond one-to-one and are close to each other in location. Figure 7 This is a schematic diagram of the tracheal and arterial subsegments identified in a 3D model of the target lung. Figure 7 As shown in the figure, the arterial subsegment and the tracheal subsegment are located close to each other within the same lung subsegment and correspond one-to-one.

[0155] In this embodiment of the application, after determining the second connected domain, based on the tracheal subsegments identified in the previous embodiments, the second connected domain can be matched to the corresponding arterial subsegments based on the positional relationship between the tracheal subsegments and the second connected domain, thereby determining the true value of the arterial segmentation result.

[0156] In one optional embodiment of this application, determining the arterial subsegment corresponding to the second connected domain based on the positional relationship between the tracheal subsegment and the second connected domain includes:

[0157] The expanded region is obtained by extending the edge of the second connected domain outward by a preset distance.

[0158] The target tracheal subsegment is determined from the tracheal subsegments based on the number of points in the expanded region for each tracheal subsegment.

[0159] Based on the pre-defined correspondence between tracheal subsegments and arterial subsegments, the target arterial subsegment corresponding to the target tracheal subsegment is determined.

[0160] The target arterial subsegment is taken as the arterial subsegment corresponding to the second connected domain.

[0161] In this embodiment of the application, since the tracheal subsegment and the arterial subsegment are located close to each other, by expanding the edge of the second connected domain by a preset distance to obtain the expanded region, the points in the tracheal subsegment adjacent to the second connected domain can be located in the expanded region. By counting the number of points in the tracheal subsegment located in the expanded region, the target tracheal subsegment can be determined from the tracheal subsegments.

[0162] As an example, the mode can be calculated for the number of points in the expanded region where the midpoint of each tracheal subsegment is located, and the tracheal subsegment corresponding to the mode can be used as the target tracheal subsegment.

[0163] In this embodiment, the preset correspondence between tracheal subsegments and arterial subsegments is a one-to-one correspondence. The target tracheal subsegment is the tracheal subsegment adjacent to the second connected domain. Based on the preset correspondence between tracheal subsegments and arterial subsegments, the target arterial subsegment corresponding to the target tracheal subsegment can be determined. This target arterial subsegment is the arterial subsegment corresponding to the second connected domain, thereby achieving the matching of the second connected domain and the arterial subsegment.

[0164] In one optional embodiment of this application, the blood vessels include veins, and the above method further includes:

[0165] Based on a pre-defined intersegmental vein recognition model, points in the vein portion of the sample lung 3D model are predicted to obtain multiple intersegmental veins, which are then used as the ground truth for the intersegmental vein segmentation results.

[0166] In this embodiment, intersegmental veins are veins located between two lung segments or subsegments. Intersegmental veins can be identified by predicting points in the venous portion of a sample lung 3D model based on a preset intersegmental vein recognition model; that is, by classifying the points in the venous portion into multiple intersegmental vein categories.

[0167] As an example, the intersegmental vein recognition model can be ResUnet, or it can be other types of classification models.

[0168] In one optional embodiment of this application, the loss is determined based on the segmented prediction results and the true values ​​of the segmented results, including at least one of the following:

[0169] Based on the predicted results of tracheal segmentation and the true value of tracheal segmentation, the tracheal segmentation loss is determined.

[0170] Based on the predicted results of arterial segments and the true values ​​of arterial segments, the arterial segment loss is determined;

[0171] Based on the predicted results of intersegmental veins and the true value of intersegmental veins, segmental loss of intersegmental veins is determined, wherein the true value of intersegmental veins is the same as the true value of the lung segments on both sides of the intersegmental veins.

[0172] In this embodiment, both the tracheal and arterial segments are consistent with the lung segments; that is, both tracheal and arterial segments are located within the lung segments. Therefore, segmentation prediction for points in the trachea can yield predicted tracheal segments. Then, based on the predicted tracheal segments and their true values, tracheal segment loss can be determined. Similarly, segmentation prediction for the arterial segments can yield predicted arterial segments. Then, based on the predicted arterial segments and their true values, arterial segment loss can be determined.

[0173] Intersegmental veins are generally located between two lung segments. The points of intersegmental veins are not segmented into the lung segments on both sides. Therefore, the true value of intersegmental veins should be the same as the true value of the lung segments on both sides of the intersegmental veins. Based on the predicted results of intersegmental veins and the true value of intersegmental veins, the segmental loss of intersegmental veins can be determined.

[0174] In this embodiment of the application, the loss can be determined based on at least one of tracheal segmentation loss, arterial segmentation loss, and intersegmental venous segmentation loss, and the lung segmentation model can be trained based on the loss.

[0175] As an example, a comprehensive loss can be determined based on tracheal segmentation loss, arterial segmentation loss, and intersegmental venous segmentation loss, and then the lung segmentation model can be trained based on the comprehensive loss.

[0176] As an example, this application also provides a surgical planning system. Figure 8 This is a schematic diagram of the surgical planning system.

[0177] like Figure 8 As shown, the surgical planning system mainly includes the following modules:

[0178] The 3D reconstruction module 810 is used to perform 3D reconstruction of chest tissue based on medical images to obtain a 3D model of the chest tissue. This 3D model is then segmented to obtain 3D models of the lungs, lung lobes, arteries, veins, trachea, nodules, lymph nodes, ribs, and body surface tissues. In this example, the segmentation of the chest tissue can be achieved based on a preset segmentation model.

[0179] The lung, artery, vein and trachea segmentation identification module 820 is used to perform lung segmentation, and the lung segmentation results will include segments of arteries, veins and trachea.

[0180] The lesion detection module 830 is used to detect and identify lesions (such as nodules, pneumonia, tumors, etc.) based on medical images. In this example, lesion detection can be implemented based on a preset lesion detection model.

[0181] The surgery-needing module 840 is used to determine whether surgery is needed based on information about the lesion. In this example, the determination of whether surgery is needed can be based on a preset surgical prediction model.

[0182] The lesion lung segment watershed calculation module 850 is used to plan the lung segment of the surgical watershed based on the lesion area, that is, to determine the lung segment to be resected.

[0183] The surgical paradigm and surgical path calculation module 860 is used to determine the surgical paradigm and plan the surgical path. The surgical paradigm includes the lung segment or lung lobe to be removed.

[0184] The overall surgical planning process based on the above surgical planning system includes:

[0185] (1) Used to perform three-dimensional reconstruction of chest tissue based on medical images to obtain a three-dimensional model of chest tissue. Then, the three-dimensional model of chest tissue is segmented to obtain three-dimensional models of lung, lung lobe, artery, vein, trachea, nodules, lymph nodes, ribs, body surface and other tissues.

[0186] (2) Based on medical imaging to detect lesion areas. Lesions may include, but are not limited to, nodules, pneumonia, tumors, etc.

[0187] (3) Based on the lesion information, such as the size and nature of the lesion, and in conjunction with whether the patient has other diseases, determine whether surgery is necessary.

[0188] If surgery is required, preoperative planning will be conducted. Surgical planning will determine the lung segment to be removed and plan the surgical approach.

[0189] (4) The three-dimensional lung model is segmented according to medical definitions. Each lung segment may also include segments of the trachea, arteries, and veins.

[0190] (5) Determine the surgical paradigm and surgical path based on the lung segment and the area of ​​the lesion.

[0191] The surgical paradigm involves either removing a target lung segment or removing a lung lobe. Lobe resection is required when the ratio of the target lung segment to the total number of lung segments in a lobe exceeds a preset threshold; conversely, segmental resection is required when the ratio is not greater than the threshold. The surgical path refers to the access route during surgery. It can be determined by connecting the arterial root of the lung segment to be resected to the nearest adjacent surface point.

[0192] Based on and Figure 2 The method shown follows the same principle. Figure 9 This application provides a schematic diagram of the structure of a preoperative planning device according to an embodiment of the present application. Figure 9 As shown, the preoperative planning device 900 may include:

[0193] The lung segmentation module 910 is used to acquire a three-dimensional model of the target lung and perform lung segmentation on the three-dimensional model of the target lung to obtain the lung segmentation result;

[0194] The lesion region determination module 920 is used to determine the lesion region in the target lung three-dimensional model;

[0195] The target lung segment determination module 930 is used to determine the target lung segment based on the lesion region and the lung segmentation results;

[0196] The preoperative planning module 940 is used for preoperative planning based on the target lung segment.

[0197] As an optional approach, the target lung segment determination module 930 is specifically used for:

[0198] The lesion area is expanded based on a preset expansion strategy to obtain the expanded area.

[0199] The lung segments that intersect with the outward expansion region in the lung segmentation results are identified as the target lung segments.

[0200] As an optional approach, the target lung segment determination module 930, when expanding the lesion area based on a preset expansion strategy to obtain the expanded area, is specifically used for:

[0201] The outer contour of the lesion area is moved outward by a preset distance to obtain the expanded area;

[0202] Take the region of the circumscribed sphere of the expanded region as the outer expansion region.

[0203] As an optional approach, the preoperative planning module 940 is specifically used for:

[0204] The surgical path is determined based on the root point of the artery within the target lung segment and the closest surface point to the root point in the 3D chest model.

[0205] As an optional approach, the preoperative planning module 940, when determining the surgical path based on the root point of the artery within the target lung segment and the nearest surface point to the root point in the 3D chest model, is specifically used for:

[0206] Determine the line connecting the root point of the artery within the target lung segment to the nearest surface point.

[0207] If the connection does not pass through the rib area in the 3D chest model, the connection will be identified as the surgical path.

[0208] In response to the line passing through the rib region in the 3D chest model, the surface point is translated so that the line connecting the translated surface point and the root point does not pass through the rib region in the 3D chest model, and the line connecting the translated surface point and the root point is determined as the surgical path.

[0209] As an optional method, the lung segmentation module 910, when performing lung segmentation on the target lung 3D model to obtain lung segmentation results, is specifically used for:

[0210] A pre-trained lung segmentation model is used to segment the target lung into three-dimensional models, and the lung segmentation results are obtained.

[0211] As an alternative approach, the lung segmentation model is trained in the following manner:

[0212] Obtain training samples, which include the three-dimensional lung model of the sample and the ground truth of the vascular segmentation results in the three-dimensional lung model of the sample;

[0213] The lung segmentation model is used to segment and predict the points of the vascular part in the three-dimensional model of the lung sample, and the prediction results of the vascular segmentation are obtained.

[0214] Based on the predicted results of vascular segmentation and the true values ​​of vascular segmentation results, the loss is determined, and the lung segmentation model is trained based on the loss.

[0215] As an alternative, the blood vessel includes the trachea, and the device further includes a truth-determination module, which is used for:

[0216] Based on the preset first tracheal segmentation model, the points of the tracheal part in the three-dimensional model of the lungs of the sample are segmented and predicted to obtain multiple tracheal segments.

[0217] Based on the preset second tracheal segmentation model, each tracheal segment is segmented and predicted to obtain tracheal subsegments, which are then used as the true values ​​of the tracheal segmentation results.

[0218] As an optional approach, the vascular system also includes arteries, and the truth-determination module is also used for:

[0219] Based on the preset first artery segmentation model, the points of the artery part in the three-dimensional model of the lung sample are segmented into instances to obtain multiple first connected components.

[0220] Based on the preset second artery segmentation model, each first connected component is segmented into instances to obtain the second connected component.

[0221] Based on the positional relationship between the tracheal subsegment and the second connected domain, the corresponding arterial subsegment is determined, and the arterial subsegment is used as the true value of the arterial segmentation result.

[0222] As an optional approach, the truth-determination module, when determining the arterial subsegment corresponding to the second connected domain based on the positional relationship between the tracheal subsegment and the second connected domain, is specifically used for:

[0223] The expanded region is obtained by extending the edge of the second connected domain outward by a preset distance.

[0224] The target tracheal subsegment is determined from the tracheal subsegments based on the number of points in the expanded region for each tracheal subsegment.

[0225] Based on the pre-defined correspondence between tracheal subsegments and arterial subsegments, the target arterial subsegment corresponding to the target tracheal subsegment is determined.

[0226] The target arterial subsegment is taken as the arterial subsegment corresponding to the second connected domain.

[0227] As an optional approach, the blood vessels include veins, and the truth-determination module is also used for:

[0228] Based on a pre-defined intersegmental vein recognition model, points in the vein portion of the sample lung 3D model are predicted to obtain multiple intersegmental veins, which are then used as the ground truth for the intersegmental vein segmentation results.

[0229] As an optional approach, the truth-determination module determines the loss based on the piecewise prediction results and the truth values ​​of the piecewise results, specifically for at least one of the following:

[0230] Based on the predicted results of tracheal segmentation and the true value of tracheal segmentation, the tracheal segmentation loss is determined.

[0231] Based on the predicted results of arterial segments and the true values ​​of arterial segments, the arterial segment loss is determined;

[0232] Based on the predicted results of intersegmental veins and the true value of intersegmental veins, segmental loss of intersegmental veins is determined, wherein the true value of intersegmental veins is the same as the true value of the lung segments on both sides of the intersegmental veins.

[0233] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0234] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0235] In addition, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of any of the methods in the foregoing method embodiments.

[0236] And an electronic device, comprising:

[0237] One or more processors; and

[0238] A memory associated with one or more processors, the memory being used to store program instructions that, when read and executed by one or more processors, perform the steps of any of the methods in the foregoing method embodiments.

[0239] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the methods in the foregoing method embodiments.

[0240] in, Figure 10 An exemplary architecture of an electronic device is shown, which may include a processor 1010, a video display adapter 1011, a disk drive 1012, an input / output interface 1013, a network interface 1014, and a memory 1020. The processor 1010, video display adapter 1011, disk drive 1012, input / output interface 1013, network interface 1014, and memory 1020 can communicate with each other via a communication bus 1030.

[0241] The processor 1010 can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs and implement the technical solution provided in this application.

[0242] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system 1021 for controlling the operation of the electronic device 1000, and the basic input / output system (BIOS) 1022 for controlling the low-level operations of the electronic device 1000. Additionally, it can store a web browser 1023, a data storage management system 1024, and a preoperative planning device 1025, etc. The aforementioned preoperative planning device 1025 can be the application program that specifically implements the aforementioned steps in this embodiment. In summary, when the technical solution provided in this application is implemented through software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0243] Input / output interface 1013 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0244] The network interface 1014 is used to connect the communication module (not shown in the figure) to enable communication and interaction between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0245] Bus 1030 includes a pathway for transmitting information between various components of the device (e.g., processor 1010, video display adapter 1011, disk drive 1012, input / output interface 1013, network interface 1014, and memory 1020).

[0246] It should be noted that although the above-described device only shows the processor 1010, video display adapter 1011, disk drive 1012, input / output interface 1013, network interface 1014, memory 1020, bus 1030, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the solution of this application, and does not necessarily include all the components shown in the figures.

[0247] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer program product. This computer program product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of the embodiments of this application.

[0248] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A preoperative planning method, characterized in that, include: A three-dimensional model of the target lung is obtained, and lung segmentation is performed on the three-dimensional model of the target lung to obtain lung segmentation results; Identify the lesion region in the three-dimensional model of the target lung; Based on the lesion area and the lung segmentation results, the target lung segment is determined; Preoperative planning is based on the target lung segment.

2. The method according to claim 1, characterized in that, The determination of the target lung segment based on the lesion region and the lung segmentation results includes: The lesion area is expanded based on a preset expansion strategy to obtain an expanded area. The lung segments that intersect with the outward expansion region in the lung segmentation results are identified as the target lung segments.

3. The method according to claim 2, characterized in that, The process of expanding the lesion area based on a preset expansion strategy to obtain the expanded area includes: The outer contour of the lesion area is moved outward by a predetermined distance to obtain the expanded area; The region of the circumscribed sphere of the expanded region is taken as the outer expansion region.

4. The method according to claim 1, characterized in that, Preoperative planning based on the target lung segment includes: The surgical path is determined based on the root point of the artery within the target lung segment and the closest surface point to the root point in the three-dimensional chest model.

5. The method according to claim 4, characterized in that, The determination of the surgical path based on the root point of the artery within the target lung segment and the nearest surface point to the root point in the three-dimensional chest model includes: Determine the line connecting the root point of the artery within the target lung segment and the nearest surface point to the root point; If the connecting line does not pass through the rib area in the three-dimensional chest model, then the connecting line is determined as the surgical path; In response to the line passing through the rib region in the three-dimensional chest model, the surface point is translated so that the line connecting the surface point and the root point after translation does not pass through the rib region in the three-dimensional chest model, and the line connecting the surface point and the root point after translation is determined as the surgical path.

6. The method according to any one of claims 1-5, characterized in that, The step of segmenting the target lung 3D model into lung segments to obtain lung segmentation results includes: The lung segmentation results are obtained by segmenting the target lung three-dimensional model using a pre-trained lung segmentation model.

7. The method according to claim 6, characterized in that, The lung segmentation model is trained in the following manner: Obtain training samples, which include a three-dimensional model of the lung sample and the ground truth of the vascular segmentation results in the three-dimensional model of the lung sample; The lung segmentation model is used to segment and predict the points of the vascular part in the three-dimensional model of the lung of the sample, and the prediction results of the vascular segmentation are obtained. Based on the predicted results of the vascular segmentation and the true values ​​of the vascular segmentation results, the loss is determined, and the lung segmentation model is trained based on the loss.

8. The method according to claim 7, characterized in that, The blood vessels include the trachea, and the method further includes: Based on the preset first tracheal segmentation model, the points of the tracheal part in the three-dimensional model of the lungs of the sample are segmented and predicted to obtain multiple tracheal segments. Based on the preset second tracheal segmentation model, each tracheal segment is segmented and predicted to obtain tracheal subsegments, and the tracheal subsegments are used as the true values ​​of the tracheal segmentation results.

9. The method according to claim 8, characterized in that, The blood vessels also include arteries, and the method further includes: Based on the preset first artery segmentation model, the points of the artery part in the three-dimensional model of the lung sample are segmented into instances to obtain multiple first connected components. Based on the preset second artery segmentation model, each first connected component is segmented into instances to obtain the second connected component; Based on the positional relationship between the tracheal subsegment and the second connected domain, the arterial subsegment corresponding to the second connected domain is determined, and the arterial subsegment is used as the true value of the arterial segmentation result.

10. The method according to claim 9, characterized in that, The step of determining the arterial subsegment corresponding to the second connected domain based on the positional relationship between the tracheal subsegment and the second connected domain includes: The expanded region is obtained by extending the edge of the second connected domain outward by a preset distance. The target tracheal subsegment is determined from the tracheal subsegments based on the number of points located within the expanded region for each of the tracheal subsegments. Based on the preset correspondence between tracheal subsegments and arterial subsegments, the target arterial subsegment corresponding to the target tracheal subsegment is determined; The target arterial subsegment is taken as the arterial subsegment corresponding to the second connected domain.

11. The method according to claim 7, characterized in that, The blood vessels include veins, and the method further includes: Based on a preset intersegmental vein recognition model, multiple intersegmental veins are predicted from points in the vein portion of the sample lung three-dimensional model, and these intersegmental veins are used as the ground truth of the intersegmental vein segmentation results.

12. The method according to claim 7, characterized in that, The determination of loss based on the segmented prediction results and the true values ​​of the segmented results includes at least one of the following: Based on the predicted results of tracheal segmentation and the true value of tracheal segmentation, the tracheal segmentation loss is determined. Based on the predicted results of arterial segments and the true values ​​of arterial segments, the arterial segment loss is determined; Based on the predicted results of intersegmental veins and the true value of intersegmental veins, segmental loss of intersegmental veins is determined, wherein the true value of the intersegmental veins is the same as the true value of the lung segments on both sides of the intersegmental veins.

13. A preoperative planning device, characterized in that, include: The lung segmentation module is used to acquire a three-dimensional model of the target lung and to segment the target lung into lung segments to obtain lung segmentation results. The lesion region determination module is used to determine the lesion region in the target lung three-dimensional model; The target lung segment determination module is used to determine the target lung segment based on the lesion region and the lung segmentation results; The preoperative planning module is used to perform preoperative planning based on the target lung segment.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method according to any one of claims 1-12.

15. An electronic device, characterized in that, include: One or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method according to any one of claims 1-12.

16. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-12.

Images