Endoscope advancing route planning method and device, storage medium and equipment

Abstract

The invention relates to an endoscope advancing route planning method and device, a storage medium and equipment, and the method comprises the steps: building a center line network corresponding to a suspected focus region according to an original medical image of the suspected focus region to be examined; according to the centerline network, determining an entry point for endoscope propulsion and constructing a connectivity table of the centerline network, the connectivity being used for representing accessibility between any two network nodes in the centerline network; according to the connectivity table, traversing from the centerline network to determine a propulsion node from the entry point to the specified target check point; and planning a propelling route of the endoscope according to the propelling node. According to the connectivity of the center line network and the network nodes, the advancing route can accurately represent each advancing node from the entrance point to the target examination point, so that the accuracy of the endoscope reaching the suspected focus area to be examined is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of medical equipment, and in particular, to a method, apparatus, storage medium and device for planning an endoscope propulsion route. Background Art

[0002] An endoscope is a medical instrument that can enter the human body through a natural orifice or through an incision made during surgery. When in use, the endoscope is introduced into the body to be examined and then pushed to the suspected lesion area to be examined. Due to the complex internal environment, in order to improve the accuracy of the endoscope reaching the suspected lesion area to be examined, it is necessary to plan the advancement route of the endoscope in the human body in advance. Summary of the invention

[0003] In view of the above technical problems, the purpose of the present disclosure is to provide an endoscope propulsion route planning method, device, storage medium and equipment.

[0004] In order to achieve the above-mentioned purpose, a first aspect of an embodiment of the present disclosure provides an endoscope advancement route planning method, comprising:

[0005] Establishing a centerline network corresponding to the suspected lesion area according to the original medical image of the suspected lesion area to be examined;

[0006] According to the centerline network, determining the entry point for the endoscope to be advanced and constructing a connectivity table of the centerline network, wherein the connectivity is used to characterize the reachability between any two network nodes in the centerline network;

[0007] According to the connectivity table, traversing from the centerline network to determine a propulsion node from the entry point to a designated target checkpoint;

[0008] The propulsion route of the endoscope is planned according to the propulsion node.

[0009] Optionally, the step of establishing a centerline network corresponding to the suspected lesion area according to the original medical image of the suspected lesion area to be examined includes:

[0010] Constructing a three-dimensional model of the suspected lesion area according to the original medical image of the suspected lesion area to be examined;

[0011] Performing image segmentation on the three-dimensional model to obtain a pipeline network corresponding to the pipeline system of each pipeline type;

[0012] The centerline of the pipeline network corresponding to each pipeline type is extracted to obtain the centerline network corresponding to each pipeline type, wherein the line width of the centerline in the centerline network is one volume element, and there is a unique path between any two volume elements on the centerline.

[0013] Optionally, performing image segmentation on the three-dimensional model to obtain a pipeline network corresponding to the pipeline system of each pipeline type includes:

[0014] Delineating a target area including the suspected lesion area in the three-dimensional model;

[0015] Performing volume element scanning on the three-dimensional model within the target area according to the input exposure value range to obtain volume elements corresponding to the target area;

[0016] Taking the input seed point as the starting point, searching for volume elements with a connected relationship in the three-dimensional model according to the exposure value range, and adding display labels to the volume elements with a connected relationship;

[0017] The volume elements without display labels in the three-dimensional model are hidden to obtain a pipeline network corresponding to the pipeline system of each pipeline type.

[0018] Optionally, the step of constructing a connectivity table of the centerline network according to the centerline network comprises:

[0019] Determine the three-dimensional coordinates of the network nodes in the centerline network and the side lengths of two adjacent network nodes along the centerline according to the three-dimensional stereoscopic model;

[0020] Selecting a backup network node from the centerline network according to the three-dimensional coordinates and the side lengths of two adjacent network nodes along the centerline, wherein the backup network node satisfies that the Euclidean distance to the adjacent network node is less than or equal to a preset distance threshold, and the maximum difference between the Euclidean distance to the adjacent network node and the side lengths of the adjacent corresponding network nodes along the centerline is less than or equal to a preset difference threshold;

[0021] A connectivity table of the centerline network is constructed according to the plurality of backup network nodes, wherein the plurality of backup network nodes include network nodes of preset types, and the preset types include a bifurcation point type and a termination point type.

[0022] Optionally, the step of constructing a connectivity table of the centerline network according to the plurality of standby network nodes comprises:

[0023] For any two standby network nodes, determining from the centerline network whether the two standby network nodes are adjacent network nodes;

[0024] In the case where the two standby network nodes are adjacent network nodes, calculating the Euclidean distance between the two standby network nodes according to the three-dimensional coordinates, and filling the two standby network nodes and the corresponding Euclidean distance into a two-dimensional matrix;

[0025] In the case that the two standby network nodes are not adjacent network nodes, recursively searching for an adjacent node for any one of the two standby network nodes until the found network node is an adjacent node of the other of the two standby network nodes, traversing and calculating the Euclidean distances between the found network node and its adjacent network nodes, and taking the sum of the Euclidean distances as the Euclidean distance between the two standby network nodes;

[0026] The two backup network nodes and the corresponding Euclidean distances are filled into a two-dimensional matrix to construct a connectivity table of the centerline network.

[0027] Optionally, the step of determining the entry point for advancing the endoscope according to the centerline network comprises:

[0028] Determine the necessary network nodes to reach the specified target checkpoint from the centerline network;

[0029] Extracting features of the necessary network nodes to obtain node features of each necessary network node;

[0030] At least one entry point for advancing the endoscope is determined from the necessary network nodes according to the node characteristics.

[0031] Optionally, the endoscope advancement route planning method further includes:

[0032] Extracting features from the pipeline area where the entry point is located, and quantizing the extracted entry point features to obtain a standard value of the entry point features;

[0033] Performing feature detection on the endoscopic image collected by the endoscope, and performing feature vectorization on the detected image features to obtain image feature values;

[0034] Comparing the entry point feature standard value with the image feature value;

[0035] If the entry point feature standard value is successfully compared with the image feature value, a synchronization relationship between the endoscope advancement and the advancement route is established;

[0036] Present the surgical guide following the described advancement route.

[0037] According to a second aspect of the embodiments of the present disclosure, there is provided an endoscope propulsion route planning device, comprising:

[0038] A network establishment module is configured to establish a centerline network corresponding to the suspected lesion area according to the original medical image of the suspected lesion area to be examined;

[0039] A first determination module is configured to determine the entry point for the endoscope advancement according to the centerline network and to construct a connectivity table of the centerline network, wherein the connectivity is used to characterize the reachability between any two network nodes in the centerline network;

[0040] A second determination module is configured to traverse and determine a propulsion node from the entry point to a designated target checkpoint from the centerline network according to the connectivity table;

[0041] A route planning module is configured to plan the propulsion route of the endoscope according to the propulsion node.

[0042] Optionally, the network establishing module is configured to:

[0043] Constructing a three-dimensional model of the suspected lesion area according to the original medical image of the suspected lesion area to be examined;

[0044] Performing image segmentation on the three-dimensional model to obtain a pipeline network corresponding to the pipeline system of each pipeline type;

[0045] The centerline of the pipeline network corresponding to each pipeline type is extracted to obtain the centerline network corresponding to each pipeline type, wherein the line width of the centerline in the centerline network is one volume element, and there is a unique path between any two volume elements on the centerline.

[0046] Optionally, the network establishing module is configured to:

[0047] Delineating a target area including the suspected lesion area in the three-dimensional model;

[0048] Performing volume element scanning on the three-dimensional model within the target area according to the input exposure value range to obtain volume elements corresponding to the target area;

[0049] Taking the input seed point as the starting point, searching for volume elements with a connected relationship in the three-dimensional model according to the exposure value range, and adding display labels to the volume elements with a connected relationship;

[0050] The volume elements without display labels in the three-dimensional model are hidden to obtain a pipeline network corresponding to the pipeline system of each pipeline type.

[0051] Optionally, the first determining module is configured to:

[0052] Determine the three-dimensional coordinates of the network nodes in the centerline network and the side lengths of two adjacent network nodes along the centerline according to the three-dimensional stereoscopic model;

[0053] Selecting a backup network node from the centerline network according to the three-dimensional coordinates and the side lengths of two adjacent network nodes along the centerline, wherein the backup network node satisfies that the Euclidean distance to the adjacent network node is less than or equal to a preset distance threshold, and the maximum difference between the Euclidean distance to the adjacent network node and the side lengths of the adjacent corresponding network nodes along the centerline is less than or equal to a preset difference threshold;

[0054] A connectivity table of the centerline network is constructed according to the plurality of backup network nodes, wherein the plurality of backup network nodes include network nodes of preset types, and the preset types include a bifurcation point type and a termination point type.

[0055] Optionally, the first determining module is configured to:

[0056] For any two standby network nodes, determining from the centerline network whether the two standby network nodes are adjacent network nodes;

[0057] In the case where the two standby network nodes are adjacent network nodes, calculating the Euclidean distance between the two standby network nodes according to the three-dimensional coordinates, and filling the two standby network nodes and the corresponding Euclidean distance into a two-dimensional matrix;

[0058] In the case that the two standby network nodes are not adjacent network nodes, recursively searching for an adjacent node for any one of the two standby network nodes until the found network node is an adjacent node of the other of the two standby network nodes, traversing and calculating the Euclidean distances between the found network node and its adjacent network nodes, and taking the sum of the Euclidean distances as the Euclidean distance between the two standby network nodes;

[0059] The two backup network nodes and the corresponding Euclidean distances are filled into a two-dimensional matrix to construct a connectivity table of the centerline network.

[0060] Optionally, the first determining module is configured to:

[0061] Determine the necessary network nodes to reach the specified target checkpoint from the centerline network;

[0062] Extracting features of the necessary network nodes to obtain node features of each necessary network node;

[0063] At least one entry point for advancing the endoscope is determined from the necessary network nodes according to the node characteristics.

[0064] Optionally, the endoscope propulsion route planning device further includes a guiding module configured to:

[0065] Extracting features from the pipeline area where the entry point is located, and quantizing the extracted entry point features to obtain a standard value of the entry point features;

[0066] Performing feature detection on the endoscopic image collected by the endoscope, and performing feature vectorization on the detected image features to obtain image feature values;

[0067] Comparing the entry point feature standard value with the image feature value;

[0068] If the entry point feature standard value is successfully compared with the image feature value, a synchronization relationship between the endoscope advancement and the advancement route is established;

[0069] Present the surgical guide following the described advancement route.

[0070] According to a third aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a first processor, the steps of the endoscope propulsion route planning method described in any one of the first aspects are implemented.

[0071] According to a fourth aspect of the embodiments of the present disclosure, an electronic device is provided, including:

[0072] a first memory having a computer program stored thereon;

[0073] The second processor is used to execute the computer program in the first memory to implement the steps of the endoscope propulsion route planning method according to any one of the first aspects.

[0074] Through the above technical solution, at least the following beneficial effects can be achieved:

[0075] According to the original medical images of the suspected lesion area to be inspected, the centerline network corresponding to the suspected lesion area is established; according to the centerline network, the entry point for endoscope advancement is determined and the connectivity table of the centerline network is constructed, wherein connectivity is used to characterize the accessibility between any two network nodes in the centerline network; according to the connectivity table, the advancement node from the entry point to the designated target checkpoint is determined by traversing the centerline network; and the advancement route of the endoscope is planned according to the advancement node. According to the connectivity of the centerline network and the network nodes, the advancement route can accurately represent each advancement node from the entry point to the target checkpoint, thereby improving the accuracy of the endoscope reaching the suspected lesion area to be inspected.

[0076] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0078] Figure 1 The figure is a flow chart of a method for planning an endoscope propulsion route according to an embodiment of the present disclosure.

[0079] Figure 2 This is an implementation shown according to an embodiment of the present disclosure. Figure 1 Flow chart of step S1 in FIG.

[0080] Figure 3 This is an implementation shown according to an embodiment of the present disclosure. Figure 2 Flowchart of step S12 in FIG.

[0081] Figure 4 This is an implementation shown according to an embodiment of the present disclosure. Figure 1 Flowchart of step S2 in FIG.

[0082] Figure 5 This is an implementation shown according to an embodiment of the present disclosure. Figure 1 Flowchart of step S2 in FIG.

[0083] Figure 6 It is a block diagram of an endoscope propulsion route planning device according to an embodiment of the present disclosure.

[0084] Figure 7 It is a block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0085] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0086] Before introducing the endoscopic image display method, device, storage medium and equipment provided by the present disclosure, the application scenario of the present disclosure is first introduced. During inspection or surgery, the endoscope needs to enter the patient's body, and then gradually advance the endoscope to the tissue organ to be inspected or the surgical area. During the advancement process, the endoscope may encounter complex situations such as path turns, bifurcations, important blood vessels, and nerve organs, which will not only bring difficulties to the advancement of the endoscope, but also cause unnecessary damage to important blood vessels, tissues or nerves, which may cause pain to the patient. Therefore, the doctor needs to confirm the correct advancement direction or surgical range based on prior knowledge or with the help of external instruments.

[0087] In related scenarios, the use of Magnetic Resonance Imaging (MRI) technology, ultrasound technology, or computed tomography (CT) and other radiation technologies can capture images of the endoscope in the human body, which can help doctors determine the exact position and posture of surgical instruments in the human body. First of all, radiation technologies such as CT will cause certain harm to the human body and are not suitable for frequent use. In addition, medical imaging equipment including Magnetic Resonance Imaging (MRI) or computed tomography (CT), ultrasound, etc. are often large and inconvenient to move, so the patient must be moved and the patient is required to maintain a specific posture to cooperate with the imaging. Sometimes these requirements will conflict with other constraints, thereby limiting the applicable scenarios. Moreover, MRI, CT, etc. are all indirect imaging based on changes in physical quantities, not direct imaging, and are not suitable for minimally invasive surgery.

[0088] To this end, the embodiment of the present disclosure provides an endoscope advancement route planning method, which aims to plan the advancement route of the endoscope during the advancement process according to the surgical plan and the original medical images before the operation, find the best route for the endoscope to pass, improve the use scenario of the endoscope, and improve the accuracy of the endoscope reaching the suspected lesion area to be inspected, thereby reducing the pain caused to the patient during the advancement of the endoscope, improving the surgical efficiency and reducing the surgical risk. Figure 1 As shown, the endoscope advancement route planning method includes the following steps.

[0089] In step S1, a centerline network corresponding to the suspected lesion area is established based on the original medical image of the suspected lesion area to be examined.

[0090] In the disclosed embodiment, the original medical image may be a medical image of the patient's suspected lesion area obtained by performing at least one multi-section imaging of the patient's lesion area using digital imaging technology such as magnetic resonance imaging (MRI) or computed tomography (CT).

[0091] Among them, before step S1, all control parameters in the control parameter set of the endoscope advancement route planning can be initialized to preset default values, so as to initialize the parameter values ​​when planning the advancement route of other lesion areas. In subsequent steps, the parameter values ​​of the control parameters are gradually replaced with the parameter values ​​corresponding to the current endoscope advancement route planning, so as to avoid the parameter values ​​when planning the advancement route of other lesion areas affecting the accuracy of this planning.

[0092] In the disclosed embodiment, an image segmentation algorithm can be used to segment the pipeline network of the type related to the current surgical plan from the original medical image, thereby hiding or eliminating other types of pipeline networks, thereby avoiding the influence of irrelevant pipeline networks on the advancement of route planning.

[0093] The type of the pipeline may include at least one of a blood vessel type, a tracheal type, and an intestinal type. For example, when the operation is for the trachea, the endoscope is mainly advanced in the trachea, and the tracheal network can be segmented from the original medical image, thereby hiding or removing the blood vessel type network, the intestinal type network, etc.

[0094] The centerline network corresponding to the suspected lesion area may be constructed based on at least one of the natural pipeline system of the human body, an artificial surgical pathway, or an artificial inspection pathway.

[0095] In step S2, based on the centerline network, the entry point for endoscope advancement is determined and a connectivity table of the centerline network is constructed, wherein connectivity is used to characterize the reachability between any two network nodes in the centerline network.

[0096] Since the suspected lesion area can usually be reached from different directions or different tissues, there can be one or more entry points.

[0097] In the disclosed embodiment, the connectivity table may be a two-dimensional matrix composed of a group of network nodes, and the characteristic of the two-dimensional matrix may be that the matrix element m(i, j) represents the shortest distance between any two network nodes i and j. In other words, the connectivity table records the shortest distance between two network nodes in the centerline network along the centerline network.

[0098] In step S3, according to the connectivity table, the push node from the entry point to the designated target checkpoint is determined from the centerline network.

[0099] In the disclosed embodiment, a designated target checkpoint can generally be reached from one entry point through multiple groups of propulsion nodes, wherein the designated target checkpoint can be reached by propelling the endoscope along the centerline network according to each group of propulsion nodes.

[0100] In the disclosed embodiment, the propulsion node may generally be a pipeline bifurcation point, a pipeline sharp bend point, or a pipeline termination point.

[0101] In step S4, the propulsion route of the endoscope is planned according to the propulsion node.

[0102] In the disclosed embodiment, the propulsion nodes are connected along the centerline network to obtain the propulsion route of the endoscope, and then the connectivity score value of each propulsion route can be determined, thereby determining the difficulty of each propulsion route to reach the designated target checkpoint.

[0103] In the disclosed embodiment, a propulsion node set may be constructed based on the propulsion nodes, and the propulsion node set in the control parameter set may be replaced by the propulsion node set.

[0104] In the disclosed embodiment, the advancement route may be one or more. When there are multiple advancement routes, the multiple advancement routes may be displayed, and then an operation instruction may be received, the operation instruction including the selected target advancement route, and the target advancement route may be used as the final advancement route of the endoscope.

[0105] The above technical solution establishes a centerline network corresponding to the suspected lesion area based on the original medical image of the suspected lesion area to be inspected. According to the centerline network, the entry point for endoscope advancement is determined and the connectivity table of the centerline network is constructed, wherein connectivity is used to characterize the accessibility between any two network nodes in the centerline network. According to the connectivity table, the centerline network is traversed to determine the advancement node from the entry point to the designated target checkpoint. The advancement route of the endoscope is planned based on the advancement node. According to the connectivity of the centerline network and the network nodes, the advancement route can accurately represent each advancement node from the entry point to the target checkpoint, thereby improving the accuracy of the endoscope reaching the suspected lesion area to be inspected.

[0106] Alternatively, see Figure 2 As shown, in step S1, the step of establishing a centerline network corresponding to the suspected lesion area according to the original medical image of the suspected lesion area to be examined includes:

[0107] In step S11, a three-dimensional model of the suspected lesion area is constructed based on the original medical image of the suspected lesion area to be examined.

[0108] In the embodiment of the present disclosure, the original medical images taken from different angles for the suspected lesion area to be examined can be used to construct a three-dimensional model for the suspected lesion area. Or other existing technologies can be used to construct a three-dimensional model, which will not be described in detail here.

[0109] In step S12, the three-dimensional model is segmented to obtain a pipeline network corresponding to each pipeline type in the human body pipeline system.

[0110] In the disclosed embodiments, a traditional image segmentation algorithm based on morphological operations can be used, or a neural network image segmentation algorithm based on a deep learning network, such as a UNet network, can be used to perform image segmentation on the three-dimensional model to obtain a pipeline network corresponding to each pipeline type in the human pipeline system.

[0111] In step S13, the centerline of the pipeline network corresponding to each pipeline type is extracted to obtain the centerline network corresponding to the pipeline network of each pipeline type, wherein the line width of the centerline in the centerline network is one volume element, and there is a unique path between any two volume elements on the centerline.

[0112] In the disclosed embodiment, for the aforementioned pipeline network, the centerline of the pipeline network can be extracted by a centerline extraction algorithm, and the centerline network finally obtained is equivalent to the pipeline network in terms of topological structure and positioning function. Thus, the network structure is simplified. Among them, the centerline network has the following characteristics: 1) The line width of the centerline is one volume element; 2) There is a unique and non-repetitive path between any two voxels on the centerline network.

[0113] The above technical solution can perform image segmentation on the constructed three-dimensional model, extract the pipeline network, and then determine the centerline network corresponding to the pipeline network, so as to reduce the influence between different types of pipelines in the image.

[0114] Alternatively, see Figure 3 As shown, in step S12, the three-dimensional model is segmented to obtain a pipeline network corresponding to each pipeline type in the human body pipeline system, including:

[0115] In step S121 , a target area including a suspected lesion area is delineated in the three-dimensional model.

[0116] In the disclosed embodiment, a three-dimensional area is delineated from the spatial range of the three-dimensional stereo model as a target area, and the target area includes the patient's lesion area, wherein the shape of the target area can be a cube, an oblique cube, a cylinder, an oblique cylinder, or other irregular three-dimensional bodies.

[0117] Furthermore, in the three-dimensional model, hidden labels are added to other volume elements outside the target area, and when the three-dimensional model is presented, the volume elements marked with the hidden labels are hidden, that is, the volume elements marked with the hidden labels are not displayed.

[0118] In step S122, according to the input exposure value range, a volume element scan is performed on the three-dimensional model in the target area to obtain a volume element corresponding to the target area.

[0119] It can be understood that the exposure value range includes a lower threshold and an upper threshold, and the volume element scanning is performed on the three-dimensional stereo model in the target area, and hidden labels are added to the volume elements in the target area whose exposure values ​​fall outside the exposure value range, thereby hiding the volume elements in the target area marked with hidden labels.

[0120] In step S123, taking the input seed point as the starting point, volume elements with connectivity in the three-dimensional model are searched according to the exposure value range, and display labels are added to the volume elements with connectivity.

[0121] In the disclosed embodiment, the input seed point is input by the user in real time according to the surgical requirements. In specific implementation, display labels can be added to volume elements with connectivity, and hidden labels can be added to volume elements without connectivity.

[0122] In step S124, the volume elements without display labels in the three-dimensional model are hidden, and the pipeline network corresponding to the pipeline system of each pipeline type in the human body pipeline system is obtained.

[0123] In the disclosed embodiment, the volume elements marked with hidden labels can be hidden, and the volume elements marked with display labels can be displayed to obtain the pipeline network corresponding to the pipeline system of each pipeline type in the human body pipeline system.

[0124] Optionally, in Figure 2 Based on Figure 4 As shown, in step S2, the step of constructing a connectivity table of the centerline network according to the centerline network includes:

[0125] In step S21, the three-dimensional coordinates of the network nodes in the centerline network and the side lengths of two adjacent network nodes along the centerline are determined according to the three-dimensional stereo model.

[0126] It can be understood that the three-dimensional coordinates of the network node are the coordinates of the network node in the three-dimensional model, and the side length of two adjacent network nodes along the center line can usually represent the actual length of the pipeline, rather than the Euclidean distance between the two adjacent network nodes. That is, the side length of two adjacent network nodes along the center line is usually greater than or equal to the Euclidean distance between the two adjacent network nodes.

[0127] In step S22, a backup network node is selected from the centerline network based on the three-dimensional coordinates and the side lengths of two adjacent network nodes along the centerline, wherein the backup network node satisfies that the Euclidean distance with the adjacent network node is less than or equal to a preset distance threshold, and the maximum difference between the Euclidean distance with the adjacent network node and the side lengths of the adjacent corresponding network nodes along the centerline is less than or equal to a preset difference threshold.

[0128] In the disclosed embodiment, the Euclidean distance between two adjacent network nodes can be calculated based on the three-dimensional coordinates of the two adjacent network nodes. When the Euclidean distance between the adjacent network nodes is less than or equal to a preset distance threshold, or when the Euclidean distance between the adjacent network nodes is greater than the preset distance threshold, the network node is discarded and the Euclidean distance between the next network node and its adjacent network node is calculated.

[0129] Furthermore, the maximum difference between the Euclidean distance of the adjacent network node and the side length of the adjacent corresponding network node along the center line is calculated, and when the maximum difference is less than or equal to the preset difference threshold, the network node is used as a backup network node. When the maximum difference is greater than the preset difference threshold, the network node is discarded, and the Euclidean distance between the next network node and its adjacent network node is calculated.

[0130] In step S23, a connectivity table of the centerline network is constructed according to the plurality of spare network nodes, wherein the plurality of spare network nodes include network nodes of preset types, and the preset types include a branch point type and an end point type.

[0131] It can be understood that the connectivity table indicates the shortest distance from any standby network node to other standby network nodes. Similarly, the distance is usually greater than or equal to the Euclidean distance between the corresponding standby network nodes.

[0132] Optionally, in step S23, the step of constructing a connectivity table of the centerline network according to the plurality of spare network nodes includes:

[0133] For any two spare network nodes, it is determined from the centerline network whether the two spare network nodes are adjacent network nodes.

[0134] In the embodiment of the present disclosure, determining whether two backup network nodes are adjacent network nodes in the centerline network can be understood as whether it is possible to reach another backup network node from any backup network node along the centerline network without passing through other backup network nodes.

[0135] In the case that the two standby network nodes are adjacent network nodes, the Euclidean distance between the two standby network nodes is calculated according to the three-dimensional coordinates, and the two standby network nodes and the corresponding Euclidean distance are filled into the two-dimensional matrix.

[0136] The two standby network nodes are adjacent network nodes, that is, along the central line network, any standby network node can reach the other standby network node without passing through other standby network nodes.

[0137] In the case that the two backup network nodes are not adjacent network nodes, an adjacent node search is performed for any of the two backup network nodes in a recursive manner until the found network node is an adjacent node of the other of the two backup network nodes, and the Euclidean distances between the found network node and its adjacent network nodes are traversed and calculated, and the sum of the Euclidean distances is taken as the Euclidean distance between the two backup network nodes.

[0138] The two standby network nodes are not adjacent network nodes, that is, along the central line network, one must pass through other standby network nodes to reach another standby network node from any standby network node.

[0139] For example, when the backup network node i and the backup network node j are the same network nodes, the shortest distance between the backup network node i and itself must be 0, that is, m(i, i) = 0; when the backup network node i and the backup network node j are different network nodes, if the backup network node i and the backup network node j are two adjacent network nodes, then the shortest distance of m(i, j) can be expressed by the Euclidean distance between the two network nodes; if the backup network node i and the backup network node j are not two adjacent network nodes, then m(i, j) can be split into two parts, that is, m(i, j) = m(i, j-1) + m (j-1, j), wherein the backup network node j-1 is the previous node of the backup network node j in the centerline network, and the backup network node j-1 and the backup network node j are adjacent network nodes, the shortest distance of m(j-1, j) is the Euclidean distance between the backup network node j-1 and the backup network node j, and m(i, j-1) can be further split in a recursive manner, and finally split into the backup network node x, and the backup network node x is the subsequent adjacent node of the backup network node i, and then the Euclidean distance of each pair of adjacent network nodes is calculated, and multiple Euclidean distances are summed to obtain the shortest distance of m(i, j).

[0140] Fill each two spare network nodes and the corresponding Euclidean distance into a two-dimensional matrix to construct a connectivity table of the centerline network.

[0141] Alternatively, see Figure 5 As shown, in step S2, the step of determining the entry point for endoscope advancement according to the centerline network includes:

[0142] In step S201, the necessary network nodes to reach the designated target checkpoint are determined from the centerline network.

[0143] In step S202, feature extraction is performed on the necessary network nodes to obtain node features of each necessary network node.

[0144] In the disclosed embodiment, a network node on the necessary path of the endoscope to the suspected lesion area is searched from the centerline network, and then if the pipeline area where the network node is located has at least one feature suitable for image recognition, the network node is used as an entry point, wherein the feature suitable for image recognition may include: at least one of a node structure feature, a node texture feature, and a node color feature. The feature of the entry point can be vectorized to obtain a feature descriptor of the entry point, and the vectorized feature descriptor is saved as a standard value of the entry point feature in the control parameter set.

[0145] In step S203, at least one entry point for endoscope advancement is determined from the necessary network nodes according to the node characteristics.

[0146] Optionally, the endoscope advancement route planning method further includes:

[0147] The feature extraction is performed on the pipeline area where the entry point is located, and the extracted entry point features are vectorized to obtain the entry point feature standard value.

[0148] The endoscopic image collected by the endoscope is subjected to feature detection, and the detected image features are vectorized to obtain image feature values.

[0149] Compare the entry point feature standard value with the image feature value.

[0150] If the entry point feature standard value and the image feature value are successfully compared, a synchronization relationship between the endoscope advancement and the advancement route is established.

[0151] Demonstrate surgical guidance following advancement routes.

[0152] In specific implementation, conventional operation methods are used to push the endoscope to the vicinity of an entry point on the centerline network according to the requirements of the surgical plan; then the synchronization function of the endoscope navigation software is started to start feature detection of the real-time image output by the endoscope, and the result of the feature detection is vectorized to obtain the image feature value, and then the image feature value is compared with the entry point feature standard value saved in the control parameter set in real time, and the comparison result is output. Among them, the comparison result can be presented as a vector matching score.

[0153] As the endoscope advances, if the matching score between the image feature value corresponding to a frame of image and the entry point feature standard value reaches the preset score, the entry point feature standard value and the image feature value are successfully compared. At this time, it can be determined that the endoscope probe has reached the entry point position, and the image captured by the endoscope establishes a synchronous relationship with the corresponding position in the centerline network, and the system can enter the intraoperative navigation advancement stage. The surgical guidance is displayed according to the advancement route.

[0154] Based on the same concept, the present disclosure also provides an endoscope propulsion route planning device, see Figure 6 As shown, the endoscope propulsion route planning device includes: a network establishment module 610 , a first determination module 620 , a second determination module 630 and a route planning module 640 .

[0155] The network establishment module 610 is configured to establish a centerline network corresponding to the suspected lesion area according to the original medical image of the suspected lesion area to be examined;

[0156] The first determination module 620 is configured to determine the entry point for endoscope advancement according to the centerline network and construct a connectivity table of the centerline network, wherein connectivity is used to characterize the reachability between any two network nodes in the centerline network;

[0157] The second determination module 630 is configured to traverse and determine a propulsion node from the entry point to the designated target checkpoint from the centerline network according to the connectivity table;

[0158] The route planning module 640 is configured to plan the propulsion route of the endoscope according to the propulsion node.

[0159] Optionally, the network establishing module 610 is configured to:

[0160] Constructing a three-dimensional model of the suspected lesion area according to the original medical image of the suspected lesion area to be examined;

[0161] Perform image segmentation on the three-dimensional model to obtain a pipeline network corresponding to each pipeline type in the human body pipeline system;

[0162] The centerline of the pipeline network corresponding to each pipeline type is extracted to obtain the centerline network corresponding to each pipeline type, wherein the line width of the centerline in the centerline network is one volume element, and there is a unique path between any two volume elements on the centerline.

[0163] Optionally, the network establishing module 610 is configured to:

[0164] Delineating the target area including the suspected lesion area in the three-dimensional model;

[0165] According to the input exposure value range, the volume element scanning is performed on the three-dimensional model in the target area to obtain the volume element corresponding to the target area;

[0166] Taking the input seed point as the starting point, searching for volume elements with connectivity in the 3D model according to the exposure value range, and adding display labels to the volume elements with connectivity;

[0167] The volume elements without display labels in the three-dimensional model are hidden, and the pipeline network corresponding to the pipeline system of each pipeline type in the human pipeline system is obtained.

[0168] Optionally, the first determining module 620 is configured to:

[0169] According to the three-dimensional model, determine the three-dimensional coordinates of the network nodes in the centerline network and the side lengths of two adjacent network nodes along the centerline;

[0170] Selecting a backup network node from the centerline network according to the three-dimensional coordinates and the side lengths of two adjacent network nodes along the centerline, wherein the backup network node satisfies that the Euclidean distance to the adjacent network node is less than or equal to a preset distance threshold, and the maximum difference between the Euclidean distance to the adjacent network node and the side lengths of the adjacent corresponding network nodes along the centerline is less than or equal to a preset difference threshold;

[0171] A connectivity table of the centerline network is constructed according to a plurality of spare network nodes, wherein the plurality of spare network nodes include network nodes of preset types, and the preset types include a bifurcation point type and a termination point type.

[0172] Optionally, the first determining module 620 is configured to:

[0173] For any two spare network nodes, determining from the centerline network whether the two spare network nodes are adjacent network nodes;

[0174] In the case where the two standby network nodes are adjacent network nodes, the Euclidean distance between the two standby network nodes is calculated according to the three-dimensional coordinates, and the two standby network nodes and the corresponding Euclidean distance are filled into the two-dimensional matrix;

[0175] In the case that the two standby network nodes are not adjacent network nodes, an adjacent node search is performed for any of the two standby network nodes in a recursive manner until the found network node is an adjacent node of the other of the two standby network nodes, and the Euclidean distances between the found network node and its adjacent network nodes are calculated traversally, and the sum of the Euclidean distances is used as the Euclidean distance between the two standby network nodes;

[0176] Fill each two spare network nodes and the corresponding Euclidean distance into a two-dimensional matrix to construct a connectivity table of the centerline network.

[0177] Optionally, the first determining module 620 is configured to:

[0178] Determine the necessary network nodes to reach the specified target checkpoint from the centerline network;

[0179] Extract features of the must-pass network nodes to obtain node features of each must-pass network node;

[0180] At least one entry point for endoscope advancement is determined from the necessary network nodes according to the node characteristics.

[0181] Optionally, the endoscope propulsion route planning device 600 further includes a guiding module configured to:

[0182] Extract features of the pipeline area where the entry point is located, and vectorize the extracted entry point features to obtain the entry point feature standard value;

[0183] Performing feature detection on the endoscopic image collected by the endoscope, and performing feature vectorization on the detected image features to obtain image feature values;

[0184] Compare the entry point feature standard value with the image feature value;

[0185] If the entry point feature standard value and the image feature value are successfully compared, a synchronization relationship between the endoscope advancement and the advancement route is established;

[0186] Demonstrate surgical guidance following advancement routes.

[0187] Regarding the endoscope propulsion route planning device 600 in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0188] Those skilled in the art should understand that the above-described embodiment of the endoscope propulsion route planning device 600 is merely illustrative. For example, the division of modules is merely a logical function division. There may be other division methods in actual implementation, such as multiple modules may be combined or integrated into one module. In addition, the modules described as separate components may or may not be physically separated. Moreover, each module may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. When implemented using hardware, it may be implemented in whole or in part in the form of an integrated circuit or chip.

[0189] The embodiment of the present disclosure also provides a non-temporary computer-readable storage medium having a computer program stored thereon, which, when executed by a first processor, implements the steps of the endoscope advancement route planning method described in any one of the first aspects.

[0190] The present disclosure also provides an electronic device, including:

[0191] a first memory having a computer program stored thereon;

[0192] The second processor is used to execute the computer program in the first memory to implement the steps of the endoscope propulsion route planning method according to any one of the first aspects.

[0193] Figure 7 is a block diagram of an electronic device 600 according to an exemplary embodiment. The electronic device 600 can be configured as an endoscope propulsion route planning device, such as Figure 7 As shown, the electronic device 600 may include: a third processor 601 , a second memory 602 . The electronic device 600 may also include one or more of a multimedia component 603 , an input / output (I / O) interface 604 , and a communication component 605 .

[0194] The third processor 601 is used to control the overall operation of the electronic device 600 to complete all or part of the steps in the above-mentioned endoscope advancement route planning method. The second memory 602 is used to store various types of data to support the operation of the electronic device 600, and these data may include instructions for any application or method used to operate on the electronic device 600, as well as application-related data, such as pictures, etc. The second memory 602 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, disk or optical disk. The multimedia component 603 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the second memory 602 or sent through the communication component 605. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 604 provides an interface between the third processor 601 and other interface modules, and the other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons for operating endoscopic image storage and display. The communication component 605 is used for wired or wireless communication between the electronic device 600 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited here. Therefore, the corresponding communication component 605 may include: Wi-Fi module, Bluetooth module, NFC module, etc.

[0195] In an exemplary embodiment, the electronic device 600 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned endoscope propulsion route planning method.

[0196] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned endoscope advancement route planning method are implemented. For example, the computer-readable storage medium can be the above-mentioned second memory 602 including program instructions, and the above-mentioned program instructions can be executed by the third processor 601 of the electronic device 600 to complete the above-mentioned endoscope advancement route planning method.

[0197] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0198] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0199] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A method for planning an endoscope propulsion route, characterized in that: include: Establishing a centerline network corresponding to the suspected lesion area according to the original medical image of the suspected lesion area to be examined; According to the centerline network, determining the entry point for the endoscope to be advanced and constructing a connectivity table of the centerline network, wherein the connectivity is used to characterize the reachability between any two network nodes in the centerline network; According to the connectivity table, traversing from the centerline network to determine a propulsion node from the entry point to a designated target checkpoint; The propulsion route of the endoscope is planned according to the propulsion node.

2. The endoscope propulsion route planning method according to claim 1, characterized in that: The step of establishing a centerline network corresponding to the suspected lesion area according to the original medical image of the suspected lesion area to be examined comprises: Constructing a three-dimensional model of the suspected lesion area according to the original medical image of the suspected lesion area to be examined; Performing image segmentation on the three-dimensional model to obtain a pipeline network corresponding to the pipeline system of each pipeline type; The centerline of the pipeline network corresponding to each pipeline type is extracted to obtain the centerline network corresponding to each pipeline type, wherein the line width of the centerline in the centerline network is one volume element, and there is a unique path between any two volume elements on the centerline.

3. The endoscope propulsion route planning method according to claim 2, characterized in that: The performing image segmentation on the three-dimensional model to obtain a pipeline network corresponding to the pipeline system of each pipeline type includes: Delineating a target area including the suspected lesion area in the three-dimensional model; Performing volume element scanning on the three-dimensional model within the target area according to the input exposure value range to obtain volume elements corresponding to the target area; Taking the input seed point as the starting point, searching for volume elements with a connected relationship in the three-dimensional model according to the exposure value range, and adding display labels to the volume elements with a connected relationship; The volume elements without display labels in the three-dimensional model are hidden to obtain a pipeline network corresponding to the pipeline system of each pipeline type.

4. The endoscope propulsion route planning method according to claim 2, characterized in that: The step of constructing a connectivity table of the centerline network according to the centerline network comprises: Determine the three-dimensional coordinates of the network nodes in the centerline network and the side lengths of two adjacent network nodes along the centerline according to the three-dimensional stereoscopic model; Selecting a backup network node from the centerline network according to the three-dimensional coordinates and the side lengths of two adjacent network nodes along the centerline, wherein the backup network node satisfies that the Euclidean distance to the adjacent network node is less than or equal to a preset distance threshold, and the maximum difference between the Euclidean distance to the adjacent network node and the side lengths of the adjacent corresponding network nodes along the centerline is less than or equal to a preset difference threshold; A connectivity table of the centerline network is constructed according to the plurality of backup network nodes, wherein the plurality of backup network nodes include network nodes of preset types, and the preset types include a bifurcation point type and a termination point type.

5. The endoscope propulsion route planning method according to claim 4, characterized in that: The step of constructing a connectivity table of the centerline network according to the plurality of backup network nodes comprises: For any two standby network nodes, determining from the centerline network whether the two standby network nodes are adjacent network nodes; In the case where the two standby network nodes are adjacent network nodes, calculating the Euclidean distance between the two standby network nodes according to the three-dimensional coordinates, and filling the two standby network nodes and the corresponding Euclidean distance into a two-dimensional matrix; In the case that the two standby network nodes are not adjacent network nodes, recursively searching for an adjacent node for any one of the two standby network nodes until the found network node is an adjacent node of the other of the two standby network nodes, traversing and calculating the Euclidean distances between the found network node and its adjacent network nodes, and taking the sum of the Euclidean distances as the Euclidean distance between the two standby network nodes; The two backup network nodes and the corresponding Euclidean distances are filled into a two-dimensional matrix to construct a connectivity table of the centerline network.

6. The endoscope propulsion route planning method according to claim 1, characterized in that: The step of determining the entry point for advancing the endoscope according to the centerline network comprises: Determine the necessary network nodes to reach the specified target checkpoint from the centerline network; Extracting features of the necessary network nodes to obtain node features of each necessary network node; At least one entry point for advancing the endoscope is determined from the necessary network nodes according to the node characteristics.

7. The endoscope propulsion route planning method according to any one of claims 1 to 6, characterized in that: The endoscope propulsion route planning method further includes: Extracting features from the pipeline area where the entry point is located, and quantizing the extracted entry point features to obtain a standard value of the entry point features; Performing feature detection on the endoscopic image collected by the endoscope, and performing feature vectorization on the detected image features to obtain image feature values; Comparing the entry point feature standard value with the image feature value; If the entry point feature standard value is successfully compared with the image feature value, a synchronization relationship between the endoscope advancement and the advancement route is established; Present the surgical guide following the described advancement route.

8. An endoscope propulsion route planning device, characterized in that: include: A network establishment module is configured to establish a centerline network corresponding to the suspected lesion area according to the original medical image of the suspected lesion area to be examined; A first determination module is configured to determine the entry point for the endoscope advancement and construct a connectivity table of the centerline network according to the centerline network, wherein the connectivity is used to characterize the reachability between any two network nodes in the centerline network; A second determination module is configured to traverse and determine a propulsion node from the entry point to a designated target checkpoint from the centerline network according to the connectivity table; A route planning module is configured to plan the propulsion route of the endoscope according to the propulsion node.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by the first processor, the steps of the endoscope advancement route planning method described in any one of claims 1 to 7 are implemented.

10. An electronic device, characterized in that: include: a first memory having a computer program stored thereon; A second processor is used to execute the computer program in the first memory to implement the steps of the endoscope propulsion route planning method according to any one of claims 1 to 7.

Images