Endoscope advancement route planning method, apparatus, storage medium, and device

Abstract

The present disclosure relates to an endoscope pushing route planning method and device, a storage medium and equipment, comprising: establishing a center line network corresponding to a suspected lesion area to be examined according to an original medical image of the suspected lesion area; determining an entry point of the endoscope pushing and constructing a connectivity table of the center line network according to the center line network, wherein the connectivity is used to represent the reachability between any two network nodes in the center line network; determining a pushing node from the entry point to a specified target examination point by traversing the center line network according to the connectivity table; and planning the pushing route of the endoscope according to the pushing node. According to the center line network and the connectivity of the network nodes, the pushing route can accurately represent each pushing node from the entry point to the target examination point, thereby improving the accuracy of the endoscope reaching the suspected lesion area to be examined.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of medical equipment, in particular, to an endoscope pushing route planning method and device, a storage medium and equipment. BACKGROUND

[0002] An endoscope is a medical instrument that can be introduced into a human body through a natural orifice or a surgically created incision. When used, the endoscope is introduced into the human body to be examined and then pushed to a suspected lesion area to be examined. Due to the complexity of the internal environment, in order to improve the accuracy of the endoscope reaching the suspected lesion area to be examined, the pushing route of the endoscope in the human body needs to be planned in advance. SUMMARY

[0003] In view of the above technical problems, the purpose of the present disclosure is to provide an endoscope pushing route planning method, device, storage medium and equipment.

[0004] In order to achieve the above purpose, the first aspect of the embodiment of the present disclosure provides an endoscope pushing route planning method, comprising:

[0005] According to the original medical image of the suspected lesion area to be examined, a center line network corresponding to the suspected lesion area is established;

[0006] According to the center line network, an entry point of the endoscope pushing is determined and a connectivity table of the center line network is constructed, wherein the connectivity is used to represent the reachability between any two network nodes in the center line network;

[0007] According to the connectivity table, a pushing node from the entry point to a specified target examination point is determined by traversing the center line network;

[0008] According to the pushing node, the pushing route of the endoscope is planned.

[0009] Optionally, the step of establishing the center line network corresponding to the suspected lesion area according to the original medical image of the suspected lesion area to be examined comprises:

[0010] According to the original medical image of the suspected lesion area to be examined, a three-dimensional model for the suspected lesion area is constructed;

[0011] The three-dimensional model is subjected to image segmentation to obtain a pipe network corresponding to each pipe type of the pipe system;

[0012] extracting a center line of the pipe network corresponding to each pipe type to obtain a center line network corresponding to the pipe network of the pipe system of each pipe type, wherein a line width of the center line in the center line network is one volume element, and there is a unique path between any two volume elements on the center line.

[0013] Optionally, the image segmentation on the three-dimensional model to obtain the pipe network corresponding to the pipe system of each pipe type comprises:

[0014] delimiting a target region including the suspected lesion region in the three-dimensional model;

[0015] According to the input exposure value range, volume element scanning is performed on the three-dimensional model in the target region to obtain the volume element corresponding to the target region.

[0016] Taking the input seed point as a starting point, according to the exposure value range, volume elements having a connected relationship in the three-dimensional model are searched, and display labels are added to the volume elements having the connected relationship.

[0017] The volume elements in the three-dimensional model that are not added with display labels are hidden to obtain the pipe network corresponding to the pipe system of each pipe type.

[0018] Optionally, according to the center line network, the step of constructing a connectivity table of the center line network comprises:

[0019] According to the three-dimensional model, the three-dimensional coordinates of the network nodes in the center line network are determined, and the edge length of adjacent two network nodes along the center line is determined.

[0020] According to the three-dimensional coordinates and the edge length of adjacent two network nodes along the center line, a backup network node is selected from the center line network, wherein the backup network node satisfies that the Euclidean distance with the adjacent network node is less than or equal to a preset distance threshold, and the maximum difference between the Euclidean distance with the adjacent network node and the edge length of the adjacent corresponding network node along the center line is less than or equal to a preset difference threshold.

[0021] According to a plurality of backup network nodes, a connectivity table of the center line network is constructed, wherein the plurality of backup network nodes include network nodes of a preset type, and the preset type includes a bifurcation point type and a termination point type.

[0022] Optionally, the step of constructing the connectivity table of the center line network according to a plurality of backup network nodes comprises:

[0023] For any two backup network nodes, it is determined from the center line network whether the two backup network nodes are adjacent network nodes.

[0024] In the case that the two backup network nodes are adjacent network nodes, a Euclidean distance between the two backup network nodes is calculated according to the three-dimensional coordinates, and the two backup network nodes and the corresponding Euclidean distance are filled into a two-dimensional matrix;

[0025] In the case that the two backup network nodes are not adjacent network nodes, a neighbor node search is performed on any network node of the two backup network nodes in a recursive manner until a network node searched is a neighbor node of the other network node of the two backup network nodes, and Euclidean distances between the searched network node and its neighbor network nodes are calculated and iterated, and a sum of the Euclidean distances is taken as the Euclidean distance of the two backup network nodes;

[0026] The two backup network nodes and the corresponding Euclidean distance are filled into a two-dimensional matrix, and a connectivity table of the centerline network is constructed.

[0027] Optionally, the step of determining the entry point of the endoscope pushing according to the centerline network comprises:

[0028] determining, from the centerline network, network nodes that must be passed through to reach the specified target checkpoint;

[0029] performing feature extraction on the network nodes that must be passed through to obtain node features of each of the network nodes that must be passed through;

[0030] determining at least one entry point of the endoscope pushing from the network nodes that must be passed through according to the node features.

[0031] Optionally, the endoscope pushing route planning method further comprises:

[0032] performing feature extraction on a pipe region where the entry point is located, and performing feature vectorization on the extracted entry point features to obtain an entry point feature standard value;

[0033] performing feature detection on an endoscope image collected by the endoscope, and performing feature vectorization on the detected image features to obtain an image feature value;

[0034] comparing the entry point feature standard value with the image feature value;

[0035] if the comparison of the entry point feature standard value and the image feature value is successful, establishing a synchronization relationship between the endoscope pushing and the pushing route;

[0036] displaying a surgical guide according to the pushing route.

[0037] In a second aspect, the present disclosure provides an endoscope pushing route planning device, comprising:

[0038] a network establishing module configured to establish a centerline network corresponding to a suspected lesion area to be examined according to an original medical image of the suspected lesion area;

[0039] a first determining module configured to determine an entry point of the endoscope pushing and a connectivity table of the centerline network according to the centerline network, wherein the connectivity is used to represent the reachability between any two network nodes in the centerline network;

[0040] a second determining module configured to determine a pushing node from the entry point to a specified target examination point by traversing the centerline network according to the connectivity table;

[0041] a route planning module configured to plan the pushing route of the endoscope according to the pushing node.

[0042] Optionally, the network establishing module is configured to:

[0043] construct a three-dimensional model for the suspected lesion area according to the original medical image of the suspected lesion area to be examined;

[0044] perform image segmentation on the three-dimensional model to obtain a pipe network corresponding to a pipe system of each pipe type;

[0045] extract a centerline from the pipe network corresponding to each pipe type to obtain a centerline network corresponding to the pipe network of each pipe type, wherein the line width of the centerline in the centerline network is one volume element, and there is a unique path between any two volume elements on the centerline.

[0046] Optionally, the network establishing module is configured to:

[0047] delimit a target area including the suspected lesion area in the three-dimensional model;

[0048] perform volume element scanning on the three-dimensional model in the target area according to an input exposure value range to obtain a volume element corresponding to the target area;

[0049] take an input seed point as a starting point, search for volume elements having a connectivity relationship in the three-dimensional model according to the exposure value range, and add a display label to the volume elements having the connectivity relationship;

[0050] hide the volume elements in the three-dimensional model that have not been added with the display label to obtain a pipe network corresponding to a pipe system of each pipe type.

[0051] Optionally, the first determining module is configured to:

[0052] determine, according to the three-dimensional model, three-dimensional coordinates of the network nodes in the centerline network, and edge lengths of adjacent two network nodes along the centerline;

[0053] select, according to the three-dimensional coordinates and the edge lengths of adjacent two network nodes along the centerline, backup network nodes from the centerline network, wherein the backup network nodes satisfy that a Euclidean distance with an adjacent network node is less than or equal to a preset distance threshold, and a maximum difference between the Euclidean distance with the adjacent network node and the edge length of the adjacent corresponding network node along the centerline is less than or equal to a preset difference threshold;

[0054] construct a connectivity table of the centerline network according to a plurality of the backup network nodes, wherein the plurality of the backup network nodes include network nodes of a preset type, and the preset type includes a bifurcation point type and a termination point type.

[0055] Optionally, the first determining module is configured to:

[0056] determine, for any two backup network nodes, whether the two backup network nodes are adjacent network nodes from the centerline network;

[0057] in a case where the two backup network nodes are adjacent network nodes, calculate a Euclidean distance between the two backup network nodes according to the three-dimensional coordinates, and fill the two backup network nodes and the corresponding Euclidean distance into a two-dimensional matrix;

[0058] in a case where the two backup network nodes are not adjacent network nodes, perform adjacent node searching on any network node of the two backup network nodes in a recursive manner until a searched network node is an adjacent node of the other network node of the two backup network nodes, iteratively calculate Euclidean distances between the searched network node and its adjacent network nodes, and take a sum of the Euclidean distances as the Euclidean distance of the two backup network nodes;

[0059] fill each of the two backup network nodes and the corresponding Euclidean distance into a two-dimensional matrix, and construct a connectivity table of the centerline network.

[0060] Optionally, the first determining module is configured to:

[0061] determine, from the centerline network, network nodes that must be passed through to reach the specified target checkpoint;

[0062] extracting features of the network nodes, to obtain node features of each of the network nodes;

[0063] determining at least one entry point of the endoscope from the network nodes according to the node features.

[0064] Optionally, the endoscope route planning device further comprises a guiding module configured to:

[0065] extracting features of a pipe region where the entry point is located, and vectorizing the extracted entry point features to obtain entry point feature standard values;

[0066] detecting features of the endoscope images collected by the endoscope, and vectorizing the detected image features to obtain image feature values;

[0067] comparing the entry point feature standard values with the image feature values;

[0068] if the comparison of the entry point feature standard values and the image feature values is successful, establishing a synchronization relationship between the endoscope advancement and the route;

[0069] displaying the surgical guidance according to the route.

[0070] In a third aspect of the embodiments of the present disclosure, a non-transitory computer readable storage medium is provided, which stores a computer program. The program is executed by a first processor to implement the steps of the endoscope route planning method according to any one of the first aspect.

[0071] In a fourth aspect of the embodiments of the present disclosure, an electronic device is provided, which comprises:

[0072] a first memory storing a computer program;

[0073] a second processor configured to execute the computer program in the first memory to implement the steps of the endoscope route planning method according to any one of the first aspect.

[0074] By means of the above technical solution, at least the following beneficial effects can be achieved:

[0075] According to the original medical image of the suspected lesion area to be checked, a center line network corresponding to the suspected lesion area is established; according to the center line network, an entry point of the endoscope is determined and a connectivity table of the center line network is constructed, wherein the connectivity is used to represent the reachability between any two network nodes in the center line network; according to the connectivity table, a pushing node from the entry point to a specified target check point is determined by traversing the center line network; and the pushing route of the endoscope is planned according to the pushing node. According to the center line network and the connectivity of the network nodes, the pushing route can accurately represent each pushing node from the entry point to the target check point, thereby improving the accuracy of the endoscope reaching the suspected lesion area to be checked.

[0076] Other features and advantages of the present disclosure will be described in detail in the following detailed description section. BRIEF DESCRIPTION OF DRAWINGS

[0077] The accompanying drawings are included to provide a further understanding of the present disclosure and constitute a part of the specification, which together with the following detailed description, serve to explain the present disclosure. In the drawings:

[0078] Figure 1 FIG. 1 is a flowchart of an endoscope pushing route planning method according to an embodiment of the present disclosure.

[0079] Figure 2 FIG. 2 is a flowchart of implementing step S1 in FIG. 1 according to an embodiment of the present disclosure. Figure 1

[0080] Figure 3 FIG. 3 is a flowchart of implementing step S12 in FIG. 1 according to an embodiment of the present disclosure. Figure 2

[0081] Figure 4 FIG. 4 is a flowchart of implementing step S2 in FIG. 1 according to an embodiment of the present disclosure. Figure 1

[0082] Figure 5 FIG. 5 is a flowchart of implementing step S2 in FIG. 1 according to an embodiment of the present disclosure. Figure 1

[0083] Figure 6 FIG. 6 is a block diagram of an endoscope pushing route planning device according to an embodiment of the present disclosure.

[0084] Figure 7 FIG. 7 is a block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0085] ​​​​The specific embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present disclosure, and are not used to limit the present disclosure.

[0086] Before introducing the endoscope image display method, device, storage medium and equipment provided by the present disclosure, the scene to which the present disclosure is applied is introduced. During examination or surgery, an endoscope needs to be inserted into the patient's body, and then gradually pushed to the tissue organ to be examined or the operation area. During the pushing process, the endoscope may encounter complex situations such as path turning, branching, important blood vessels, nerve organs, etc. Not only will it bring difficulties to the endoscope pushing, but also may cause unnecessary damage to important blood vessels, tissues or nerves, which may cause pain to the patient. Therefore, the doctor needs to confirm the correct pushing direction or operation range according to prior knowledge or with the help of external instruments.

[0087] In the related scenario, the image of the endoscope in the human body can be taken by using the magnetic resonance imaging (MRI) technology, or the ultrasonic technology, or the radiological technology such as computed tomography (CT), which can help the doctor to judge the accurate position and posture of the surgical instrument in the human body. First, the radiological technology such as CT will cause certain harm to the human body, and is not suitable for frequent use. In addition, medical imaging equipment including magnetic resonance imaging (MRI) or computed tomography (CT), ultrasound, etc. are often very large in size and inconvenient to move, so the patient must be moved and needs to maintain a specific posture to cooperate with imaging. Sometimes these requirements will conflict with other constraints, thereby limiting the applicable scenarios. And MRI, CT, etc. are indirect imaging according to physical quantity changes, not direct imaging, which is not suitable for minimally invasive surgery.

[0088] Therefore, the embodiment of the present disclosure provides an endoscope pushing route planning method, which aims to plan the pushing route of the endoscope in the pushing process according to the operation scheme and the original medical image before the operation, find a better route for the endoscope to pass through, improve the use scene of the endoscope, improve the accuracy of the endoscope to reach the suspected lesion area to be examined, thereby reducing the pain to the patient caused by the endoscope in the pushing process, improving the operation efficiency and reducing the operation risk, as shown in Figure 1 The endoscope pushing route planning method includes the following steps.

[0089] In step S1, a center line network corresponding to the suspected lesion area is established according to the original medical image of the suspected lesion area to be examined.

[0090] In the embodiments of the present disclosure, the original medical image can be obtained by using a digital imaging technology such as magnetic resonance imaging (MRI) or computed tomography (CT) to perform at least one multi-slice imaging on a lesion area of a patient.

[0091] Before step S1, all the control parameters in the control parameter set of the endoscope advancing route planning can be initialized to preset default values first, so as to initialize the parameter values when planning the advancing route of other lesion areas, and then in the subsequent steps, the parameter values of the control parameters are replaced by the parameter values corresponding to the current endoscope advancing route planning, so as to avoid the influence of the parameter values when planning the advancing route of other lesion areas on the accuracy of the current planning.

[0092] In the embodiments of the present disclosure, the image segmentation algorithm can be used to segment the pipeline network of the type related to the current surgical plan from the original medical image, so as to hide or exclude other types of pipeline networks, and avoid the influence of irrelevant pipeline networks on the advancing route planning.

[0093] The type of the pipeline can include at least one of a blood vessel type, a trachea type, and an intestinal tract type. For example, when the current surgery is for the trachea, the endoscope mainly advances in the trachea, and the trachea network can be segmented from the original medical image, so as to hide or exclude the blood vessel type network and the intestinal tract type network.

[0094] The centerline network corresponding to the suspected lesion area can be built according to at least one of a natural pipeline system of a human body, an artificial surgical access, or an artificial examination access.

[0095] In step S2, the entry point of the endoscope advancing and the connectivity table of the centerline network are determined according to the centerline network, wherein the connectivity is used to represent the reachability between any two network nodes in the centerline network.

[0096] Since the suspected lesion area can usually be reached from different directions or different tissues, the entry point can be one or more.

[0097] In the embodiments of the present disclosure, the connectivity table can be a two-dimensional matrix composed of a group of network nodes, and the characteristics of the two-dimensional matrix can be that the matrix element m(i, j) represents the shortest distance between any two network nodes i and j. That is, the connectivity table records the shortest distance between two network nodes in the centerline network along the centerline network.

[0098] In step S3, according to the connectivity table, the advancing nodes from the entry point to the specified target checkpoint are determined by traversing the center line network.

[0099] In the embodiments of the present disclosure, there can be multiple sets of advancing nodes from one entry point to the specified target checkpoint, and each set of advancing nodes can reach the specified target checkpoint.

[0100] In the embodiments of the present disclosure, the advancing nodes can be generally pipe bifurcation points, pipe sharp bend points, and pipe termination points.

[0101] In step S4, the advancing route of the endoscope is planned according to the advancing nodes.

[0102] In the embodiments of the present disclosure, the advancing nodes are connected along the center line network, and the advancing route of the endoscope can be obtained. Furthermore, the connectivity score value of each advancing route can be determined, so as to determine the difficulty of each advancing route to reach the specified target checkpoint.

[0103] In the embodiments of the present disclosure, the advancing node set can be constructed according to the advancing nodes, and the advancing node set in the control parameter set can be replaced by the advancing node set.

[0104] In the embodiments of the present disclosure, the advancing route can be one or multiple, and in the case of multiple advancing routes, the multiple advancing routes can be displayed, and then an operation instruction including a selected target advancing route is received, and the target advancing route is taken as the final advancing route of the endoscope.

[0105] The above technical solution establishes a center line network corresponding to a suspected lesion area according to an original medical image of the suspected lesion area to be examined. According to the center line network, an entry point of the endoscope advancing and a connectivity table of the center line network are determined, wherein the connectivity is used to represent the reachability between any two network nodes in the center line network. According to the connectivity table, the advancing nodes from the entry point to the specified target checkpoint are determined by traversing the center line network. According to the advancing nodes, the advancing route of the endoscope is planned. According to the connectivity of the center line network and the network nodes, the advancing route can accurately represent each advancing node from the entry point to the target checkpoint, thereby improving the accuracy of the endoscope reaching the suspected lesion area to be examined.

[0106] Optionally, referring to Figure 2 as shown, in step S1, the step of establishing a center line network corresponding to a suspected lesion area according to an original medical image of the suspected lesion area to be examined, comprises:

[0107] In step S11, a three-dimensional model for the suspected lesion area to be examined is constructed according to the original medical image of the suspected lesion area to be examined.

[0108] In the embodiments of the present disclosure, the original medical image obtained from different angles for the suspected lesion area to be checked can be used to construct a three-dimensional model for the suspected lesion area. Or other existing technologies construct a three-dimensional model, which is not described here.

[0109] In step S12, the three-dimensional model is subjected to image segmentation to obtain a pipe network corresponding to each pipe type in the human pipe system.

[0110] In the embodiments of the present disclosure, a traditional image segmentation algorithm based on morphological operation can be used, or a neural network image segmentation algorithm based on a deep learning network, such as a UNet network, can be used to segment the three-dimensional model to obtain a pipe network corresponding to each pipe type in the human pipe system.

[0111] In step S13, the centerline of each pipe network corresponding to each pipe type is extracted to obtain a centerline network corresponding to the pipe network of each pipe type, wherein the line width of the centerline in the centerline network is one volume element, and there is a unique path between any two volume elements on the centerline.

[0112] In the embodiments of the present disclosure, for the aforementioned obtained pipe network, the centerline of the pipe network can be extracted by a centerline extraction algorithm, and finally the centerline network has equivalence in topological structure and positioning function with the pipe network. Thus, the network structure is simplified. The centerline network has the following characteristics: 1) the line width of the centerline is one volume element; 2) there is a unique and non-repeating path between any two voxels on the centerline network.

[0113] The above technical solution can segment the constructed three-dimensional model, extract the pipe network, and then determine the centerline network corresponding to the pipe network, so that the influence between different types of pipes in the image is reduced.

[0114] Optionally, referring to Figure 3 As shown in the figure, in step S12, the three-dimensional model is subjected to image segmentation to obtain a pipe network corresponding to each pipe type in the human pipe system, which includes:

[0115] In step S121, a target region including the suspected lesion area is demarcated in the three-dimensional model.

[0116] In the embodiments of the present disclosure, a three-dimensional region is demarcated from the space range in which the three-dimensional model is located as a target region, and the target region includes the lesion area of the patient, wherein the shape of the target region can be a cube, an oblique cube, a cylinder, an oblique cylinder, or other irregular three-dimensional bodies.

[0117] Further, in the three-dimensional model, other volume elements located outside the target region are added with a hidden label, and the volume elements marked with the hidden label are hidden when the three-dimensional model is presented, i.e., the volume elements marked with the hidden label are not displayed.

[0118] In step S122, according to the input exposure value range, the three-dimensional model in the target region is scanned for volume elements, and the volume elements corresponding to the target region are obtained.

[0119] It can be understood that the exposure value range includes a lower threshold and an upper threshold, the three-dimensional model in the target region is scanned for volume elements, and the volume elements in the target region whose exposure values fall outside the exposure value range are added with a hidden label, and then the volume elements in the target region marked with the hidden label are hidden.

[0120] In step S123, the input seed point is taken as a starting point, and according to the exposure value range, the volume elements in the three-dimensional model that exist in a connected relationship are searched for, and the volume elements that exist in a connected relationship are added with a display label.

[0121] In the embodiments of the present disclosure, the input seed point is input by the user in real time according to the surgical requirements, and in specific implementation, the volume elements that exist in a connected relationship can be added with a display label, and the volume elements that do not exist in a connected relationship can be added with a hidden label.

[0122] In step S124, the volume elements in the three-dimensional model that are not added with a display label are hidden, and the pipe network corresponding to the pipe system of each pipe type in the human pipe system is obtained.

[0123] In the embodiments of the present disclosure, the volume elements marked with the hidden label can be hidden, and the volume elements marked with the display label can be displayed, and the pipe network corresponding to the pipe system of each pipe type in the human pipe system is obtained.

[0124] Optionally, on the basis of Figure 2 , referring to Figure 4 , in step S2, the step of constructing a connectivity table of the centerline network according to the centerline network includes:

[0125] In step S21, according to the three-dimensional model, the three-dimensional coordinates of the network nodes in the centerline network and the edge length of the adjacent two network nodes along the centerline are determined.

[0126] It can be understood that the three-dimensional coordinates of the network nodes are the coordinates of the network nodes in the three-dimensional model, and the edge length of the adjacent two network nodes along the center line can generally represent the actual length of the pipeline, rather than the Euclidean distance between the adjacent two network nodes, that is, the edge length of the adjacent two network nodes along the center line is generally greater than or equal to the Euclidean distance between the adjacent two network nodes.

[0127] In step S22, a backup network node is selected from the center line network according to the three-dimensional coordinates and the edge length of the adjacent two network nodes along the center line, wherein the backup network node satisfies that the Euclidean distance with the adjacent network node is less than or equal to the preset distance threshold, and the maximum difference between the Euclidean distance with the adjacent network node and the edge length of the adjacent corresponding network node along the center line is less than or equal to the preset difference threshold.

[0128] In the embodiments of the present disclosure, the Euclidean distance of the adjacent network nodes can be calculated according to the three-dimensional coordinates of the adjacent two network nodes, in the case that the Euclidean distance of the adjacent network nodes is less than or equal to the preset distance threshold, the network node is discarded, and the Euclidean distance of the next network node and its adjacent network node is continued to be calculated.

[0129] Further, the maximum difference between the Euclidean distance of the adjacent network nodes and the edge length of the adjacent corresponding network node along the center line is calculated, in the case that the maximum difference is less than or equal to the preset difference threshold, the network node is taken as a backup network node. In the case that the maximum difference is greater than the preset difference threshold, the network node is discarded, and the Euclidean distance of the next network node and its adjacent network node is continued to be calculated.

[0130] In step S23, a connectivity table of the center line network is constructed according to the plurality of backup network nodes, wherein the plurality of backup network nodes include network nodes of a preset type, and the preset type includes a branch point type and a termination point type.

[0131] It can be understood that the shortest distance from any backup network node to other backup network nodes is indicated in the connectivity table, and the distance is generally greater than or equal to the Euclidean distance between the corresponding backup network nodes.

[0132] Optionally, in step S23, the step of constructing the connectivity table of the center line network according to the plurality of backup network nodes comprises:

[0133] For any two backup network nodes, it is determined from the center line network whether the two backup network nodes are adjacent network nodes.

[0134] In the embodiments of the present disclosure, determining whether two backup network nodes are adjacent network nodes in the center line network can be understood as whether the other backup network nodes are passed along the center line network from any one of the two backup network nodes to the other backup network node.

[0135] In the case that the two backup network nodes are adjacent network nodes, the Euclidean distance between the two backup network nodes is calculated according to the three-dimensional coordinates, and the two backup network nodes and the corresponding Euclidean distance are filled into the two-dimensional matrix.

[0136] In the case that the two backup network nodes are adjacent network nodes, that is, the other backup network nodes are not passed along the center line network from any one of the two backup network nodes to the other backup network node.

[0137] In the case that the two backup network nodes are not adjacent network nodes, the adjacent node searching is performed on any one of the two backup network nodes in a recursive manner until the found network node is the adjacent node of the other network node of the two backup network nodes, the Euclidean distances between the found network node and its adjacent network nodes are calculated, and the sum of the Euclidean distances is taken as the Euclidean distance of the two backup network nodes.

[0138] In the case that the two backup network nodes are not adjacent network nodes, that is, the other backup network nodes must be passed along the center line network from any one of the two backup network nodes to the other backup network node.

[0139] For example, when the backup network node i and the backup network node j are the same network node, the shortest distance of the backup network node i to itself is certainly 0, that is, m(i, i) = 0; when the backup network node i and the backup network node j are different network nodes, if the backup network node i and the backup network node j are two adjacent network nodes, the shortest distance of m(i, j) can be represented by the Euclidean distance between the two network nodes; if the backup network node i and the backup network node j are not two adjacent network nodes, m(i, j) can be divided into two parts, that is, m(i, j) = m(i, j-1) + m(j-1, j), wherein the backup network node j-1 is the previous node of the backup network node j in the center line network, and the backup network node j-1 and the backup network node j are adjacent network nodes, the shortest distance of m(j-1, j) is the Euclidean distance between the backup network node j-1 and the backup network node j, and m(i, j-1) can be further divided in a recursive manner, and finally divided to the backup network node x, which is the subsequent adjacent node of the backup network node i, and then the Euclidean distance of each pair of adjacent network nodes is calculated, and the sum of the multiple Euclidean distances is taken as the shortest distance of m(i, j).

[0140] Fill each two standby network nodes and the corresponding Euclidean distance into a two-dimensional matrix to construct the connectivity table of the centerline network.

[0141] Optionally, referring to Figure 5 As shown in the figure, in step S2, the step of determining the entry point of the endoscope pushing according to the centerline network comprises:

[0142] In step S201, the network nodes that must be passed through to reach the designated target checkpoint are determined from the centerline network.

[0143] In step S202, feature extraction is performed on the network nodes that must be passed through to obtain the node features of each network node that must be passed through.

[0144] In the embodiments of the present disclosure, the network nodes on the path that the endoscope must pass through to reach the suspected lesion area are found from the centerline network, and then if the pipeline area where the network node is located has at least one feature suitable for image recognition, the network node is taken as an entry point, wherein the feature suitable for image recognition can include at least one of a node structure feature, a node texture feature, and a node color feature. The feature of the entry point can be vectorized to obtain a feature descriptor of the entry point, and the vectorized feature descriptor is saved as an entry point feature standard value in the control parameter set.

[0145] In step S203, at least one entry point of the endoscope pushing is determined from the network nodes that must be passed through according to the node features.

[0146] Optionally, the endoscope pushing route planning method further comprises:

[0147] Feature extraction is performed on the pipeline area where the entry point is located, and feature vectorization is performed on the extracted entry point features to obtain an entry point feature standard value.

[0148] Feature detection is performed on the endoscope image collected by the endoscope, and feature vectorization is performed on the detected image features to obtain an image feature value.

[0149] The entry point feature standard value and the image feature value are compared.

[0150] If the entry point feature standard value and the image feature value are successfully compared, a synchronization relationship between the endoscope pushing and the pushing route is established.

[0151] The surgical guidance is displayed according to the pushing route.

[0152] In a specific implementation, the endoscope is pushed to a vicinity of an entry point on the centerline network according to a surgical plan using a conventional operation method; then the synchronization function of the endoscope navigation software is started to perform feature detection on the real-time image output by the endoscope, and the feature detection result is vectorized to obtain an image feature value, which is then compared with an entry point feature standard value stored in the control parameter set in real time, and a comparison result is output. The comparison result can be presented in the form of a score of vector matching.

[0153] With the advancement of the endoscope, if the matching score of the image feature value corresponding to a frame of image and the entry point feature standard value reaches a preset score, the comparison of the entry point feature standard value and the image feature value is successful, and it can be determined that the probe of the endoscope reaches the entry point position. The image captured by the endoscope establishes a synchronization relationship with the corresponding position in the centerline network, and the system can enter the intraoperative navigation and advancement stage. The surgical guidance is displayed according to the advancement route.

[0154] Based on the same concept, the disclosure also provides an endoscope advancement route planning device, as shown in Figure 6 The endoscope advancement route planning device includes a network establishment module 610, a first determination module 620, a second determination module 630, and a route planning module 640.

[0155] The network establishment module 610 is configured to establish a centerline network corresponding to a suspected lesion area according to an original medical image of the suspected lesion area to be examined;

[0156] The first determination module 620 is configured to determine an entry point for endoscope advancement and construct a connectivity table of the centerline network according to the centerline network, wherein the connectivity is used to represent the reachability between any two network nodes in the centerline network;

[0157] The second determination module 630 is configured to determine an advancement node from the entry point to a specified target examination point by traversing the centerline network according to the connectivity table;

[0158] The route planning module 640 is configured to plan an advancement route of the endoscope according to the advancement node.

[0159] Optionally, the network establishment module 610 is configured to:

[0160] construct a three-dimensional model for the suspected lesion area according to the original medical image of the suspected lesion area to be examined;

[0161] perform image segmentation on the three-dimensional model to obtain a pipeline network corresponding to each pipeline type in the human pipeline system;

[0162] The center line extraction is performed on the pipeline network corresponding to each pipeline type to obtain a center line network corresponding to the pipeline network of each pipeline type, wherein the line width of the center line in the center line network is one volume element, and there is a unique path between any two volume elements on the center line.

[0163] Optionally, the network establishing module 610 is configured to:

[0164] The target region including the suspected lesion region is demarcated in the three-dimensional model;

[0165] According to the input exposure value range, the volume elements in the three-dimensional model in the target region are scanned to obtain the volume elements corresponding to the target region;

[0166] Taking the input seed point as the starting point, the volume elements having a connection relationship in the three-dimensional model are searched according to the exposure value range, and the volume elements having the connection relationship are added with display labels;

[0167] The volume elements in the three-dimensional model which are not added with display labels are hidden to obtain the pipeline network corresponding to the pipeline system of each pipeline type in the human body pipeline system.

[0168] Optionally, the first determining module 620 is configured to:

[0169] According to the three-dimensional model, the three-dimensional coordinates of the network nodes in the center line network and the edge length of the adjacent two network nodes along the center line are determined;

[0170] According to the three-dimensional coordinates and the edge length of the adjacent two network nodes along the center line, a backup network node is selected from the center line network, wherein the backup network node satisfies that the Euclidean distance with the adjacent network node is less than or equal to a preset distance threshold, and the maximum difference between the Euclidean distance with the adjacent network node and the edge length of the adjacent corresponding network node along the center line is less than or equal to a preset difference threshold;

[0171] According to the plurality of backup network nodes, a connectivity table of the center line network is constructed, wherein the plurality of backup network nodes include network nodes of a preset type, and the preset type includes a bifurcation point type and a termination point type.

[0172] Optionally, the first determining module 620 is configured to:

[0173] For any two backup network nodes, it is determined whether the two backup network nodes are adjacent network nodes from the center line network;

[0174] In the case that the two backup network nodes are adjacent network nodes, the Euclidean distance between the two backup network nodes is calculated according to the three-dimensional coordinates, and the two backup network nodes and the corresponding Euclidean distance are filled into a two-dimensional matrix;

[0175] In the case that the two standby network nodes are not adjacent network nodes, recursively performing adjacent node lookup on any one of the two standby network nodes until a network node found is an adjacent node of the other network node of the two standby network nodes, calculating the Euclidean distance between the found network node and its adjacent network nodes, and taking the sum of the Euclidean distances as the Euclidean distance of the two standby network nodes;

[0176] Filling the two standby network nodes and the corresponding Euclidean distances into a two-dimensional matrix to construct a connectivity table of the centerline network.

[0177] Optionally, the first determining module 620 is configured to:

[0178] Determining, from the centerline network, network nodes that must be passed through to reach a specified target checkpoint;

[0179] Performing feature extraction on the network nodes that must be passed through to obtain node features of each network node that must be passed through;

[0180] Determining, according to the node features, at least one entry point for endoscope advancement from the network nodes that must be passed through.

[0181] Optionally, the endoscope advancement route planning device 600 further comprises a guiding module configured to:

[0182] Performing feature extraction on a pipe region where the entry point is located, and performing feature vectorization on the extracted entry point features to obtain an entry point feature standard value;

[0183] Performing feature detection on endoscope images collected by the endoscope, and performing feature vectorization on the detected image features to obtain image feature values;

[0184] Comparing the entry point feature standard value with the image feature values;

[0185] If the comparison of the entry point feature standard value and the image feature values is successful, establishing a synchronization relationship between the endoscope advancement and the advancement route;

[0186] Displaying surgical guidance according to the advancement route.

[0187] As to the endoscope advancement route planning device 600 in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be described in detail here.

[0188] Those skilled in the art should understand that the above-described endoscope advancing route planning device 600 embodiment is only illustrative, for example, the division of modules is only a logical functional division, and other divisions can be used in actual implementation, for example, multiple modules can be combined or integrated into one module. In addition, the modules described as separate components can or can not be physically separated. Moreover, each module can be implemented entirely or partially by software, hardware, firmware, or any combination thereof. When implemented by software, it can be implemented entirely or partially in the form of a computer program product. When implemented by hardware, it can be implemented entirely or partially in the form of an integrated circuit or a chip.

[0189] The embodiments of the present disclosure also provide a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a first processor, implements the steps of the endoscope advancing route planning method of any one of the first aspect.

[0190] The embodiments of the present disclosure also provide an electronic device, comprising:

[0191] a first memory having a computer program stored thereon;

[0192] a second processor configured to execute the computer program in the first memory to implement the steps of the endoscope advancing route planning method of any one of the first aspect.

[0193] Figure 7 is a block diagram of an electronic device 600 according to an exemplary embodiment. The electronic device 600 can be configured as an endoscope advancing route planning device, as shown in Figure 7 The electronic device 600 can include a third processor 601, a second memory 602. The electronic device 600 can also include one or more of a multimedia component 603, an input / output (I / O) interface 604, and a communication component 605.

[0194] The third processor 601 is configured to control the overall operation of the electronic device 600 to complete all or part of the steps of the endoscope pushing route planning method described above. The second memory 602 is configured to store various types of data to support the operation of the electronic device 600, which can include, for example, instructions for any application or method operating on the electronic device 600, and application-related data, such as pictures, etc. The second memory 602 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The multimedia component 603 can include a screen and an audio component. The screen can be a touch screen, for example, and the audio component is configured to output and / or input audio signals. For example, the audio component can include a microphone configured to receive external audio signals. The received audio signals can be further stored in the second memory 602 or transmitted through the communication component 605. The audio component also includes at least one speaker configured to output audio signals. The I / O interface 604 provides an interface between the third processor 601 and other interface modules, which can be a keyboard, a mouse, a button, etc. These buttons can be virtual buttons or physical buttons, which are used to operate the endoscope image saving and display. The communication component 605 is configured to perform wired or wireless communication between the electronic device 600 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, near field communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited herein. Therefore, the corresponding communication component 605 can include a Wi-Fi module, a Bluetooth module, an NFC module, etc.

[0195] In an exemplary embodiment, the electronic device 600 can be implemented by one or more Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic elements for performing the above-mentioned endoscope advancing route planning method.

[0196] In another exemplary embodiment, a computer readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the above-mentioned endoscope advancing route planning method. For example, the computer readable storage medium can be the above-mentioned second memory 602 including program instructions, and the above-mentioned program instructions can be executed by the third processor 601 of the electronic device 600 to complete the above-mentioned endoscope advancing route planning method.

[0197] The preferred embodiments of the present disclosure are described in detail above with reference to the accompanying drawings, but the present disclosure is not limited to the specific details in the above-described embodiments. Various simple modifications can be made to the technical solutions of the present disclosure within the technical concept of the present disclosure, and these simple modifications all belong to the protection scope of the present disclosure.

[0198] In addition, it should be noted that each specific technical feature described in the above specific embodiments can be combined in any appropriate manner without contradiction. In order to avoid unnecessary repetition, various possible combinations are not described again in the present disclosure.

[0199] In addition, any combination of various different embodiments of the present disclosure can also be made, as long as it does not deviate from the idea of the present disclosure, and it should also be considered as disclosed in the present disclosure.

Claims

1. An endoscope advancement route planning method characterized by, The method comprises the steps of: establishing a center line network corresponding to a suspected lesion area to be examined according to an original medical image of the suspected lesion area; determining an entry point of the endoscope and constructing a connectivity table of the center line network according to the center line network, wherein the connectivity is used to represent the reachability between any two network nodes in the center line network, and the connectivity table is a two-dimensional matrix composed of a group of network nodes, and the two-dimensional matrix is characterized in that: a matrix element m(i, j) represents the shortest distance between any two network nodes i and j, and the connectivity table is used to record the shortest distance along the center line network between the two network nodes in the center line network; determining a pushing node from the entry point to a specified target examination point by traversing the center line network according to the connectivity table; planning the pushing route of the endoscope according to the pushing node.

2. The endoscope advancement route planning method according to claim 1, characterized by, The step of establishing a center line network corresponding to a suspected lesion area to be examined according to an original medical image of the suspected lesion area comprises the steps of: constructing a three-dimensional model for the suspected lesion area according to the original medical image of the suspected lesion area to be examined; performing image segmentation on the three-dimensional model to obtain a pipe network corresponding to a pipe system of each pipe type; extracting a center line from each pipe network corresponding to a pipe type to obtain a center line network corresponding to the pipe network of each pipe type, wherein the line width of the center line in the center line network is one volume element, and there is a unique path between any two volume elements on the center line.

3. The endoscope advancement route planning method according to claim 2, characterized by, The step of performing image segmentation on the three-dimensional model to obtain a pipe network corresponding to a pipe system of each pipe type comprises the steps of: defining a target region including the suspected lesion area in the three-dimensional model; performing volume element scanning on the three-dimensional model in the target region according to an input exposure value range to obtain a volume element corresponding to the target region; searching for volume elements having a connection relationship in the three-dimensional model according to the exposure value range and adding a display label to the volume elements having the connection relationship, taking an input seed point as a starting point; hiding volume elements in the three-dimensional model that have not been added with a display label to obtain a pipe network corresponding to a pipe system of each pipe type.

4. The endoscope advancement route planning method according to claim 2, characterized by, The step of constructing a connectivity table of the center line network according to the center line network comprises the steps of: determining three-dimensional coordinates of the network nodes in the center line network and the edge length of adjacent two network nodes along the center line according to the three-dimensional model; selecting a standby network node from the center line network according to the three-dimensional coordinates and the edge length of adjacent two network nodes along the center line, wherein the standby network node satisfies that the Euclidean distance with the adjacent network node is less than or equal to a preset distance threshold, and the maximum difference between the Euclidean distance with the adjacent network node and the edge length of the adjacent corresponding network node along the center line is less than or equal to a preset difference threshold. According to a plurality of the backup network nodes, a connectivity table of the centerline network is constructed, wherein the plurality of the backup network nodes include preset types of network nodes, and the preset types include a branch point type and a termination point type.

5. The endoscope advancement route planning method according to claim 4, characterized by, The step of constructing the connectivity table of the centerline network according to a plurality of the backup network nodes includes: For any two backup network nodes, it is determined from the centerline network whether the two backup network nodes are adjacent network nodes; In the case that the two backup network nodes are adjacent network nodes, a Euclidean distance between the two backup network nodes is calculated according to the three-dimensional coordinates, and the two backup network nodes and the corresponding Euclidean distance are filled into a two-dimensional matrix; In the case that the two backup network nodes are not adjacent network nodes, in a recursive manner, adjacent node searching is performed on any network node of the two backup network nodes until the found network node is an adjacent node of the other network node of the two backup network nodes, the Euclidean distances between the found network node and its adjacent network nodes are calculated, and the sum of the Euclidean distances is taken as the Euclidean distance of the two backup network nodes; The two backup network nodes and the corresponding Euclidean distance are filled into a two-dimensional matrix, and the connectivity table of the centerline network is constructed.

6. The endoscope push routing method of claim 1, wherein, According to the centerline network, the step of determining the entry point of the endoscope pushing includes: Determining the necessary network nodes to reach the specified target checkpoint from the centerline network; Feature extraction is performed on the necessary network nodes to obtain node features of each necessary network node; According to the node features, at least one entry point of the endoscope pushing is determined from the necessary network nodes.

7. The endoscope push route planning method according to any one of claims 1-6, characterized by, The endoscope pushing route planning method further includes: Feature extraction is performed on the pipeline region where the entry point is located, and the extracted entry point features are vectorized to obtain entry point feature standard values; Feature detection is performed on the endoscope images collected by the endoscope, and the detected image features are vectorized to obtain image feature values; The entry point feature standard values are compared with the image feature values; If the entry point feature standard values and the image feature values are successfully compared, a synchronization relationship between the endoscope pushing and the pushing route is established; The operation guide is displayed according to the pushing route.

8. An endoscope advancement route planning apparatus characterized by comprising: It includes: A network establishment module configured to establish a centerline network corresponding to a suspected lesion area to be examined according to original medical images of the suspected lesion area; a first determining module configured to determine an entry point of the endoscope and construct a connectivity table of the centerline network according to the centerline network, wherein the connectivity is used to represent the reachability between any two network nodes in the centerline network, and the connectivity table is a two-dimensional matrix composed of a group of network nodes, and the two-dimensional matrix is characterized in that: a matrix element m(i, j) represents the shortest distance between any two network nodes i and j, and the connectivity table is used to record the shortest distance along the centerline network between the two network nodes in the centerline network; a second determining module configured to determine a pushing node from the entry point to a specified target checkpoint by traversing the centerline network according to the connectivity table; a route planning module configured to plan the pushing route of the endoscope according to the pushing node.

9. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the first processor to implement the steps of the endoscope pushing route planning method in any one of claims 1-7.

10. An electronic device, comprising: comprise: a first memory having a computer program stored thereon; a second processor configured to execute the computer program in the first memory to implement the steps of the endoscope pushing route planning method in any one of claims 1-7.

Images