Path planning method and device, computer device and readable storage medium

Abstract

The present disclosure relates to the technical field of endoscopy, and provides a path planning method. In the method, a cross-sectional image of a target object and hardware parameters of a bronchoscope robot corresponding to the target object are obtained; according to the cross-sectional image of the target object, lesion feature information, trachea feature information and blood vessel feature information of the target object are determined; according to the hardware parameters of the bronchoscope robot, and the lesion feature information, the trachea feature information and the blood vessel feature information of the target object, an airway wall puncture biopsy path of the bronchoscope robot is determined. In this way, since multiple constraint conditions are considered in the generation process of the airway wall puncture biopsy path, the most efficient and safe airway wall puncture biopsy path can be obtained, thereby the problems of subjectivity and human error that occur when a scheme is artificially formulated can be avoided, and thus the airway wall puncture biopsy path obtained by the embodiment is more accurate and has more reference value, and the safety and efficiency of the puncture biopsy are also improved.

Description

Technical Field

[0001] This disclosure relates to the field of endoscopy technology, and more particularly to a path planning method, apparatus, computer device, and computer-readable storage medium. Background Technology

[0002] Endoscopic examination is playing an increasingly important role in the early detection and treatment of tumors. Conventional endoscopy allows access to natural body cavities to obtain images from the endoscope tip for observation, and biopsy tools are used for forceps, brushing, and needle aspiration of living tissue, thus enabling accurate diagnosis. However, traditional bronchial catheters have a large outer diameter, making them difficult to access into the higher-order peripheral bronchi of the lung, and their bending control is generally limited, requiring considerable skill and experience from the operator. More than 40% of all pulmonary nodules do not have direct airway access. For these nodules requiring biopsy, percutaneous puncture or trans-airway puncture is typically used. Percutaneous puncture carries serious risks such as pneumothorax and infection; therefore, from a safety perspective, trans-airway intervention is clinically recommended. This involves pre-operative path planning to reach the optimal puncture point within the airway, performing the puncture, and then traversing a section of lung parenchyma to reach the target nodule for sampling. However, since blood vessels usually accompany each level of the trachea, there is a risk of vascular damage during the puncture process.

[0003] Current preoperative airway wall puncture biopsy path planning mainly relies on manual assessment based on subjective experience. However, this manual assessment method has certain subjectivity, bias, and limitations, which leads to poor accuracy in the existing methods of manually determining the airway wall puncture biopsy path. There is a risk of "blind spot puncture," which may result in damage to blood vessels. Therefore, the current airway wall puncture biopsy path planning method is difficult to obtain the optimal puncture path. Summary of the Invention

[0004] In view of this, the present disclosure provides a path planning method, apparatus, computer device, and computer-readable storage medium to solve the problem that the accuracy of the prior art in determining the airway wall puncture biopsy path is poor, leading to the risk of damaging blood vessels.

[0005] A first aspect of this disclosure provides a path planning method, the method comprising:

[0006] Obtain cross-sectional images of the target object and the hardware parameters of the bronchoscopic robot corresponding to the target object;

[0007] Based on the cross-sectional image of the target object, determine the lesion feature information, tracheal feature information, and vascular feature information of the target object;

[0008] Based on the hardware parameters of the bronchoscopic robot, as well as the lesion characteristics, tracheal characteristics, and vascular characteristics of the target object, the airway wall puncture biopsy path of the bronchoscopic robot is determined.

[0009] A second aspect of this disclosure provides a path planning apparatus, the apparatus comprising:

[0010] The data acquisition unit is used to acquire cross-sectional images of the target object and the hardware parameters of the bronchoscopic robot corresponding to the target object.

[0011] The information determination unit is used to determine the lesion feature information, tracheal feature information and vascular feature information of the target object based on the cross-sectional image of the target object;

[0012] The path determination unit is used to determine the airway wall puncture biopsy path of the bronchoscopy robot based on the hardware parameters of the bronchoscopy robot and the lesion feature information, tracheal feature information, and vascular feature information of the target object.

[0013] A third aspect of this disclosure provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0014] A fourth aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0015] The beneficial effects of this disclosure compared to the prior art are as follows: The method disclosed in this disclosure can first acquire a cross-sectional image of the target object and the hardware parameters of the bronchoscope robot corresponding to the target object; then, based on the cross-sectional image of the target object, the lesion feature information, tracheal feature information, and vascular feature information of the target object can be determined; next, based on the hardware parameters of the bronchoscope robot and the lesion feature information, tracheal feature information, and vascular feature information of the target object, the airway wall puncture biopsy path of the bronchoscope robot can be determined. It is evident that this embodiment considers not only the lesion feature information of the target object, but also the hardware parameters of the bronchoscope robot, the tracheal feature information, and the vascular feature information of the target object in determining the airway wall puncture biopsy path of the bronchoscope robot. Thus, compared to the traditional method of manually determining the airway wall puncture biopsy path, the airway wall puncture biopsy path planning method using the bronchoscope robot can greatly reduce the risk of "blind spot puncture" and avoid damage to blood vessels; furthermore, with the help of machine control, more precise puncture operations can be obtained, ensuring that the puncture operation is perfectly executed according to the planned scheme. Furthermore, since multiple constraints (i.e., lesion characteristics of the target object, hardware parameters of the bronchoscopic robot, tracheal characteristics of the target object, and vascular characteristics) are considered during the generation of the airway wall puncture biopsy path, the most efficient and safest airway wall puncture biopsy path can be obtained. This avoids the subjectivity and human error that occur when manually formulating a plan, thus making the airway wall puncture biopsy path of the bronchoscopic robot obtained through this embodiment more accurate and more valuable for reference, and also improving the safety and efficiency of the puncture biopsy procedure. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram illustrating an application scenario of an embodiment of this disclosure;

[0018] Figure 2 This is a flowchart of a path planning method provided in an embodiment of this disclosure;

[0019] Figure 3 This is a block diagram of the path planning device provided in the embodiments of this disclosure;

[0020] Figure 4 This is a schematic diagram of a computer device provided in an embodiment of this disclosure. Detailed Implementation

[0021] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, so as to provide a thorough understanding of the embodiments of this disclosure. However, those skilled in the art will understand that this disclosure may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this disclosure with unnecessary detail.

[0022] A path planning method and apparatus according to embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings.

[0023] In existing technologies, the planning of preoperative airway wall puncture biopsy pathways mainly relies on manual assessment based on subjective experience. This manual assessment method has certain subjectivity, bias and limitations, which leads to poor accuracy in the existing manual determination of airway wall puncture biopsy pathways. There is a risk of "blind puncture", which may result in damage to blood vessels. Therefore, the existing airway wall puncture biopsy pathway planning methods are difficult to obtain the optimal puncture path.

[0024] To address the aforementioned problems, this invention provides a path planning method. In this method, the determination of the airway wall puncture biopsy path for a bronchoscopic robot considers not only the lesion characteristics of the target object but also the hardware parameters of the bronchoscopic robot, the tracheal characteristics of the target object, and the vascular characteristics. Therefore, compared to the traditional method of manually determining the airway wall puncture biopsy path, this method using a bronchoscopic robot significantly reduces the risk of "blind spot puncture" and avoids vascular damage. Furthermore, machine control enables more precise puncture operations, ensuring perfect execution of the planned procedure. Moreover, because multiple constraints are considered during the generation of the airway wall puncture biopsy path, the most efficient and safest path can be obtained, thus avoiding the subjectivity and human error inherent in manually formulated plans. Consequently, the airway wall puncture biopsy path obtained through this embodiment is more accurate and valuable, improving the safety and efficiency of the puncture biopsy procedure.

[0025] For example, embodiments of the present invention can be applied to, for example... Figure 1 The application scenario shown can include a bronchoscopy robot 1, a server 2, and a client 3.

[0026] Client 3 can be either hardware or software. When client 3 is hardware, it can be various electronic devices with a display screen that support communication with server 2, including but not limited to smartphones, tablets, laptops, and desktop computers; when client 3 is software, it can be installed in the aforementioned electronic devices. Client 3 can be implemented as multiple software programs or software modules, or as a single software program or software module; this disclosure does not limit this. Furthermore, various applications can be installed on client 3, such as route planning applications, instant messaging tools, social media platform software, search applications, shopping applications, etc.

[0027] Server 2 can be a path planning server that provides various services, such as a background path planning server that receives requests sent by terminal devices with which it has established communication connections. This background path planning server can receive and analyze the requests sent by the terminal devices (e.g., the bronchoscopic robot 1 and the client 3) and generate processing results. Server 2 can be a single path planning server, a path planning server cluster consisting of several path planning servers, or a cloud computing service center. This embodiment of the disclosure does not limit this.

[0028] It should be noted that server 2 can be either hardware or software. When server 2 is hardware, it can be various electronic devices that provide various services to the bronchoscopy robot 1 and the client 3. When server 2 is software, it can be multiple software programs or software modules that provide various services to the bronchoscopy robot 1 and the client 3, or it can be a single software program or software module that provides various services to the bronchoscopy robot 1 and the client 3. This disclosure does not limit the scope of the embodiments.

[0029] In one implementation, the bronchoscopy robot 1 may include: a robotic arm lifting platform 40, a robotic arm lifting base 410, a robotic arm 30, an end effector 20, and an end tool 10. The end effector 20 can be used to control the end tool 10; the end tool 10 can be a bronchoscopic catheter with an end-effector, a light source, a positioning sensor, and a working channel, or it can be a tool used for biopsy sampling or treatment. The bronchoscopy robot 1 may include one or more robotic arms, end effectors, and end tools. In one implementation, multiple robotic arms can be integrated into the same robotic arm lifting platform. One or two end effectors can be fixed to the end of one robotic arm. One effector can be used to control the bending, advancement, sampling, or other treatment-related operations of an end tool. Furthermore, the combination of multiple end tools can form specific lung diagnosis and treatment plans; for example, a tool combination of one bronchoscopic catheter and one biopsy tool can complete a bronchoscopic robotic biopsy sampling procedure.

[0030] The bronchoscopy robot 1, client 3, and server 2 can communicate via a network. The network can be a wired network using coaxial cable, twisted pair, or fiber optic connection, or a wireless network that enables interconnection of various communication devices without wiring, such as Bluetooth, Near Field Communication (NFC), or Infrared. This disclosure does not limit the scope of the embodiments.

[0031] Specifically, server 2 can obtain the cross-sectional image of the target object sent by client 3 and the hardware parameters of the bronchoscope robot corresponding to the target object sent by bronchoscope robot 1; server 2 can determine the lesion feature information, tracheal feature information and vascular feature information of the target object based on the cross-sectional image of the target object; server 2 can determine the airway wall puncture biopsy path of the bronchoscope robot based on the hardware parameters of the bronchoscope robot and the lesion feature information, tracheal feature information and vascular feature information of the target object. Thus, in determining the airway wall puncture biopsy path for the bronchoscopic robot, this embodiment considers not only the lesion characteristics of the target object, but also the hardware parameters of the bronchoscopic robot, the tracheal characteristics of the target object, and the vascular characteristics. Compared to the traditional method of manually determining the airway wall puncture biopsy path, the airway wall puncture biopsy path planning method using the bronchoscopic robot can greatly reduce the risk of "blind spot puncture" and avoid vascular damage. Furthermore, machine control enables more precise puncture operations, allowing the puncture operation to be perfectly executed according to the planned scheme. Moreover, because multiple constraints are considered in the generation of the airway wall puncture biopsy path, the most efficient and safest airway wall puncture biopsy path can be obtained, thereby avoiding the subjectivity and human error problems that occur when manually formulating a scheme. Therefore, the airway wall puncture biopsy path obtained by the bronchoscopic robot in this embodiment is more accurate and has more reference value, and it also improves the safety and efficiency of puncture biopsy.

[0032] It should be noted that the specific type, quantity and combination of the bronchoscopy robot 1, server 2, user client 3 and network can be adjusted according to the actual needs of the application scenario, and this disclosure embodiment does not limit this.

[0033] It should be noted that the above application scenarios are shown only for the purpose of understanding this disclosure, and the implementation of this disclosure is not limited in any way. On the contrary, the implementation of this disclosure can be applied to any applicable scenario.

[0034] Figure 2 This is a flowchart of a path planning method provided in an embodiment of this disclosure. Figure 2One path planning method can be derived from Figure 1 The server executes this. For example... Figure 2 As shown, the path planning method includes:

[0035] S201: Obtain the cross-sectional image of the target object and the hardware parameters of the bronchoscopic robot corresponding to the target object.

[0036] In this embodiment, the target object can be understood as the part of the human body that needs to be subjected to a puncture biopsy, such as the lungs or other organs with similar structures. The cross-sectional image of the target object can be CT data (such as CT images) acquired using CT (Computed Tomography) equipment.

[0037] The bronchoscopic robot corresponding to the target object can be understood as a robot used to perform puncture biopsy on the target object. The hardware parameters of the bronchoscopic robot can be understood as the parameter information of each component in the bronchoscopic robot. For example, the hardware parameters of the bronchoscopic robot may include parameters such as robot stroke size, catheter outer diameter, catheter tip bending angle, biopsy tool controllable bending angle, biopsy tool extension length, catheter controllable bending angle, adapter forceps channel size, wall-breaking tool type, and wall-breaking tool controllable bending angle.

[0038] S202: Based on the cross-sectional image of the target object, determine the lesion feature information, tracheal feature information, and vascular feature information of the target object.

[0039] Once a cross-sectional image of the target object is obtained, data extraction can be performed on the cross-sectional image to obtain the lesion feature information, tracheal feature information, and vascular feature information of the target object. The lesion feature information of the target object can be understood as the characteristics of the lesion area within the target object. In this embodiment, the lesion area can be defined as a single connected closed three-dimensional region of arbitrary shape, such as a regular-shaped single connected closed three-dimensional region, like an ellipsoid, or it can be defined as a single connected closed three-dimensional region of any irregular shape. For example, the lesion feature information of the target object can include features such as the size, location, and shape of the lesion. The tracheal feature information of the target object can be understood as the characteristics of each trachea within the target object. For example, the tracheal feature information of the target object can include features such as the size, location, shape, airway pathway, airway diameter, and airway topology of each trachea. The vascular feature information of the target object can be understood as the characteristics of each blood vessel within the target object. For example, the vascular feature information of the target object can include features such as the vessel type, size, location, shape, vascular pathway, vessel diameter, and vascular topology of each blood vessel.

[0040] In this embodiment, there are multiple ways to extract data from the cross-sectional image of the target object. For example, the lesion feature information, tracheal feature information and vascular feature information of the target object can be determined based on CT images; or the lesion feature information, tracheal feature information and vascular feature information of the target object can be determined based on the three-dimensional model corresponding to the CT data.

[0041] Next, the method for determining the lesion feature information, tracheal feature information, and vascular feature information of the target object based on the three-dimensional model corresponding to CT data will be described. In this embodiment, the cross-sectional image can be a CT image; step S202, "determining the lesion feature information, tracheal feature information, and vascular feature information of the target object based on the cross-sectional image of the target object," may include the following steps:

[0042] S202a: Reconstruct the image based on the CT image to obtain a three-dimensional model corresponding to the CT image.

[0043] S202b: Based on the three-dimensional model, determine the lesion feature information, tracheal feature information, and vascular feature information of the target object.

[0044] In this embodiment, after acquiring CT images, a corresponding 3D model can be built based on the CT images. This allows for the acquisition of 3D models corresponding to lesions, trachea, and blood vessels within the target object. For example, the 3D model can be segmented to determine the lesion region (i.e., the lesion), trachea region, and blood vessel region on the diseased organ. Therefore, the lesion feature information of the target object can be determined based on the 3D model corresponding to the lesion, the trachea feature information based on the 3D model corresponding to the trachea, and the blood vessel feature information based on the 3D model corresponding to the blood vessels.

[0045] It should be noted that, in one implementation of this embodiment, after determining the lesion feature information, tracheal feature information, and vascular feature information of the target object, these features can be verified. For example, it can be determined whether the determined lesion feature information, tracheal feature information, and vascular feature information are abnormal. For instance, it can be checked whether the airway diameter meets the requirements for the outer diameter of the catheter, whether the blood vessels around the lesion are abnormal, and whether the size of the lesion meets the requirements for biopsy (e.g., lesion diameter > 1 cm). If the verification of the lesion feature information, tracheal feature information, and vascular feature information of the target object passes, the subsequent steps can be continued. If the verification fails, S202 is continued until the verification of the lesion feature information, tracheal feature information, and vascular feature information of the target object passes.

[0046] S203: Based on the hardware parameters of the bronchoscopic robot, and the lesion characteristics, tracheal characteristics, and vascular characteristics of the target object, determine the airway wall puncture biopsy path of the bronchoscopic robot.

[0047] In this embodiment, after obtaining the hardware parameters of the bronchoscopic robot and the lesion feature information, tracheal feature information, and vascular feature information of the target object, the airway wall puncture biopsy path of the bronchoscopic robot can be determined using these parameters. The airway wall puncture biopsy path of the bronchoscopic robot can be understood as the movement path of the bronchoscopic robot within the target object. That is, the bronchoscopic robot can use this airway wall puncture biopsy path to deliver a sampling tool equipped with a positioning sensor into the natural airway leading to the lesion area (i.e., the lesion area). It should be understood that in this embodiment, the airway wall puncture biopsy path refers to the puncture path within the airway, where the puncture mentioned only refers to breaking through the airway wall.

[0048] As an example, S203 may include the following steps:

[0049] S203a: Determine the lesion puncture sampling point based on the lesion feature information of the target object and the tracheal feature information.

[0050] In this embodiment, the centroid of the lesion on the target object can be determined first based on the lesion feature information of the target object. For example, the centroid of the lesion can be calculated based on the three-dimensional spatial shape of the lesion. Specifically, the centroid of the lesion on the target object can be determined based on the size, location, shape, and other features of the lesion.

[0051] Then, based on the centroid of the lesion and the tracheal feature information, the target trachea closest to the centroid of the lesion can be determined. For example, the distance between each trachea and the centroid of the lesion can be determined first based on the position of each trachea and the position of the centroid of the lesion, and the trachea with the smallest distance can be selected as the target trachea.

[0052] Next, the initial lesion puncture and sampling location can be determined based on the centroid of the lesion and the target trachea. For example, the initial lesion puncture and sampling location can be selected where the line connecting the centroid and the target trachea passes through the boundary of the lesion.

[0053] If the initial lesion puncture sampling location meets the first preset condition, then the initial lesion puncture sampling location is used as the lesion puncture sampling point. If the initial lesion puncture sampling location does not meet the first preset condition, then the initial lesion puncture sampling location is adjusted until the adjusted puncture sampling location meets the first preset condition, and the puncture sampling location that meets the first preset condition is used as the lesion puncture sampling point. In one implementation, the first preset condition may be that the path between the lesion puncture sampling point and the target trachea does not pass through or traverse blood vessels, for example, it does not pass through or traverse any blood vessels or arteries or veins.

[0054] S203b: Determine the airway puncture point based on the hardware parameters of the bronchoscopic robot and the lesion puncture sampling location.

[0055] In this embodiment, the hardware parameters of the bronchoscopic robot include at least the outer diameter parameter of the endotracheal tube. As an example, a tracheal model accessible by the endotracheal tube can be generated first based on the outer diameter parameter, and the endpoints and centerlines of the tracheal model can be determined. The tracheal model accessible by the endotracheal tube can be understood as a three-dimensional model of the trachea through which the endotracheal tube can pass, and the endpoints and centerlines of each tracheal model. It is understood that the endpoint of the tracheal model refers to the end of a branch airway in the reconstructed three-dimensional tracheal tree model.

[0056] Then, among all the endpoints and centerlines of the tracheal model, the initial airway puncture point closest to the lesion puncture sampling location can be determined. For example, based on the positions of the endpoints and centerlines of each tracheal model and the position of the lesion puncture sampling location, an airway location point on the tracheal centerline that can form a reasonable puncture angle with the lesion can be determined, i.e., the starting point of the puncture tool in the airway. The intersection of the line connecting this starting point and the lesion puncture sampling location point with the airway wall is the initial airway puncture point.

[0057] If the initial airway puncture point meets the second preset condition, then the initial airway puncture point is used as the airway puncture point. It should be noted that the second preset condition can be understood as the path corresponding to the initial airway puncture point satisfying constraints such as the maximum controllable bending angle of the robot's tubing, the tubing's outer diameter, the length of the wall-breaking tool, and the controllable bending angle of the wall-breaking tool. If the initial airway puncture point does not meet the second preset condition, the initial airway puncture point can be adjusted according to the hardware parameters of the bronchoscopic robot. For example, the position of the initial airway puncture point can be fine-tuned according to the hardware parameters of the bronchoscopic robot to obtain an adjusted airway puncture point; until the adjusted airway puncture point meets the second preset condition, and the adjusted airway puncture point that meets the second preset condition is used as the airway puncture point.

[0058] S203c: Determine the airway wall puncture biopsy path of the bronchoscopic robot based on the lesion puncture sampling point, the airway puncture point, and the tracheal feature information.

[0059] As an example, several airway wall puncture biopsy paths can be determined based on the lesion puncture sampling point and the airway puncture point. For example, the airway puncture point can be used as the starting point of the path and the lesion puncture sampling point as the ending point of the path to determine multiple airway wall puncture biopsy paths in the target object.

[0060] Then, based on the tracheal feature information, the several airway wall puncture biopsy paths can be screened to obtain the target airway wall puncture biopsy path. In one implementation, for each airway wall puncture biopsy path, the blood vessels traversed or passed through by the path can be analyzed, and the path risk level can be determined based on the blood vessels traversed or passed through. For example, the path risk level value can be calculated based on the weights corresponding to the size, type, and number of blood vessels traversed or passed through by the path. Different blood vessel sizes correspond to different weight values. For example, the smaller the blood vessel size, the lower its corresponding weight value, and vice versa. That is, if the blood vessel size is less than a threshold, the weight corresponding to the blood vessel is low; if the blood vessel size is greater than a threshold, the weight corresponding to the blood vessel is high; and the weight of the tissue surrounding blood vessels larger than the threshold is low. Understandably, the fewer blood vessels a puncture biopsy path passes through and the smaller the vessel size (e.g., vessel diameter), the lower the risk level of that path. Conversely, the more blood vessels a puncture biopsy path passes through and the larger the vessel size (e.g., vessel diameter), the higher the risk level. Finally, the puncture biopsy path with the lowest risk can be selected as the target puncture biopsy path, and thus, the target puncture biopsy path can be used as the puncture biopsy path for the bronchoscopic robot.

[0061] As can be seen, the method disclosed in this embodiment can first obtain a cross-sectional image of the target object and the hardware parameters of the bronchoscopic robot corresponding to the target object; then, based on the cross-sectional image of the target object, the lesion feature information, tracheal feature information, and vascular feature information of the target object can be determined; next, based on the hardware parameters of the bronchoscopic robot and the lesion feature information, tracheal feature information, and vascular feature information of the target object, the airway wall puncture biopsy path of the bronchoscopic robot can be determined. As can be seen, this embodiment considers not only the lesion characteristics of the target object, but also the hardware parameters of the bronchoscopy robot, the tracheal characteristics of the target object, and the vascular characteristics of the target object when determining the airway wall puncture biopsy path of the bronchoscopy robot. Therefore, compared with the traditional method of manually determining the airway wall puncture biopsy path, the airway wall puncture biopsy path planning method using the bronchoscopy robot can greatly reduce the risk of "blind spot puncture" and avoid vascular damage. Furthermore, machine control enables more precise puncture operations, allowing the puncture operation to be perfectly executed according to the planned scheme. Moreover, because multiple constraints are considered in the generation of the airway wall puncture biopsy path, the most efficient and safest airway wall puncture biopsy path can be obtained, thus avoiding the subjectivity and human error problems that occur when manually formulating a plan. Therefore, the airway wall puncture biopsy path obtained by the bronchoscopy robot in this embodiment is more accurate and has greater reference value, and also improves the safety and efficiency of the puncture biopsy procedure.

[0062] To further improve the accuracy and reference value of the airway wall puncture biopsy path obtained by the bronchoscopic robot in this embodiment, and to further improve the safety and efficiency of the puncture biopsy procedure, in one implementation of this embodiment, the method further includes the following steps:

[0063] Step a: Verify the airway wall puncture biopsy path of the bronchoscopic robot and obtain the verification result;

[0064] Step b: If the verification result does not meet the preset verification conditions, continue to execute the step of determining the airway wall puncture biopsy path of the bronchoscopy robot based on the hardware parameters of the bronchoscopy robot and the lesion feature information, tracheal feature information and vascular feature information of the target object, until the verification result meets the preset verification conditions.

[0065] Step c: If the verification result meets the preset verification conditions, then based on the airway wall puncture biopsy path of the bronchoscopic robot, control the bronchoscopic robot to perform puncture biopsy on the target object.

[0066] In this embodiment, after determining the airway wall puncture biopsy path of the bronchoscopy robot, the airway wall puncture biopsy path of the bronchoscopy robot can be verified. For example, based on the airway wall puncture biopsy path of the bronchoscopy robot, the bronchoscopy robot can be controlled to simulate moving along the airway wall puncture biopsy path in the three-dimensional model corresponding to the cross-sectional image of the target object, and it can be determined whether the robot passes through any blood vessels or arteries or veins during the movement, or whether the distance to any blood vessels or arteries or veins is less than a preset threshold, and whether the end effector and end tool of the bronchoscopy robot can pass through the airway in the path, and whether they can reach the lesion location of the target object.

[0067] If the verification result does not meet the preset verification conditions, for example, if, based on the airway wall puncture biopsy path of the bronchoscopic robot, the bronchoscopic robot is controlled to simulate moving along the airway wall puncture biopsy path in the 3D model corresponding to the cross-sectional image of the target object, and it passes through any blood vessel or artery or vein, or the distance to any blood vessel or artery or vein is less than a preset threshold, or the end effector and end tool of the bronchoscopic robot cannot pass through the airway in the path, or cannot reach the lesion location of the target object; then the verification result is considered not to meet the preset verification conditions. In this case, the airway wall puncture biopsy path of the bronchoscopic robot is readjusted and determined. That is, the step of determining the airway wall puncture biopsy path of the bronchoscopic robot based on the hardware parameters of the bronchoscopic robot, as well as the lesion feature information, tracheal feature information, and blood vessel feature information of the target object, continues until the verification result meets the preset verification conditions.

[0068] If the verification result meets the preset verification conditions, for example, based on the airway wall puncture biopsy path of the bronchoscopic robot, during the process of controlling the bronchoscopic robot to move along the airway wall puncture biopsy path in the three-dimensional model corresponding to the cross-sectional image of the target object, without passing through any blood vessels or arteries or veins, or the distance to any blood vessels or arteries or veins is greater than or equal to a preset threshold, and the end effector and end tool of the bronchoscopic robot can pass through the airway in the path and reach the lesion location of the target object; then the verification result can be considered to meet the preset verification conditions, and the bronchoscopic robot can be controlled to perform puncture biopsy sampling on the target object based on the airway wall puncture biopsy path of the bronchoscopic robot.

[0069] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.

[0070] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0071] Figure 3 This is a schematic diagram of the path planning device provided in an embodiment of this disclosure. Figure 3 As shown, the path planning device includes:

[0072] The data acquisition unit 301 is used to acquire cross-sectional images of the target object and the hardware parameters of the bronchoscope robot corresponding to the target object;

[0073] The information determination unit 302 is used to determine the lesion feature information, tracheal feature information and vascular feature information of the target object based on the cross-sectional image of the target object;

[0074] The path determination unit 303 is used to determine the airway wall puncture biopsy path of the bronchoscopy robot based on the hardware parameters of the bronchoscopy robot and the lesion feature information, tracheal feature information and vascular feature information of the target object.

[0075] In some embodiments, the cross-sectional image is a CT image; the information determination unit 302 is configured to:

[0076] Image reconstruction is performed based on the CT images to obtain a three-dimensional model corresponding to the CT images;

[0077] Based on the three-dimensional model, the lesion features, tracheal features, and vascular features of the target object are determined.

[0078] In some embodiments, the path determination unit 303, based on the hardware parameters of the bronchoscopic robot, is configured to:

[0079] Based on the lesion feature information and the tracheal feature information of the target object, the lesion puncture and sampling point is determined;

[0080] The airway puncture point is determined based on the hardware parameters of the bronchoscopic robot and the lesion puncture and sampling location.

[0081] Based on the lesion puncture sampling point, the airway puncture point, and the tracheal characteristic information, the airway wall puncture biopsy path of the bronchoscopic robot is determined.

[0082] In some embodiments, the path determination unit 303 is specifically used for:

[0083] Based on the lesion feature information of the target object, determine the centroid of the lesion of the target object;

[0084] Based on the centroid of the lesion and the tracheal feature information, determine the target trachea that is closest to the centroid of the lesion;

[0085] Based on the centroid of the lesion and the target trachea, determine the initial lesion puncture and sampling location.

[0086] If the initial lesion puncture sampling location meets the first preset condition, then the initial lesion puncture sampling location is used as the lesion puncture sampling point.

[0087] If the initial lesion puncture sampling location does not meet the first preset condition, the initial lesion puncture sampling location is adjusted until the adjusted puncture sampling location meets the first preset condition, and the puncture sampling location that meets the first preset condition is used as the lesion puncture sampling point.

[0088] In some embodiments, the path determination unit 303 is specifically used for:

[0089] Based on the outer diameter parameters of the duct, a tracheal model accessible by the bronchoscope is generated, and the endpoint and centerline of the tracheal model are determined.

[0090] Among all the endpoints and centerlines of the tracheal model, determine the initial airway puncture point that is closest to the lesion puncture sampling point;

[0091] If the initial airway puncture point meets the second preset condition, then the initial airway puncture point is taken as the airway puncture point.

[0092] If the initial airway puncture point does not meet the second preset condition, the initial airway puncture point is adjusted according to the hardware parameters of the bronchoscopic robot until the adjusted airway puncture point meets the second preset condition, and the adjusted airway puncture point that meets the second preset condition is used as the airway puncture point. In one implementation, the second preset condition may be that the path between the lesion puncture sampling point and the target trachea does not pass through or traverse blood vessels, for example, it does not pass through or traverse any blood vessels or arteries or veins.

[0093] In some embodiments, the path determination unit 303 is specifically used for:

[0094] Based on the lesion puncture sampling point and the airway puncture point, several airway wall puncture biopsy paths are determined;

[0095] Based on the tracheal feature information, the several airway wall puncture biopsy paths are screened to obtain the target airway wall puncture biopsy path.

[0096] The target airway wall puncture biopsy path is used as the airway wall puncture biopsy path of the bronchoscopic robot.

[0097] In some embodiments, the apparatus further includes a verification unit, configured to:

[0098] The airway wall puncture biopsy path of the bronchoscopic robot was verified, and the verification results were obtained.

[0099] If the verification result does not meet the preset verification conditions, the step of determining the airway wall puncture biopsy path of the bronchoscopy robot based on the hardware parameters of the bronchoscopy robot and the lesion feature information, tracheal feature information and vascular feature information of the target object shall continue to be executed until the verification result meets the preset verification conditions.

[0100] If the verification result meets the preset verification conditions, then based on the airway wall puncture biopsy path of the bronchoscopic robot, the bronchoscopic robot is controlled to perform a puncture biopsy on the target object.

[0101] According to the technical solution provided in this disclosure, the device considers not only the lesion characteristics of the target object, but also the hardware parameters of the bronchoscopy robot, the tracheal characteristics of the target object, and the vascular characteristics of the target object when determining the airway wall puncture biopsy path of the bronchoscopy robot. Thus, compared to the traditional method of manually determining the airway wall puncture biopsy path, the airway wall puncture biopsy path planning method using the bronchoscopy robot can greatly reduce the risk of "blind spot puncture" and avoid damaging blood vessels. Furthermore, machine control enables more precise puncture operations, allowing the puncture operation to follow the planned scheme. The procedure is executed perfectly. Furthermore, because multiple constraints (i.e., lesion characteristics of the target object, hardware parameters of the bronchoscopic robot, tracheal characteristics of the target object, and vascular characteristics) are considered during the generation of the airway wall puncture biopsy path, the most efficient and safest airway wall puncture biopsy path can be obtained. This avoids the subjectivity and human error that occur when manually formulating a plan, thus making the airway wall puncture biopsy path of the bronchoscopic robot obtained through this embodiment more accurate and more valuable for reference. It also improves the safety and efficiency of the puncture biopsy procedure.

[0102] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.

[0103] Figure 4 This is a schematic diagram of the computer device 4 provided in an embodiment of this disclosure. Figure 4As shown, the computer device 4 in this embodiment includes a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, it implements the steps in the various method embodiments described above. Alternatively, when the processor 401 executes the computer program 403, it implements the functions of each module / unit in the various device embodiments described above.

[0104] Exemplarily, computer program 403 may be divided into one or more modules / units, which are stored in memory 402 and executed by processor 401 to perform the present disclosure. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 403 in computer device 4.

[0105] Computer device 4 can be a desktop computer, laptop, handheld computer, cloud server, or other similar computer device. Computer device 4 may include, but is not limited to, processor 401 and memory 402. Those skilled in the art will understand that... Figure 4 This is merely an example of computer device 4 and does not constitute a limitation on computer device 4. It may include more or fewer components than shown, or combine certain components, or different components. For example, computer device may also include input / output devices, network access devices, buses, etc.

[0106] Processor 401 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0107] The memory 402 can be an internal storage unit of the computer device 4, such as a hard disk or RAM of the computer device 4. The memory 402 can also be an external storage device of the computer device 4, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device 4. Furthermore, the memory 402 can include both internal and external storage units of the computer device 4. The memory 402 is used to store computer programs and other programs and data required by the computer device. The memory 402 can also be used to temporarily store data that has been output or will be output.

[0108] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this disclosure. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0109] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0110] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0111] In the embodiments provided in this disclosure, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0112] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0113] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0114] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in a computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0115] The above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit it. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be included within the protection scope of this disclosure.

Claims

1. A path planning method, characterized in that, The method includes: Obtain cross-sectional images of the target object and the hardware parameters of the bronchoscopic robot corresponding to the target object; Based on the cross-sectional image of the target object, determine the lesion feature information, tracheal feature information, and vascular feature information of the target object; Based on the hardware parameters of the bronchoscopic robot, as well as the lesion characteristics, tracheal characteristics, and vascular characteristics of the target object, the airway wall puncture biopsy path of the bronchoscopic robot is determined. The step of determining the airway wall puncture biopsy path of the bronchoscopy robot based on the hardware parameters of the bronchoscopy robot and the lesion feature information, tracheal feature information, and vascular feature information of the target object includes: Based on the lesion feature information and the tracheal feature information of the target object, the lesion puncture and sampling point is determined; The airway puncture point is determined based on the hardware parameters of the bronchoscopic robot and the lesion puncture and sampling location. Based on the lesion puncture sampling point, the airway puncture point, and the tracheal feature information, the airway wall puncture biopsy path of the bronchoscopic robot is determined. The step of determining the lesion puncture sampling point based on the lesion feature information and the tracheal feature information of the target object includes: Based on the lesion feature information of the target object, determine the centroid of the lesion of the target object; Based on the centroid of the lesion and the tracheal feature information, determine the target trachea that is closest to the centroid of the lesion; Based on the centroid of the lesion and the target trachea, determine the initial lesion puncture and sampling location. If the initial lesion puncture sampling location meets the first preset condition, then the initial lesion puncture sampling location is used as the lesion puncture sampling point. If the initial lesion puncture sampling location does not meet the first preset condition, the initial lesion puncture sampling location is adjusted until the adjusted puncture sampling location meets the first preset condition, and the puncture sampling location that meets the first preset condition is used as the lesion puncture sampling point.

2. The method according to claim 1, characterized in that, The cross-sectional image is a CT image; the step of determining the lesion feature information, tracheal feature information, and vascular feature information of the target object based on the cross-sectional image of the target object includes: Image reconstruction is performed based on the CT images to obtain a three-dimensional model corresponding to the CT images; Based on the three-dimensional model, the lesion features, tracheal features, and vascular features of the target object are determined.

3. The method according to claim 1, characterized in that, The step of determining the airway wall puncture biopsy path of the bronchoscopic robot based on the lesion puncture sampling point, the airway puncture point, and the tracheal feature information includes: Based on the lesion puncture sampling point and the airway puncture point, several airway wall puncture biopsy paths are determined; Based on the tracheal feature information, the several airway wall puncture biopsy paths are screened to obtain the target airway wall puncture biopsy path. The target airway wall puncture biopsy path is used as the airway wall puncture biopsy path of the bronchoscopic robot.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: The airway wall puncture biopsy path of the bronchoscopic robot was verified, and the verification results were obtained. If the verification result does not meet the preset verification conditions, the step of determining the airway wall puncture biopsy path of the bronchoscopy robot based on the hardware parameters of the bronchoscopy robot and the lesion feature information, tracheal feature information and vascular feature information of the target object shall continue to be executed until the verification result meets the preset verification conditions. If the verification result meets the preset verification conditions, then based on the airway wall puncture biopsy path of the bronchoscopic robot, the bronchoscopic robot is controlled to perform a puncture biopsy on the target object.

5. A path planning device, characterized in that, The device includes: The data acquisition unit is used to acquire cross-sectional images of the target object and the hardware parameters of the bronchoscopic robot corresponding to the target object. The information determination unit is used to determine the lesion feature information, tracheal feature information and vascular feature information of the target object based on the cross-sectional image of the target object; The path determination unit is used to determine the airway wall puncture biopsy path of the bronchoscopy robot based on the hardware parameters of the bronchoscopy robot and the lesion feature information, tracheal feature information and vascular feature information of the target object. The path determination unit is specifically used for: Based on the lesion feature information and the tracheal feature information of the target object, the lesion puncture and sampling point is determined; The airway puncture point is determined based on the hardware parameters of the bronchoscopic robot and the lesion puncture and sampling location. Based on the lesion puncture sampling point, the airway puncture point, and the tracheal feature information, the airway wall puncture biopsy path of the bronchoscopic robot is determined. The path determination unit is specifically used to determine the lesion puncture sampling point when: Based on the lesion feature information of the target object, determine the centroid of the lesion of the target object; Based on the centroid of the lesion and the tracheal feature information, determine the target trachea that is closest to the centroid of the lesion; Based on the centroid of the lesion and the target trachea, determine the initial lesion puncture and sampling location. If the initial lesion puncture sampling location meets the first preset condition, then the initial lesion puncture sampling location is used as the lesion puncture sampling point. If the initial lesion puncture sampling location does not meet the first preset condition, the initial lesion puncture sampling location is adjusted until the adjusted puncture sampling location meets the first preset condition, and the puncture sampling location that meets the first preset condition is used as the lesion puncture sampling point.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 4.

Images