Bronchoscope real-time auxiliary navigation method and device and electronic equipment

Abstract

The invention provides a bronchoscope real-time auxiliary navigation method and device and electronic equipment, and relates to the technical field of medical instruments.The bronchoscope real-time auxiliary navigation method comprises the steps that bronchoscope historical motion data and bronchoscope real-time operation data are obtained; analyzing the historical motion data of the bronchoscope to obtain an airway key structure, and constructing an airway three-dimensional model; and comparing the real-time operation data of the bronchoscope with the three-dimensional model to generate a real-time virtual induction pipeline. According to the invention, real-time trajectory tracking and pipeline induction technologies are combined, and auxiliary navigation support is provided for bronchoscope operation.

Description

Technical Field

[0001] The present invention relates to the technical field of medical devices, and in particular to a real-time auxiliary navigation method, device and electronic equipment for a bronchoscope. Background Art

[0002] In modern medical technology, bronchoscopy, as a non-invasive or minimally invasive examination method, is of great significance for the evaluation and treatment of respiratory and lung diseases. This examination uses a bronchoscope, a sophisticated optical instrument, to enable doctors to visually observe the fine structure inside the airway, thereby performing operations such as lesion identification and tissue sampling. However, although bronchoscopy has achieved remarkable results in clinical applications, it still faces a series of challenges in actual operation, especially in the tissue sampling process.

[0003] The bronchial structure is complex and varied, including numerous branches and tortuous passages, which makes it extremely difficult for doctors to operate. When taking tissue samples, due to the fragility of the airway mucosa, local bleeding is easily caused by instrument contact or improper operation. Once bleeding occurs, blood tends to quickly cover the lens, resulting in a severely limited field of view, making it difficult for doctors to accurately determine the sampling location, which in turn increases the difficulty and risk of the operation.

[0004] At present, the bronchoscope systems widely used in the market mainly rely on the doctor's own experience and intuition for navigation and operation, and lack effective auxiliary means when the field of vision is blocked. Although some advanced bronchoscope systems are equipped with high-definition imaging and magnification functions, these improvements still cannot fundamentally solve the problem of blood blocking the field of vision. Therefore, in actual operation, doctors often need to make judgments quickly within a limited field of view, which not only increases the risk of reduced sampling accuracy, but may also lead to missed or missampled samples, posing a potential threat to the patient's treatment effect and safety.

[0005] Therefore, a real-time bronchoscope assisted navigation method, device and electronic equipment are proposed. Summary of the invention

[0006] The present specification provides a real-time assisted navigation method, device and electronic equipment for bronchoscope, which combines real-time trajectory tracking and induced pipeline technology to provide assisted navigation support for bronchoscope operation.

[0007] This specification provides a real-time assisted bronchoscope navigation method, including:

[0008] Obtain bronchoscope historical movement data and bronchoscope real-time operation data;

[0009] Analyzing the historical motion data of the bronchoscope to obtain key airway structures and constructing a three-dimensional airway model;

[0010] Based on the comparison between the real-time operation data of the bronchoscope and the three-dimensional model, a real-time virtual induction pipeline is generated.

[0011] Optionally, the analyzing the historical movement data of the bronchoscope to obtain key airway structures and constructing a three-dimensional airway model includes:

[0012] Analyze the historical motion data of the bronchoscope to obtain the three-dimensional coordinates of key airway structures, the three-dimensional coordinates of trajectory points and their timestamps;

[0013] Correcting the three-dimensional coordinates of the trajectory points by using a Kalman filter model and / or a particle filter model, and saving the corrected three-dimensional coordinates of the trajectory points and their timestamps;

[0014] Smoothing the corrected three-dimensional coordinates of the trajectory points and their timestamps by using Bezier curves and / or spline interpolation methods to obtain a historical trajectory curve;

[0015] Based on the three-dimensional coordinates of the key airway structures and the historical trajectory curve, a three-dimensional airway model is constructed.

[0016] Optionally, the generating a real-time virtual induction pipeline based on comparing the real-time operation data of the bronchoscope with the three-dimensional model includes:

[0017] The real-time operation data of the bronchoscope include the current visual field data of the bronchoscope and the three-dimensional coordinates of the current position of the bronchoscope;

[0018] Identifying the current visual field data of the bronchoscope through an image processing model to determine the occluded area of ​​the current visual field of the bronchoscope;

[0019] Performing blood occlusion identification on the current visual field occlusion area of ​​the bronchus to determine the current blood occlusion area of ​​the bronchoscope;

[0020] Based on the comparison between the real-time operation data of the bronchoscope and the three-dimensional model, a compensation path for the blood-occluded area is constructed, and a real-time virtual induction pipeline is generated.

[0021] Optionally, the image processing model includes an edge detection model and a color detection model.

[0022] Optionally, the blood occlusion recognition method includes a histogram equalization model and / or the edge detection model.

[0023] Optionally, the step of performing blood occlusion recognition on the current visual field occlusion area of ​​the bronchus to determine the current blood occlusion area of ​​the bronchoscope includes:

[0024]

[0025] Among them, G x is the gradient operator matrix of the image in the horizontal direction, G y is the gradient operator matrix of the image in the vertical direction; G is the total gradient amplitude.

[0026] Optionally, the generating of the real-time virtual induction pipeline includes:

[0027] Q(s,a)=Q(s,a)+α[r+γ*maxQ(s ∞ ,a ∞ )-Q(s,a)]

[0028] Among them, Q(s,a) is the score of executing action a in the current state s, α is the learning rate, r is the reward value, γ is the discount rate, Q(s ∞ ,a ∞ ) is the score of the next action a' selected in the new state s';

[0029] The best path is selected by iteratively updating the Q value to generate a real-time virtual induction pipeline.

[0030] This specification provides a real-time auxiliary navigation device for bronchoscope, including:

[0031] An acquisition module is used to acquire the historical movement data of the bronchoscope and the real-time operation data of the bronchoscope;

[0032] An analysis module, used for analyzing the historical motion data of the bronchoscope to obtain key airway structures and construct a three-dimensional airway model;

[0033] A generation module is used to generate a real-time virtual induction pipeline based on the comparison of the real-time operation data of the bronchoscope with the three-dimensional model.

[0034] Optionally, the analysis module includes:

[0035] Analyze the historical motion data of the bronchoscope to obtain the three-dimensional coordinates of key airway structures, the three-dimensional coordinates of trajectory points and their timestamps;

[0036] Correcting the three-dimensional coordinates of the trajectory points by using a Kalman filter model and / or a particle filter model, and saving the corrected three-dimensional coordinates of the trajectory points and their timestamps;

[0037] Smoothing the corrected three-dimensional coordinates of the trajectory points and their timestamps by using Bezier curves and / or spline interpolation methods to obtain a historical trajectory curve;

[0038] Based on the three-dimensional coordinates of the key airway structures and the historical trajectory curve, a three-dimensional airway model is constructed.

[0039] Optionally, the generating module includes:

[0040] The real-time operation data of the bronchoscope include the current visual field data of the bronchoscope and the three-dimensional coordinates of the current position of the bronchoscope;

[0041] Identifying the current visual field data of the bronchoscope through an image processing model to determine the occluded area of ​​the current visual field of the bronchoscope;

[0042] Performing blood occlusion identification on the current visual field occlusion area of ​​the bronchus to determine the current blood occlusion area of ​​the bronchoscope;

[0043] Based on the comparison between the real-time operation data of the bronchoscope and the three-dimensional model, a compensation path for the blood-occluded area is constructed, and a real-time virtual induction pipeline is generated.

[0044] Optionally, the image processing model includes an edge detection model and a color detection model.

[0045] Optionally, the blood occlusion recognition method includes a histogram equalization model and / or the edge detection model.

[0046] Optionally, the step of performing blood occlusion recognition on the current visual field occlusion area of ​​the bronchus to determine the current blood occlusion area of ​​the bronchoscope includes:

[0047]

[0048] Among them, G x The gradient operator matrix of the image in the horizontal direction, G y is the gradient operator matrix of the image in the vertical direction; G is the total gradient amplitude.

[0049] Optionally, the generating of the real-time virtual induction pipeline includes:

[0050] Q(s,a)=Q(s,a)+α[r+γ*maxQ(s ′ ,a ′ )-Q(s,a)]

[0051] Among them, Q(s,a) is the score of executing action a in the current state s, α is the learning rate, r is the reward value, γ is the discount rate, Q(s ′ ,a ′ ) is the score of the next action a' selected in the new state s';

[0052] The best path is selected by iteratively updating the Q value to generate a real-time virtual induction pipeline.

[0053] This specification also provides an electronic device, wherein the electronic device includes:

[0054] processor; and,

[0055] A memory storing computer executable instructions, which when executed cause the processor to perform any of the above methods.

[0056] The present specification also provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and when the one or more programs are executed by a processor, any of the above methods is implemented.

[0057] In the present invention, real-time trajectory tracking and induced pipeline technology are combined to provide auxiliary navigation support for bronchoscopic operations; even in the case of limited field of view such as bleeding, path guidance can be performed based on previously recorded trajectory data. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0059] Figure 1 A schematic diagram of the principle of a real-time assisted navigation method for a bronchoscope provided in an embodiment of this specification;

[0060] Figure 2 A schematic diagram of the structure of a real-time auxiliary navigation device for a bronchoscope provided in an embodiment of this specification;

[0061] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification;

[0062] Figure 4 A schematic diagram of a computer-readable medium provided for an embodiment of this specification. DETAILED DESCRIPTION

[0063] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not deviate from the spirit and scope of the present invention.

[0064] The following is combined with Figure 1-4The exemplary embodiments of the present invention are described more fully. However, the exemplary embodiments can be implemented in various forms, and it should not be understood that the present invention is limited to the embodiments set forth herein. On the contrary, providing these exemplary embodiments can make the present invention more comprehensive and complete, and it is more convenient to fully convey the inventive concept to those skilled in the art. The same reference numerals in the figures represent the same or similar elements, components or parts, and thus their repeated description will be omitted.

[0065] Under the premise of being consistent with the technical concept of the present invention, the features, structures, characteristics or other details described in a specific embodiment do not exclude that they can be combined in one or more other embodiments in a suitable manner.

[0066] In the description of specific embodiments, the features, structures, characteristics or other details described in the present invention are intended to enable those skilled in the art to fully understand the embodiments. However, it does not exclude that those skilled in the art can practice the technical solutions of the present invention without one or more of the specific features, structures, characteristics or other details.

[0067] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0068] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0069] The term "and / or" or "and / or" includes all combinations of any one or more of the associated listed items.

[0070] Figure 1 A schematic diagram of the principle of a real-time assisted navigation method for a bronchoscope provided in an embodiment of this specification, the method may include:

[0071] S110: Acquire historical movement data of the bronchoscope and real-time operation data of the bronchoscope;

[0072] In a specific embodiment of the present specification, the position and posture of the bronchoscope in three-dimensional space are tracked by a positioning sensor, such as electromagnetic positioning, optical positioning or an inertial measurement unit (IMU).

[0073] The system can continuously record every movement of the bronchoscope during the examination through integrated positioning sensors and image processing algorithms. This includes the specific position of the bronchoscope entering and exiting each airway branch, changes in direction, and identification of key anatomical landmarks along the way. These movement data are collected in real time and stored in the system's database, forming a complete and traceable historical data of bronchoscopy examinations.

[0074] S120: Analyze the historical movement data of the bronchoscope to obtain key airway structures, and construct an airway three-dimensional model;

[0075] Optionally, the S120 includes:

[0076] Analyze the historical motion data of the bronchoscope to obtain the three-dimensional coordinates of key airway structures, the three-dimensional coordinates of trajectory points and their timestamps;

[0077] Correcting the three-dimensional coordinates of the trajectory points by using a Kalman filter model and / or a particle filter model, and saving the corrected three-dimensional coordinates of the trajectory points and their timestamps;

[0078] Smoothing the corrected three-dimensional coordinates of the trajectory points and their timestamps by using Bezier curves and / or spline interpolation methods to obtain a historical trajectory curve;

[0079] Based on the three-dimensional coordinates of the key airway structures and the historical trajectory curve, a three-dimensional airway model is constructed.

[0080] In the specific implementation of this specification, the Kalman filter model is used for real-time noise filtering and precise trajectory positioning. Specifically:

[0081]

[0082] in, is the state prediction, A is the state transfer matrix, is the current state, B is the control matrix, u k For control input.

[0083] The state transfer matrix A is used to describe how the system state evolves over time. Specifically, this matrix reflects the changes in the system state between different time steps. Taking the position and velocity of an object as an example, the state transfer matrix can contain detailed information about the changes in the position of the object during the transfer process and the changes in the velocity. This information can be used to better understand the evolution of the system state over time.

[0084] The control matrix B is responsible for converting the control input (e.g., externally applied force or acceleration, etc.) into a specific impact on the system state. Through this conversion process, the control matrix B ensures that the control input can effectively act on the system, thereby achieving regulation of the system behavior.

[0085] Control input u k are external influences on the system, such as thrust or acceleration.

[0086] In summary, the Kalman filter model combines the state transfer matrix A, the control matrix B, and the control input u k The state is predicted and then corrected by real data of the bronchoscope's historical motion data.

[0087] S130: Generate a real-time virtual induction pipeline based on comparing the real-time operation data of the bronchoscope with the three-dimensional model.

[0088] Optionally, the S130 includes:

[0089] The real-time operation data of the bronchoscope include the current visual field data of the bronchoscope and the three-dimensional coordinates of the current position of the bronchoscope;

[0090] Identifying the current visual field data of the bronchoscope through an image processing model to determine the occluded area of ​​the current visual field of the bronchoscope;

[0091] Performing blood occlusion identification on the current visual field occlusion area of ​​the bronchus to determine the current blood occlusion area of ​​the bronchoscope;

[0092] Based on the comparison between the real-time operation data of the bronchoscope and the three-dimensional model, a compensation path for the blood-occluded area is constructed, and a real-time virtual induction pipeline is generated.

[0093] In a specific implementation of the present specification, the current visual field data of the bronchoscope is identified and analyzed, and by processing and analyzing the visual field data, it can be determined which areas are blocked, that is, areas that are not visible in the visual field.

[0094] The occluded area of ​​the current visual field of the bronchoscope refers to the part of the visual field of the bronchoscope that is blocked by other objects (such as bronchial wall, secretions, blood, etc.). These blocked areas can be identified through the image processing model.

[0095] In the occluded area, the blood-occluded area is further identified, which is crucial for diagnosing bleeding points, assessing bleeding volume, and formulating hemostasis strategies.

[0096] The three-dimensional model here represents the anatomical structure of the patient's bronchus. By comparing the real-time operation data with the three-dimensional model, the relative position of the bronchoscope in the bronchus and the position of the occluded area in three-dimensional space can be understood. Based on the comparison results of the real-time operation data and the three-dimensional model, as well as the identification of the blood occluded area, the system can calculate a path to bypass the occluded area (especially the blood occluded area). This path can help doctors assist in navigating the bronchoscope more safely and effectively, and avoid direct contact with bleeding points or other sensitive areas.

[0097] Based on the compensated path, the system can generate a real-time auxiliary virtual guidance channel, which appears as a visual prompt in the bronchoscope's field of view to guide the doctor to operate the bronchoscope along a safe path.

[0098] Optionally, the image processing model includes an edge detection model and a color detection model.

[0099] In the specific implementation of this specification, edge detection is intended to detect edges with obvious changes or discontinuous areas in an image, and these edges usually represent the boundary lines between different areas in the image. Edge detection models include Sobel model, Laplace model, and Canny model.

[0100] Color detection models usually extract and analyze color information in images based on color spaces (such as RGB, HSI, HSV, etc.). By comparing the target color with the color value of each pixel in the image, the target area can be identified. In bronchoscopic image processing, color detection models can be used to identify substances of specific colors such as blood and secretions. For example, by detecting the red area, the presence and distribution range of blood can be identified.

[0101] In bronchoscopic image processing, edge detection models and color detection models can complement each other and jointly improve the accuracy and reliability of image processing. For example, the color detection model can be used to first identify the blood area, and then the edge detection model can be used to further extract the edge contour of the blood area, so as to more accurately determine the location and range of the bleeding point.

[0102] Optionally, the blood occlusion recognition method includes a histogram equalization model and / or the edge detection model.

[0103] In the specific implementation of this specification, the A* (A-star) algorithm or Dijkstra algorithm is used to find the best path to ensure that the induction tube can avoid obstacles (such as bleeding areas) and keep the path short. For complex three-dimensional airway structures, a combination of topological maps and path planning is used.

[0104] For each feasible path, the path weight is calculated, including the length of the path, the number of branch points, the safety of the sampling points (based on the success or failure of the sampling points), and the path is scored. The optimal path is evaluated by the Bellman-Ford algorithm. A three-dimensional induction pipeline is generated, and multiple path points are smoothly connected through segmented Bezier curves or CatmullRom spline interpolation technology to ensure the operability and continuity of the pipeline. The induction pipeline is updated in real time, and the path is corrected in combination with real-time trajectory data to ensure that the path guidance of the doctor during operation is consistent with the surgical environment.

[0105] The A* algorithm is used for path optimization by calculating the cost function f(n)=g(n)+h(n) from the starting point to the target point, where g(n) is the actual cost from the current node to the starting point, and h(n) is the estimated cost from the current node to the target point estimated by the heuristic function.

[0106] Dijkstra algorithm, specifically:

[0107]

[0108] Among them, P i is the control point and t is the parameter.

[0109] Optionally, the step of performing blood occlusion recognition on the current visual field occlusion area of ​​the bronchus to determine the current blood occlusion area of ​​the bronchoscope includes:

[0110]

[0111] Among them, G x The gradient operator matrix of the image in the horizontal direction, G y is the gradient operator matrix of the image in the vertical direction; G is the total gradient amplitude.

[0112] Optionally, the generating of the real-time virtual induction pipeline includes:

[0113] Q(s,a)=Q(s,a)+α[r+γ*maxQ(s ′ ,a ′ )-Q(s,a)]

[0114] Among them, Q(s,a) is the score of executing action a in the current state s, α is the learning rate, r is the reward value, γ is the discount rate, Q(s ′ ,a ′ ) is the score of the next action a' selected in the new state s';

[0115] The best path is selected by iteratively updating the Q value to generate a real-time virtual induction pipeline.

[0116] In a specific embodiment of the present specification, the real-time virtual induction pipeline is stored and played back through three-dimensional visualization technologies such as OpenGL or WebGL, so that doctors can review the operation path after surgery and further optimize the path.

[0117] In the present invention, real-time trajectory tracking and induced pipeline technology are combined to provide auxiliary navigation support for bronchoscopic operations; even in the case of limited field of view such as bleeding, path guidance can be performed based on previously recorded trajectory data.

[0118] Figure 2 A schematic diagram of the principle of a real-time auxiliary navigation device for a bronchoscope provided in an embodiment of this specification, the device may include:

[0119] An acquisition module 10 is used to acquire historical movement data of the bronchoscope and real-time operation data of the bronchoscope;

[0120] An analysis module 20, for analyzing the historical movement data of the bronchoscope to obtain key airway structures and construct a three-dimensional airway model;

[0121] The generating module 30 is used to generate a real-time virtual induction pipeline based on the comparison between the real-time operation data of the bronchoscope and the three-dimensional model.

[0122] Optionally, the analysis module 20 includes:

[0123] Analyze the historical motion data of the bronchoscope to obtain the three-dimensional coordinates of key airway structures, the three-dimensional coordinates of trajectory points and their timestamps;

[0124] Correcting the three-dimensional coordinates of the trajectory points by using a Kalman filter model and / or a particle filter model, and saving the corrected three-dimensional coordinates of the trajectory points and their timestamps;

[0125] Smoothing the corrected three-dimensional coordinates of the trajectory points and their timestamps by using Bezier curves and / or spline interpolation methods to obtain a historical trajectory curve;

[0126] Based on the three-dimensional coordinates of the key airway structures and the historical trajectory curve, a three-dimensional airway model is constructed.

[0127] Optionally, the generating module 30 includes:

[0128] The real-time operation data of the bronchoscope include the current visual field data of the bronchoscope and the three-dimensional coordinates of the current position of the bronchoscope;

[0129] Identifying the current visual field data of the bronchoscope through an image processing model to determine the occluded area of ​​the current visual field of the bronchoscope;

[0130] Performing blood occlusion identification on the current visual field occlusion area of ​​the bronchus to determine the current blood occlusion area of ​​the bronchoscope;

[0131] Based on the comparison between the real-time operation data of the bronchoscope and the three-dimensional model, a compensation path for the blood-occluded area is constructed, and a real-time virtual induction pipeline is generated.

[0132] Optionally, the image processing model includes an edge detection model and a color detection model.

[0133] Optionally, the blood occlusion recognition method includes a histogram equalization model and / or the edge detection model.

[0134] Optionally, the step of performing blood occlusion recognition on the current visual field occlusion area of ​​the bronchus to determine the current blood occlusion area of ​​the bronchoscope includes:

[0135]

[0136] Among them, G x is the gradient operator matrix of the image in the horizontal direction, G y is the gradient operator matrix of the image in the vertical direction; G is the total gradient amplitude.

[0137] Optionally, the generating of the real-time virtual induction pipeline includes:

[0138] Q(s,a)=Q(s,a)+α[r+γ*maxQ(s ∞ ,a ∞ )-Q(s,a)]

[0139] Among them, Q(s,a) is the score of executing action a in the current state s, α is the learning rate, r is the reward value, γ is the discount rate, Q(s ∞ ,a ∞ ) is the score of the next action a' selected in the new state s';

[0140] The best path is selected by iteratively updating the Q value to generate a real-time virtual induction pipeline.

[0141] The functions of the device in the embodiment of the present invention have been described in the above method embodiment, so for details not covered in the description of this embodiment, please refer to the relevant description in the above embodiment, and no further description will be given here.

[0142] Based on the same inventive concept, an embodiment of this specification also provides an electronic device.

[0143] The following describes an electronic device embodiment of the present invention, which can be regarded as a specific physical implementation of the method and device embodiments of the present invention. The details described in the electronic device embodiment of the present invention should be regarded as a supplement to the above method or device embodiments; details not disclosed in the electronic device embodiment of the present invention can be implemented with reference to the above method or device embodiments.

[0144] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. Figure 3 The electronic device 300 according to the embodiment of the present invention is described. Figure 3 The electronic device 300 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0145] like Figure 3 As shown, the electronic device 300 is in the form of a general computing device. The components of the electronic device 300 may include, but are not limited to: at least one processing unit 310, at least one storage unit 320, a bus 330 connecting different system components (including the storage unit 320 and the processing unit 310), a display unit 340, etc.

[0146] The storage unit stores program codes, which can be executed by the processing unit 310, so that the processing unit 310 performs the steps according to various exemplary embodiments of the present invention described in the above processing method section of this specification. For example, the processing unit 310 can perform the following steps: Figure 1 Steps shown.

[0147] The storage unit 320 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 3201 and / or a cache memory unit 3202 , and may further include a read-only memory unit (ROM) 3203 .

[0148] The storage unit 320 may also include a program / utility 3204 having a set (at least one) of program modules 3205, such program modules 3205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0149] Bus 330 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0150] The electronic device 300 may also communicate with one or more external devices 400 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable viewers to interact with the electronic device 300, and / or any device that enables the electronic device 300 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 350. Furthermore, the electronic device 300 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 360. The network adapter 360 may communicate with other modules of the electronic device 300 through the bus 330. It should be understood that although Figure 3 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0151] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the exemplary embodiments described in the present invention can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation method of the present invention can be embodied in the form of a software product, which can be stored in a computer-readable storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.) or on a network, including a number of instructions to enable a computing device (which can be a personal computer, server, or network device, etc.) to execute the above method according to the present invention. When the computer program is executed by a data processing device, the computer-readable medium can implement the above method of the present invention, that is: Figure 1 The method shown.

[0152] Figure 4 A schematic diagram of a computer-readable medium provided for an embodiment of this specification.

[0153] accomplish Figure 1The computer program of the method shown can be stored on one or more computer readable media. The computer readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0154] The computer readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or used in combination with it. The program code contained on the readable storage medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0155] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the viewer computing device, partially on the viewer device, as a stand-alone software package, partially on the viewer computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the viewer computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0156] In summary, the present invention can be implemented in hardware, or in a software module running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that general data processing devices such as microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0157] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the present invention is not inherently related to any specific computer, virtual device or electronic device, and various general devices can also implement the present invention. The above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0158] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0159] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A real-time assisted bronchoscope navigation method, characterized in that: include: Obtain bronchoscope historical movement data and bronchoscope real-time operation data; Analyzing the historical motion data of the bronchoscope to obtain key airway structures and constructing a three-dimensional airway model; Based on the comparison between the real-time operation data of the bronchoscope and the three-dimensional model, a real-time virtual induction pipeline is generated.

2. The real-time assisted bronchoscope navigation method according to claim 1, characterized in that: The analyzing the historical movement data of the bronchoscope to obtain key airway structures and constructing a three-dimensional airway model includes: Analyze the historical motion data of the bronchoscope to obtain the three-dimensional coordinates of key airway structures, the three-dimensional coordinates of trajectory points and their timestamps; Correcting the three-dimensional coordinates of the trajectory points by using a Kalman filter model and / or a particle filter model, and saving the corrected three-dimensional coordinates of the trajectory points and their timestamps; Smoothing the corrected three-dimensional coordinates of the trajectory points and their timestamps by using Bezier curves and / or spline interpolation methods to obtain a historical trajectory curve; Based on the three-dimensional coordinates of the key airway structures and the historical trajectory curve, a three-dimensional airway model is constructed.

3. The real-time assisted bronchoscope navigation method according to claim 2, characterized in that: The step of comparing the real-time operation data of the bronchoscope with the three-dimensional model to generate a real-time virtual induction pipeline includes: The real-time operation data of the bronchoscope include the current visual field data of the bronchoscope and the three-dimensional coordinates of the current position of the bronchoscope; Identifying the current visual field data of the bronchoscope through an image processing model to determine the occluded area of ​​the current visual field of the bronchoscope; Performing blood occlusion identification on the current visual field occlusion area of ​​the bronchus to determine the current blood occlusion area of ​​the bronchoscope; Based on the comparison between the real-time operation data of the bronchoscope and the three-dimensional model, a compensation path for the blood-occluded area is constructed, and a real-time virtual induction pipeline is generated.

4. The real-time assisted bronchoscope navigation method according to claim 3, characterized in that: The image processing model includes an edge detection model and a color detection model.

5. The real-time assisted bronchoscope navigation method according to claim 4, characterized in that: The blood occlusion recognition method includes a histogram equalization model and / or the edge detection model.

6. The real-time assisted bronchoscope navigation method according to claim 5, characterized in that: The step of identifying the blood occlusion of the current visual field occlusion area of ​​the bronchus to determine the current blood occlusion area of ​​the bronchoscope includes: Among them, G x is the gradient operator matrix of the image in the horizontal direction, G y is the gradient operator matrix of the image in the vertical direction; G is the total gradient amplitude.

7. The real-time assisted bronchoscope navigation method according to claim 6, characterized in that: The method of generating a real-time virtual induction pipeline comprises: Q(s,a)=Q(s,a)+α[r+γ*maxQ(s ∞ ,a ∞ )-Q(s,a)] Among them, Q(s,a) is the score of executing action a in the current state s, α is the learning rate, r is the reward value, γ is the discount rate, Q(s ′ ,a ′ ) is the score of the next action a' selected in the new state s'; The best path is selected by iteratively updating the Q value to generate a real-time virtual induction pipeline.

8. A real-time auxiliary navigation device for bronchoscope, characterized in that: include: An acquisition module is used to acquire the historical movement data of the bronchoscope and the real-time operation data of the bronchoscope; An analysis module, used for analyzing the historical motion data of the bronchoscope to obtain key airway structures and construct a three-dimensional airway model; A generation module is used to generate a real-time virtual induction pipeline based on the comparison of the real-time operation data of the bronchoscope with the three-dimensional model.

9. An electronic device, wherein: The electronic device includes: processor; and, A memory storing computer executable instructions which, when executed, cause the processor to perform a method according to any one of claims 1-7.

10. A computer-readable storage medium, wherein: The computer-readable storage medium stores one or more programs, and when the one or more programs are executed by a processor, the method of any one of claims 1 to 7 is implemented.

Images