A three-dimensional navigation method, system, medium and device for flexible cystoscope

Abstract

The present application provides a three-dimensional navigation method, system, medium and device for a flexible bladder scope. The navigation method includes: establishing a bladder model library storing a virtual bladder model; importing a bladder CT image to obtain size data, and screening out a virtual bladder model closest to the size data from the bladder model library; inserting the flexible bladder scope into the patient's bladder, and when detecting that the distal end of the flexible bladder scope reaches the bladder outlet, setting the bladder outlet as the three-dimensional space coordinate zero point; establishing a marker at the bladder outlet of the virtual bladder model to replace the distal end of the flexible bladder scope, and establishing a mapping relationship between the position of the marker in the virtual bladder model and the position of the flexible bladder scope in the patient's bladder; obtaining the position data of the distal end of the flexible bladder scope in real time, and updating the position of the marker in the virtual bladder model in real time. The present application maps the position relationship of the distal end of the flexible bladder scope relative to the patient's bladder through the relative position of the marker and the virtual bladder model, thereby realizing the spatial positioning and navigation function of the flexible bladder scope.

Description

Technical Field

[0001] The present application relates to the technical field of flexible cystoscopes, and in particular to a three-dimensional navigation method, system, medium and device for flexible cystoscopes. Background Art

[0002] Cystoscopy is a minimally invasive surgery method in which the lesion is located inside the human organs. The doctor needs to insert the cystoscope through the urethra into the bladder for surgery. During the operation, the doctor can observe the tissue characteristics of the local tissue area inside the bladder through the images collected by the image sensor at the distal end of the cystoscope. The surgical field of view is limited.

[0003] Because doctors can only obtain local tissue images, they can only judge the approximate position of the distal end of the cystoscope in the bladder through images captured by the image sensor, as well as the relative coordinates of the distal end of the cystoscope and the bladder, as well as the bending state of the distal end of the cystoscope. They cannot know the specific position. Therefore, it is inevitable that the distal end of the cystoscope moves blindly in the bladder, resulting in relatively low inspection efficiency.

[0004] At the same time, if there are multiple lesions in the patient's bladder, ideally, the doctor can comprehensively observe the locations of all lesions through the images of the image sensor. However, due to the complexity of the situation in the patient's bladder, the doctor lacks effective directional guidance when operating the cystoscope, and is very likely to repeat observations of the same tissue area. It is also possible that some tissue areas are relatively similar, causing the doctor to miss certain areas. This not only affects efficiency, but may also cause misdiagnosis of the disease due to incomplete images. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide a three-dimensional navigation method, system, medium and equipment for a cystoscope, which can map the position relationship of the distal end of the cystoscope relative to the patient's bladder through the relative position relationship between the marker and the virtual bladder model, thereby forming a guiding effect on the distal end of the cystoscope, realizing the spatial positioning and navigation function of the distal end of the cystoscope, thereby expanding the surgical field of view and improving surgical efficiency.

[0006] In a first aspect, a method for three-dimensional navigation of a flexible cystoscope is provided, comprising the following steps:

[0007] S1. Establishing a bladder model library: The bladder model library stores virtual bladder models with different size data;

[0008] S2, importing the bladder CT image to obtain the size data of the patient's bladder, and screening out the virtual bladder model that is closest to the size data of the patient's bladder from the bladder model library;

[0009] S3. inserting the flexible cystoscope into the patient's bladder, and when detecting that the distal end of the flexible cystoscope reaches the bladder outlet, setting the bladder outlet as the zero point of the three-dimensional space coordinates;

[0010] S4. Establishing a marker at the bladder outlet of the virtual bladder model to replace the distal end of the flexible cystoscope, and establishing a mapping relationship between the position of the marker in the virtual bladder model and the position of the flexible cystoscope in the patient's bladder; the position of the marker in the virtual bladder model changes as the position of the distal end of the flexible cystoscope in the patient's bladder changes;

[0011] S5. acquiring real-time position data of the distal end of the flexible cystoscope in the patient's bladder;

[0012] S6. Based on the real-time position data obtained in step S5, the position of the marker is updated in real time in the virtual bladder model.

[0013] In one feasible solution, the position data in steps S4 to S6 include spatial coordinate data of the distal end of the flexible cystoscope and a bending angle of the distal end of the flexible cystoscope.

[0014] In an practicable solution, after selecting the virtual bladder model closest to the size data of the patient's bladder from the bladder model library, the method further includes the following steps:

[0015] Determine the size difference between the selected virtual bladder model and the patient's bladder CT image;

[0016] If the size difference is not within the preset range, the size of the selected virtual bladder model is adjusted;

[0017] If the size difference is within the preset range, the selected virtual bladder model is directly used.

[0018] In one feasible solution, in step S3, detecting whether the distal end of the flexible cystoscope reaches the bladder outlet includes the following steps:

[0019] Establishing a bladder outlet image set;

[0020] Real-time acquisition of images of the distal end of the flexible cystoscope;

[0021] Comparing and matching the acquired images with images in the bladder outlet image set;

[0022] If the difference between the acquired image and the image in the bladder outlet image set is within a predetermined threshold range, it indicates a successful match, i.e., the distal end of the cystoscope has reached the bladder outlet;

[0023] If the difference between the acquired image and the image in the bladder outlet image set is not within a predetermined threshold range, it indicates that the distal end of the flexible cystoscope has not reached the bladder outlet.

[0024] In one feasible solution, after the flexible cystoscope enters the patient's bladder, the following steps are further included:

[0025] S7, generating trajectory information of the marker; Step S7 includes the following steps:

[0026] S71, acquiring the spatial coordinate data of the marker once at every predetermined sampling time interval and drawing a position point;

[0027] S72. Connect all the position points in sequence to form the trajectory information of the marker.

[0028] In one feasible solution, after the flexible cystoscope enters the patient's bladder, the following steps are further included:

[0029] S8. Generate a heat map of attention for flexible cystoscopes. Step S8 includes the following steps:

[0030] S81, dividing the interior of the virtual bladder model into a predetermined number of tissue regions;

[0031] S82, recording the spatial coordinate data of the flexible cystoscope and the bending angle of the distal end of the flexible cystoscope in real time;

[0032] S83, determining the tissue region toward which the distal end of the flexible cystoscope is directed based on the spatial coordinate data and the bending angle of the distal end of the flexible cystoscope;

[0033] S84. Record the time during which the distal end of the flexible cystoscope is directed toward the corresponding tissue region, and add a predetermined color to the tissue region according to the length of time.

[0034] In one feasible solution, in step S4, establishing a mapping relationship between the position of the marker on the virtual bladder model and the position of the cystoscope on the patient's bladder includes the following steps:

[0035] Obtaining spatial coordinate data of the distal end of the flexible cystoscope and calculating the relative coordinate relationship between the distal end of the flexible cystoscope and the patient's bladder, adjusting the spatial coordinates of the marker in the virtual bladder model based on the aforementioned relative coordinate relationship, and calculating the spatial coordinate mapping relationship between the marker and the flexible cystoscope;

[0036] The bending angle of the distal end of the flexible cystoscope is obtained, the orientation of the marker in the virtual bladder model is adjusted according to the aforementioned bending angle, and the mapping relationship between the bending angle of the marker and the flexible cystoscope is calculated.

[0037] In a second aspect, a three-dimensional navigation system for a flexible cystoscope is provided, comprising an image sensor, a spatial positioning module, and a control module. The image sensor is positioned at the distal end of the flexible cystoscope. The spatial positioning module is mounted at the distal end of the flexible cystoscope or on the image sensor and is configured to collect position data from the image sensor, including spatial coordinate data and the bending angle of the distal end of the flexible cystoscope. The control module is signal-connected to the image sensor and the spatial positioning module and includes a processing module, a screening module, a detection module, a correlation module, a collection module, and a display module. A processing module is used to establish a bladder model library and store the virtual bladder model in the bladder model library; a screening module is used to obtain the size data of the patient's bladder through the imported bladder CT image, and to screen out the virtual bladder model that is closest to the size data of the patient's bladder from the bladder model library; a detection module is used to obtain image data of the image sensor at the distal end of the cystoscope, and determine whether the image sensor has reached the position of the bladder outlet, and send a flag signal; an association module is used to receive the flag signal, and if the flag signal is received, establish a marker at the bladder outlet of the virtual bladder model to replace the distal end of the cystoscope, and establish a mapping relationship between the position of the marker in the virtual bladder model and the position of the cystoscope in the patient's bladder; the position of the marker in the virtual bladder model changes as the position of the distal end of the cystoscope in the patient's bladder changes; an acquisition module is used to obtain the position data acquired by the spatial positioning module and send the position data; a display module is used to receive the position data and update the position of the marker in the virtual bladder model.

[0038] In one practicable solution, the control module further includes a trajectory module for generating trajectory information of the marker;

[0039] The trajectory module obtains the spatial coordinate data of the marker once every predetermined sampling time, draws a position point, and connects all the position points in sequence to form the trajectory information of the marker.

[0040] In an practicable solution, the control module further comprises a heat map module for generating a focus heat map of the flexible cystoscope;

[0041] The thermal map module divides the interior of the virtual bladder model into a predetermined number of tissue areas, and records the spatial coordinate data of the cystoscope and the bending angle of the distal end of the cystoscope in real time. The tissue area toward which the distal end of the cystoscope is directed is determined based on the spatial coordinate data and the bending angle of the distal end of the cystoscope, and the time when the distal end of the cystoscope is directed toward the aforementioned tissue area is recorded. A predetermined color is added to the tissue area according to the length of time.

[0042] In an implementable solution, the control module further includes a marking module, configured to add marking information to a predetermined area of the virtual bladder model.

[0043] The flexible cystoscope consists of a handpiece and a flexible tube, from proximal to distal. The distal end of the flexible tube is a serpentine section. The handpiece is equipped with a knob, which is connected to the serpentine section via a traction line. Turning the knob bends the serpentine section. In one implementation, the spatial positioning module includes an inertial measurement unit (IMU) and an angle sensor. The IMU is located in the serpentine section, and the angle sensor is located on the knob to detect the knob's rotation angle.

[0044] In a third aspect, a computer storage medium is provided, which stores a computer program, and when the program is executed by a processor, the steps of the above-mentioned flexible cystoscope three-dimensional navigation method are implemented.

[0045] In a fourth aspect, a computer device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and when the program is executed by the processor, the steps of the above-mentioned flexible cystoscope three-dimensional navigation method are implemented.

[0046] Compared with the prior art, the present invention has the following advantages:

[0047] The three-dimensional navigation method of the flexible cystoscope of the present application maps the patient's real bladder through a virtual bladder model, and can reflect the position data of the distal end of the flexible cystoscope in real time through the markers in the virtual bladder model, thereby guiding the distal forward direction of the flexible cystoscope, thereby helping the doctor to understand and control the relative position relationship between the distal end of the flexible cystoscope and the patient's bladder in a global sense, and expanding the surgical field of view.

[0048] Because the marker can reflect the position data of the distal end of the cystoscope in the virtual bladder model, it can guide the surgical path, so the lesion area can be located more purposefully. The relative relationship between the marker and the virtual bladder model can also easily determine which positions have been examined and which positions have not been examined, thereby avoiding invalid repeated examinations of the same positions as much as possible, and basically avoiding the problem of missing inspection areas due to similar tissue areas in the bladder, thereby improving inspection efficiency, reducing the probability of missed inspections, and improving inspection efficiency, which can correspondingly reduce operation time and reduce trauma to patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0050] Figure 1 This is a flow chart of a flexible cystoscope three-dimensional navigation method according to an embodiment of the present application;

[0051] Figure 2 1 is a simplified diagram of the composition of a flexible cystoscope three-dimensional navigation system according to an embodiment of the present application;

[0052] Figure 3 for Figure 2 A partial enlarged view of point A in the middle.

[0053] In the figure: 10, image sensor; 20, spatial positioning module; 30, control module; 31, processing module; 32, screening module; 33, detection module; 34, association module; 35, acquisition module; 36, display module; 37, trajectory module; 38, thermal map module; 39, marking module; 100, flexible cystoscope; 101, handheld part; 102, flexible endoscope tube; 103, snake bone segment; 104, knob. DETAILED DESCRIPTION

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0055] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without creative work are within the scope of protection of the present application.

[0056] It should be noted that, under the premise that the technical solutions do not conflict or contradict each other, the technical features of the following different embodiments can be combined with each other.

[0057] According to the first aspect of this application, Figure 1 As shown, firstly, a flexible cystoscope three-dimensional navigation method is provided, comprising the following steps S1 to S6.

[0058] S1. Establishing a bladder model library: The bladder model library stores virtual bladder models with different size data.

[0059] In step S1, age groups can be divided into different age groups. For example, for individuals under 20 years old, corresponding standard virtual bladder models can be established at one-year or even six-month intervals. For individuals between 20 and 35 years old, standard virtual bladder models can be established at three- to five-year intervals. For individuals between 35 and 50 years old, standard virtual bladder models can be established at five- to seven-year intervals. For individuals between 50 and 70 years old, standard virtual bladder models can be established at eight- to ten-year intervals. For individuals over 70 years old, standard virtual bladder models can be established at ten-year intervals. The data for the aforementioned virtual bladder models can be analyzed using bladder CT images of individuals at different ages to determine the corresponding average size data, which can then be modeled using 3D software.

[0060] It should be noted that, in each age group, the differences in bladders of different genders can be further explored, and then standard virtual bladder models of different genders in the same age group can be established.

[0061] S2. Importing a bladder CT image to obtain the size data of the patient's bladder, and selecting a virtual bladder model that is closest to the size data of the patient's bladder from a bladder model library.

[0062] Among them, the full name of CT is Computed Tomography, which is an electronic computer tomography scan. It is a three-dimensional image that is reconstructed by slicing the human body. Each image is a slice of the human body. All single CT slices are combined together to form a three-dimensional image. The patient's bladder size can be obtained through the bladder CT image.

[0063] In one embodiment, after selecting the virtual bladder model closest to the size data of the patient's bladder from the bladder model library, the following steps are included:

[0064] Determine the size difference between the selected virtual bladder model and the patient's bladder CT image;

[0065] If the size difference is not within the preset range, the size of the selected virtual bladder model is adjusted;

[0066] If the size difference is within the preset range, the selected virtual bladder model is directly used.

[0067] It should be noted that the preset range may be values such as 0.5 mm, 1 mm, 2 mm, etc., and may be determined based on the actual allowable error.

[0068] It should be noted that for most normally developed bladders, a standard virtual bladder model can be created. However, for some people with bladder diseases, they may only be able to find a similar virtual bladder model in a bladder model library and then adjust the size of the virtual bladder model based on the size difference between the virtual bladder model and the patient's bladder CT image.

[0069] S3. Insert the cystoscope into the patient's bladder. When it is detected that the distal end of the cystoscope reaches the bladder outlet, set the bladder outlet as the zero point of the three-dimensional space coordinates.

[0070] Preferably, detecting whether the distal end of the flexible cystoscope reaches the bladder outlet comprises the following steps:

[0071] Establishing a bladder outlet image set;

[0072] Real-time acquisition of images of the distal end of the flexible cystoscope;

[0073] Comparing and matching the acquired images with images in the bladder outlet image set;

[0074] If the difference between the acquired image and the image in the bladder outlet image set is within a predetermined threshold range, it indicates a successful match, i.e., the distal end of the cystoscope has reached the bladder outlet;

[0075] If the difference between the acquired image and the image in the bladder outlet image set is not within a predetermined threshold range, it indicates that the distal end of the flexible cystoscope has not reached the bladder outlet.

[0076] The bladder outlet image set may be formed by image data of the bladder outlet of an existing patient's bladder, and existing image recognition methods may be used to compare and match the collected images with the images in the bladder outlet image set, which will not be listed here one by one.

[0077] In other embodiments, the following method can be used to detect whether the distal end of the flexible cystoscope has reached the bladder outlet: based on big data statistics, common features shared by most bladder outlets are identified, such as the color and tissue structure characteristics of the image at the bladder outlet. Then, a determination is made as to whether the collected images possess these common features. If all images possess these common features, it can be assumed that the distal end of the flexible cystoscope has reached the bladder outlet.

[0078] S4. Establish a marker at the bladder outlet of the virtual bladder model to replace the distal end of the cystoscope, and establish a mapping relationship between the position of the marker in the virtual bladder model and the position of the cystoscope in the patient's bladder.

[0079] The position of the marker in the virtual bladder model changes as the distal end of the flexible cystoscope changes within the patient's bladder. The marker can be a point on the virtual bladder model (preferably at least two points) or a three-dimensional model similar to the distal end of the flexible cystoscope, used to represent the distal end of the flexible cystoscope.

[0080] In one embodiment, in step S4, establishing a mapping relationship between the position of the marker in the virtual bladder model and the position of the cystoscope in the patient's bladder includes the following steps:

[0081] Obtaining spatial coordinate data of the distal end of the flexible cystoscope and calculating the relative coordinate relationship between the distal end of the flexible cystoscope and the patient's bladder, adjusting the spatial coordinates of the marker in the virtual bladder model based on the aforementioned relative coordinate relationship, and calculating the spatial coordinate mapping relationship between the marker and the flexible cystoscope;

[0082] The bending angle of the distal end of the flexible cystoscope is obtained, the orientation of the marker in the virtual bladder model is adjusted according to the aforementioned bending angle, and the mapping relationship between the bending angle of the marker and the flexible cystoscope is calculated.

[0083] Step S4 preferably also includes calibration of the marker position. The position of the distal end of the cystoscope relative to the bladder outlet is determined by using an image of the bladder outlet captured by the distal end of the cystoscope. The position of the marker relative to the bladder outlet of the virtual bladder model is then adjusted to ensure a one-to-one correspondence between the initial positions, thereby ensuring accurate subsequent mapping relationships.

[0084] S5. Acquire the position data of the distal end of the flexible cystoscope in the patient's bladder in real time.

[0085] In step S5, the operator controls the depth of the cystoscope inserted into the bladder and bends the distal end of the cystoscope. Therefore, the position data mentioned above includes the spatial coordinate data of the distal end of the cystoscope and the bending angle of the distal end of the cystoscope. The spatial coordinate data refers to the three-dimensional coordinate data, and the bending angle refers to the distal end of the cystoscope (such as Figure 2 or Figure 3 The bending of the snake segment 103 during operation is shown. If the distal end of the cystoscope is in the same spatial coordinate system, different orientations (different bending angles of the snake segment 103) will result in different bladder regions being observed. The spatial coordinate data and the bending angle of the distal end of the cystoscope provide a more comprehensive view of the distal end's position.

[0086] S6. Based on the real-time position data obtained in step S5, the position of the marker is updated in real time in the virtual bladder model. The spatial coordinate data of the position data is converted into the coordinate data of the marker, and the bending angle data of the distal end of the flexible cystoscope is converted into the orientation of the marker.

[0087] In summary, the three-dimensional navigation method of the cystoscope in this embodiment maps the patient's real bladder through a virtual bladder model, and can reflect the position data of the distal end of the cystoscope through the markers in the virtual bladder model, thereby guiding the distal advance direction of the cystoscope, thereby helping the doctor to understand and control the relative position relationship between the distal end of the cystoscope and the patient's bladder in a global sense, and expanding the surgical field of view.

[0088] Because the marker can reflect the position data of the distal end of the cystoscope in real time in the virtual bladder model, it can guide the surgical path, so the lesion area can be located more purposefully. The relative relationship between the marker and the virtual bladder model can also be easily used to determine which positions have been examined and which positions have not been examined, thereby avoiding invalid repeated examinations of the same positions as much as possible, and basically avoiding the problem of missed examination areas due to similar tissue areas in the bladder, thereby improving the inspection efficiency and reducing the probability of missed inspections. By improving the inspection efficiency, the operation time can be reduced accordingly, reducing the trauma to the patient.

[0089] In one embodiment, after the flexible cystoscope is placed into the patient's bladder, the following steps are further included:

[0090] S7, generating the trajectory information of the marker. Step S7 includes the following steps:

[0091] S71. Obtain the spatial coordinate data of the marker once every predetermined sampling time and draw a location point; the sampling time can be 10ms, 20ms, 30ms, etc., which can be determined by the actual performance and accuracy requirements of the equipment, but try not to make the sampling time too long to avoid low accuracy.

[0092] S72. Connect all the position points in sequence to form the trajectory information of the marker.

[0093] The trajectory of the marker can be used to record the inspection path of the distal end of the cystoscope. On the one hand, during the inspection, the doctor can determine the area that has been inspected based on the trajectory information, locate the lesion area more purposefully, and avoid the inefficient operation of repeated inspections as much as possible. On the other hand, after the inspection is completed, the surgical process can be reviewed and summarized through the recorded inspection path. Furthermore, for key locations, the doctor can directly add highlighted marking points on the location point or trajectory, or add marks similar to information annotations. When a pointer similar to a computer mouse stays on the mark or clicks the mark, the marked information is automatically displayed for subsequent viewing.

[0094] In one embodiment, after the flexible cystoscope is placed into the patient's bladder, the following steps are further included:

[0095] S8. Generate a heat map of attention for flexible cystoscopes. Step S8 includes the following steps:

[0096] S81, dividing the interior of the virtual bladder model into a predetermined number of tissue regions; the tissue regions may be of the same area, or may be divided into tissue regions of different sizes based on specific characteristics within the bladder;

[0097] S82, recording the spatial coordinate data of the flexible cystoscope and the bending angle of the distal end of the flexible cystoscope in real time;

[0098] S83, determining the tissue region toward which the distal end of the flexible cystoscope is directed based on the spatial coordinate data and the bending angle of the distal end of the flexible cystoscope;

[0099] S84. Record the time when the distal end of the flexible cystoscope is directed toward the corresponding tissue area, and add a predetermined color to the tissue area according to the length of time, for example, a longer time corresponds to a darker color, and a shorter time corresponds to a lighter color.

[0100] After a flexible cystoscope completes a procedure, the areas the distal end passes through and faces are colored according to the time spent there, creating a heat map of attention. This color-coded heat map reveals the time and degree of attention paid to different tissue areas within the bladder during surgery, facilitating subsequent analysis. It also helps the next examination by quickly locating key areas and saving time.

[0101] In addition, in one embodiment, after the flexible cystoscope enters the patient's bladder, the following steps may be further included:

[0102] Record the video and key images of the distal field of view of the flexible cystoscope;

[0103] After the surgery is completed, the video and key images are exported and stored in the established playback library;

[0104] Establishing an association relationship between the location points of the trajectory information of the markers based on the acquisition time of the video and key images;

[0105] By selecting a predetermined location point in the trajectory information, the video and key images at the time corresponding to the location point can be displayed, so that while viewing the collection, the video and image information of the corresponding time point can also be replayed, improving the comprehensiveness of the review analysis.

[0106] In addition, in one embodiment, preoperative path planning can also be included. Before the operation, the position of the lesion can be preliminarily located based on the patient's bladder CT image, and then the path can be planned in the virtual bladder model. After the cystoscope enters the patient's bladder, it can follow the planned path to reach the area of the lesion directly. This can reduce the time of searching for the lesion and improve the efficiency of the operation.

[0107] According to a second aspect of the present application, a 3D navigation system for a flexible cystoscope is provided, which is capable of implementing at least the aforementioned 3D navigation method for a flexible cystoscope. A flexible cystoscope 100 comprises at least a handpiece 101 and a flexible tube 102. The proximal end of the flexible tube 102 is connected to the handpiece 101. The handpiece 101 is provided with an adjustment assembly for adjusting the distal end of the flexible tube 102 to bend to a predetermined angle. The structure of the adjustment assembly can be referenced with existing flexible cystoscopes and will not be described in detail here.

[0108] like Figure 2 and Figure 3 As shown, the flexible cystoscope three-dimensional navigation system at least includes an image sensor 10 , a spatial positioning module 20 and a control module 30 .

[0109] The image sensor 10 extends through the inner channel of the flexible lens tube 102 to the distal end of the flexible lens tube 102. The image sensor 10 may preferably use only a CMOS image sensor, CMOS (Complementary Metal-Oxide-Semiconductor), which is known as complementary metal oxide semiconductor in Chinese.

[0110] like Figure 3 As shown, the spatial positioning module 20 can adopt an inertial measurement unit including an accelerometer and a gyroscope. The inertial measurement unit is installed at the distal end of the cystoscope 100 or on the image sensor 10, that is, on the snake bone segment 103, for collecting position data of the image sensor 10. The position data includes the spatial coordinate data of the distal end of the cystoscope and the bending angle of the distal end of the cystoscope.

[0111] In another embodiment, the flexible cystoscope 100 includes a handheld portion 101 and a flexible endoscope tube 102 from the proximal end to the distal end. The distal end of the flexible endoscope tube 102 is a serpentine segment 103. A knob 104 is provided on the handheld portion 101. The serpentine segment 103 is connected to the knob 104 via a traction line. The knob 104 is rotated to bend the serpentine segment 103. The spatial positioning module 20 may include an inertial measurement unit and an angle sensor. The inertial measurement unit is provided on the serpentine segment 103. The angle sensor is provided on the knob 104 to detect the rotation angle of the knob. The inertial measurement unit is only used to detect the coordinate data of the serpentine segment 103. The angle sensor is used to obtain the rotation angle of the knob 104 to calculate the bending angle of the serpentine segment 103. Separately detecting and calculating the two parts of data helps to improve stability.

[0112] For example, when calculating the spatial coordinate data of the distal end (snake segment 103) of the flexible cystoscope 100 using an inertial measurement unit including an accelerometer and a gyroscope, the following data is first obtained:

[0113] A i =V Ai / S A

[0114] Among them, A i V is the acceleration measured by the accelerometer in a single axis direction. Ai is the relative 0g voltage offset of the accelerometer in the single-axis direction, S A is the accelerometer sensitivity.

[0115]

[0116] Among them, A x 、A y 、A z are the accelerations in the three axes measured by the accelerometer, and A is the total acceleration.

[0117] N i =areos(A t -A)

[0118] Among them, N i is the angle between the acceleration direction and the uniaxial direction.

[0119] R i =V Ri / S R

[0120] R i V is the angular rate of change of the gyroscope rotating around a single axis. Ri is the relative zero-rate voltage offset of the gyroscope in a single axis direction, S R is the gyroscope sensitivity.

[0121] The above data can be processed by filtering and other processes to obtain the spatial displacement data of the distal end (the snake section 103) of the flexible cystoscope 100, and further the spatial coordinate data can be obtained.

[0122] Furthermore, when the bending angle of the distal end (the snake segment 103 ) of the flexible cystoscope 100 is calculated using the data from the angle sensor, the deflection angle β of the snake segment 103 can be calculated:

[0123] β=Δx*D

[0124] Wherein, β is the deflection angle of the serpentine segment 103, Δx is the displacement distance of the serpentine traction wire of the bending portion driven by the operating knob, and D is the mapping relationship between the traction wire and the deflection angle of the bending portion.

[0125] It should be noted that D can be obtained by measuring the deflection angle of the snake segment 103 and the displacement distance of the traction line multiple times and calculating multiple times to obtain a calibrated mapping relationship D, and then subsequent cystoscopes with the same structure can use this mapping relationship.

[0126] Δx can be calculated by the following conversion formula:

[0127]

[0128] Δx=α*r

[0129] Wherein, α is the rotation angle of the knob 104 (ie, the value of the angle sensor), L is the circumference of the fixed shaft around which the traction line is wound on the knob 104, and r is the radius of the fixed shaft.

[0130] Furthermore, the control module 30 is signal-connected to the image sensor 10 and the spatial positioning module 20. The control module 30 includes a processing module 31, a screening module 32, a detection module 33, a correlation module 34, an acquisition module 35, and a display module 36. The processing module 31 is used to establish a bladder model library and store virtual bladder models in the library. The screening module 32 is used to obtain the patient's bladder dimensions from imported bladder CT images and select the virtual bladder model from the bladder model library that most closely matches the patient's bladder dimensions. The detection module 33 is used to acquire image data from the image sensor 10 at the distal end of the flexible endoscope tube 102, determine whether the image sensor 10 has reached the bladder outlet, and issue a marker signal. The correlation module 34 is used to receive the marker signal and, if received, establish a marker at the bladder outlet of the virtual bladder model to replace the distal end of the flexible endoscope. A mapping relationship is established between the marker's position in the virtual bladder model and the position of the flexible endoscope in the patient's bladder. The marker's position in the virtual bladder model changes as the distal end of the flexible endoscope changes within the patient's bladder. The acquisition module 35 is used to acquire the position data acquired by the spatial positioning module 20 and to send the position data. The display module 36 is used to receive the position data and to update the position of the marker in the virtual bladder model.

[0131] In one embodiment, Figure 2 As shown, the control module 30 further includes a trajectory module 37 for generating trajectory information of the marker. The trajectory module 37 acquires the spatial coordinate data of the marker once every predetermined sampling time, draws a position point, and sequentially connects all the position points to form the trajectory information of the marker.

[0132] In one embodiment, Figure 2 As shown, the control module 30 also includes a heat map module 38 for generating a heat map of interest for the flexible cystoscope. The heat map module 38 divides the interior of the virtual bladder model into a predetermined number of tissue regions and records the spatial coordinate data and the bending angle of the flexible cystoscope's distal end in real time. Based on the spatial coordinate data and the bending angle of the flexible cystoscope's distal end, the module determines the tissue region toward which the distal end of the flexible cystoscope is directed. The module also records the time the distal end of the flexible cystoscope is directed toward the aforementioned tissue region and, depending on the duration of this time, adds a predetermined color to the tissue region.

[0133] In one embodiment, Figure 2 As shown, the control module 30 further includes a marking module 39 for adding marking information to a predetermined area of the virtual bladder model.

[0134] In one embodiment, the control module 30 further includes an output module for storing images and videos acquired by the image sensor 10 for later playback.

[0135] According to a third aspect of the present application, a computer storage medium is provided, which stores a computer program, which, when executed by a processor, implements the steps of the aforementioned flexible cystoscope three-dimensional navigation method.

[0136] According to a fourth aspect of the present application, a computer device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and when the program is executed by the processor, the steps of the aforementioned flexible cystoscope three-dimensional navigation method are implemented.

[0137] The above are merely preferred embodiments of the present application and are not intended to limit the present application. Those skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A three-dimensional navigation system for flexible cystoscope, characterized in that: include: An image sensor (10) is placed at the distal end of the flexible cystoscope (100); a spatial positioning module (20), mounted on the distal end of the flexible cystoscope (100) or on the image sensor (10), for collecting position data of the image sensor (10), the position data including spatial coordinate data of the distal end of the flexible cystoscope and a bending angle of the distal end of the flexible cystoscope; A control module (30) is connected to the image sensor (10) and the spatial positioning module (20) by signals; the control module (30) comprises: A processing module (31) is used to establish a bladder model library and store the virtual bladder model in the bladder model library; A screening module (32) is used to obtain the size data of the patient's bladder through the imported bladder CT image, and to screen out a virtual bladder model that is closest to the size data of the patient's bladder from the bladder model library; A detection module (33) is used to obtain image data of the image sensor (10) at the distal end of the cystoscope, determine whether the image sensor (10) has reached the position of the bladder outlet, and send a flag signal; An association module (34) is configured to receive the marker signal, and if the marker signal is received, establish a marker at the bladder outlet of the virtual bladder model to replace the distal end of the cystoscope, and establish a mapping relationship between the position of the marker in the virtual bladder model and the position of the cystoscope in the patient's bladder; the position of the marker in the virtual bladder model changes as the position of the distal end of the cystoscope in the patient's bladder changes; A collection module (35) is used to obtain the position data collected by the spatial positioning module (20) and send the position data; The display module (36) is used to receive the position data and update the position of the marker in the virtual bladder model.

2. The flexible cystoscope three-dimensional navigation system according to claim 1, characterized in that: The control module (30) further includes a trajectory module (37) for generating trajectory information of the marker; The trajectory module (37) acquires the spatial coordinate data of the marker once at every predetermined sampling time interval, draws a position point, and connects all the position points in sequence to form the trajectory information of the marker.

3. The flexible cystoscope three-dimensional navigation system according to claim 1, characterized in that: The control module (30) further comprises a heat map module (38) for generating a focus heat map of the flexible cystoscope; The thermal map module (38) divides the interior of the virtual bladder model into a predetermined number of tissue areas, and records the spatial coordinate data of the cystoscope and the bending angle of the distal end of the cystoscope in real time, determines the tissue area toward which the distal end of the cystoscope is directed based on the spatial coordinate data and the bending angle of the distal end of the cystoscope, records the time when the distal end of the cystoscope is directed toward the aforementioned tissue area, and adds a predetermined color to the tissue area according to the length of time.

4. The flexible cystoscope three-dimensional navigation system according to any one of claims 1 to 3, characterized in that: The control module (30) further comprises a marking module (39) for adding marking information to a predetermined area of the virtual bladder model.

5. The flexible cystoscope three-dimensional navigation system according to any one of claims 1 to 3, wherein: The flexible cystoscope (100) comprises a handheld portion (101) and a flexible cystoscope tube (102) in sequence from the proximal end to the distal end, wherein the distal end of the flexible cystoscope tube (102) is a serpentine segment (103), and a knob (104) is provided on the handheld portion (101). The serpentine segment (103) is connected to the knob (104) via a traction line, and the knob (104) is rotated to traction the serpentine segment (103) to bend, and is characterized in that: The spatial positioning module (20) comprises an inertial measurement unit and an angle sensor, wherein the inertial measurement unit is arranged at the snake bone segment (103), and the angle sensor is arranged at the knob (104) to detect the rotation angle of the knob (104).

Images