Method and system for guiding surgical navigation by combining intraoperative 2D3D real-time ultrasound and hysteroscope

Abstract

The invention relates to the technical field of gynaecology and obstetrics instruments or methods, and particularly discloses an intraoperative 2D3D real-time ultrasound and hysteroscope combined guided surgical navigation method and system. The method comprises the steps of S1, intraoperative ultrasonic inspection, S2, coordinate filtering, S3, time weighted coordinate mean value construction, S4, navigation direction calculation and S5, lens AR navigation. According to the method, the real-time coordinate parameters measured by the intraoperative ultrasonic equipment are subjected to filtering processing, the point with the fluctuation quantity smaller than the threshold value is regarded as the static point, it is indicated that no remarkable coordinate change occurs, the calculation accuracy is improved, and errors caused by theoretical individual parameter fluctuation are avoided; compared with a technical mode that a safety value is manually set for redundant navigation directly based on the fluctuation quantity in the X direction, the Y direction and the Z direction, the method has the advantages that the fluctuation quantity in a specific time interval is subjected to statistical analysis, and the time of a static point is used as a weighting parameter, so that the navigation precision is improved, and the resistance and the stability to error influence are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of gynecological and obstetric instruments or methods, and specifically discloses a method and system for guiding surgery navigation by combining 2D3D real-time ultrasound with a hysteroscope during surgery. Background Art

[0002] Endoscopic technology, a modern minimally invasive surgical technique that does not require surgery, will increasingly replace traditional surgery and is undergoing rapid changes. The application of endoscopy technology is of particular importance. The technical principle of endoscopic surgery is to use cold light source lenses, fiber optic wires, image transmission systems, screen display systems, laser illumination, to convert the image of the part to be examined into a digital fiber optic signal, and the image is transmitted to the instrument display screen through optical fibers, so that the image of the disease lesion can be stored and reproduced.

[0003] Intraoperative ultrasound refers to the use of ultrasound for diagnosis and differential diagnosis during surgery. Intraoperative ultrasound can become the surgeon's penetrating eyes and long-distance operating hands. During the operation, a special ultrasound probe is directly contacted with the surface of the organ or lesion to eliminate interference factors such as obesity, bone and iliac acoustic shadows, and gas, thereby reducing blind spots.

[0004] At present, how to use intraoperative 2D3D ultrasound technology combined with hysteroscopy technology for precise navigation and surgery is an important research topic; related technologies are disclosed in the prior art: The Chinese patent with the authorization announcement number CN102266250B discloses an ultrasonic surgical navigation system; specifically, it discloses: it is used for real-time positioning and tracking of surgical instruments during surgery, the system includes an imaging system and a tracking and positioning system, the imaging system includes an ultrasonic probe and an imaging workstation, the imaging workstation displays images according to the real-time images during surgery acquired by the ultrasonic probe, and acquires the position of the surgical instrument in the surgical image according to the tracking and positioning system. By fixing markers on the ultrasonic probe and the surgical instrument, the locator obtains the coordinates of the ultrasonic probe and the surgical instrument, and obtains the relative coordinate position of the surgical instrument relative to the ultrasonic probe, so that during intraoperative ultrasonic imaging, the navigation system can draw the relative position of the fan-shaped ultrasonic surface and the surgical instrument in three-dimensional space according to the precise positioning information provided by the locator, thereby guiding the doctor to perform precise surgical navigation.

[0005] A Chinese patent application with publication number CN116158849A discloses a vascular interventional surgery navigation system, method, electronic device and readable storage medium; specifically discloses: the navigation system includes an ultrasonic imaging device, an image acquisition device and a controller; the image acquisition device is configured to acquire intraoperative patient posture images; the ultrasonic imaging device includes a robotic arm and an ultrasonic probe, and the ultrasonic probe is configured to acquire intraoperative vascular images; the controller is configured to: match the intraoperative patient posture image with the acquired preoperative vascular image to obtain intraoperative position information of the target blood vessel; control the robotic arm to drive the ultrasonic probe to move to the starting ultrasonic observation position, and guide the surgical instrument to puncture the target blood vessel according to the intraoperative vascular image acquired by the ultrasonic probe at the starting ultrasonic observation position; and control the robotic arm to drive the ultrasonic probe to move according to the intraoperative vascular image acquired by the ultrasonic probe.

[0006] The Chinese patent with the authorization announcement number CN110477972B discloses a hysteroscopic uterine myomectomy system; specifically disclosed: including a hard hysteroscope and a light source host and a camera host connected thereto; the hard hysteroscope includes a main endoscope part and a hysteroscope sheath part connected to the main endoscope part; the end of the hard endoscope is provided with a micro-ultrasound probe part, an optical lens, a light-guiding optical fiber and an operating channel outlet. The system is characterized by combining micro-ultrasound technology and hard hysteroscope, adding a micro-ultrasound probe part to the structure of the hard hysteroscope, and starting the micro-ultrasound probe part to perform linear scanning and circular scanning to obtain the true situation of the uterine cavity, that is, to detect the pathological conditions in the uterine cavity and between the uterine walls, and to help doctors diagnose and treat. The system has good stability, ensuring that endoscopic images and ultrasonic images are easier to obtain synchronously, improving the image quality of the lesion area, greatly improving the accuracy of diagnosis, and facilitating doctors to operate and obtain information on pathological changes in the uterine cavity and uterine cavity wall.

[0007] However, the prior art represented by the above patent documents still has the following problems: (1) Intraoperative 2D3D ultrasound technology has the ability to monitor in real time, but it will be affected by the elastic deformation of the patient's tissue, pulse, and blood flow caused by the elastic deformation of the tissue in the field of view. The judgment of the monitoring results depends on the surgeon's independent observation, and the effect of automation and intelligent assistance is not good.

[0008] (2) The existing intraoperative 2D3D ultrasound technology is relatively independent of hysteroscopy and requires the surgeon to independently adjust visual observation. The real-time assisted navigation is poor, and the surgeon constantly changes his field of view, which can easily cause visual fatigue and loss of target.

[0009] (3) In the prior art, there is a technology that uses the amount of endoscope shaking caused by fluctuations in human pulse, blood flow, body position changes, etc. as an error amount for lens navigation. However, this navigation method has the problem that the error amount cannot be predicted and it is easy to introduce large deviations due to individual large error values, resulting in reduced navigation accuracy. Summary of the invention

[0010] To achieve the purpose of the present invention, the present invention is implemented by the following technical solutions: The intraoperative 2D3D real-time ultrasound combined with hysteroscopy guided surgical navigation method includes the following steps: S1. Intraoperative ultrasound examination: During hysteroscopic surgery, the surgical area is examined using intraoperative ultrasound equipment to obtain a lens coordinate set and a surgical target point coordinate set; S2, coordinate filtering: taking into account the human pulse and body deformation, filtering the lens coordinate set and the surgical target point coordinate set, filtering out the coordinate points whose offset is less than the filtering threshold, but retaining the time information of the filtered coordinate points; S3, constructing a time-weighted coordinate mean; for the filtered coordinate set, using the dwell time as a weighted value, calculating the weighted values ​​of the lens coordinates and the surgical target point coordinates; S4, calculating the navigation direction; calculating the navigation direction of the endoscope lens based on the weighted three-coordinate parameters of the lens and the weighted three-coordinate parameters of the surgical target point; S5. Lens AR navigation: Based on the navigation direction of the endoscope lens, the navigation information is displayed in the endoscope field of view using AR technology.

[0011] Furthermore, step S1 specifically includes: during the hysteroscopic surgery, using an intraoperative ultrasound device to inspect the surgical area and obtain the lens coordinates at any time and the surgical target point coordinates , construct the lens coordinate set And the surgical target point coordinate set ,satisfy: ; ; ; ; in, It means that at any sampling time t, the lens X coordinate, lens Y coordinate and lens Z coordinate determined by the orthogonal three-axis coordinate system are respectively , and ; It indicates that the X coordinate, Y coordinate and Z coordinate of the i-th surgical target point at any sampling time t are determined by the orthogonal three-axis coordinate system. , and .

[0012] Furthermore, step S2 specifically includes: Taking into account the human pulse and body deformation, the lens coordinate set And the surgical target point coordinate set Filter to get the lens filter coordinate set And the surgical target point filtered coordinate set ; Specifically include: S21. For any two coordinate points at sampling times t and t+1, if their coordinates satisfy the following formula, the coordinates at sampling times t and t+1 are regarded as stationary points: ; in, represents the X coordinate of the lens or the X coordinate of the surgical target point at the sampling time t, , , , and Similarly; is the filtering threshold; S22, establish a filter coordinate set; construct the coordinate parameters and duration of the continuous static points into a filter coordinate set, including: lens filter coordinate set And the surgical target point filter point coordinate set ,satisfy: ; ; in, and They represent the nth filter coordinate of the lens and the residence time of the lens at the coordinate respectively; and They represent the mth filtered coordinate of the ith surgical target point and the residence time of the surgical target point at the coordinate respectively; N and M represent the total number of filtered coordinates of the lens and the surgical target point respectively.

[0013] Further, step S3 specifically includes: for the filter coordinate set, using the dwell time as a weighted value, calculating the weighted three-coordinate parameters of the lens and weighted three-coordinate parameters of the surgical target point ,satisfy: ; ; ; ; in, Respectively represent the time-weighted coordinate mean of the lens at the X coordinate, Y coordinate, and Z coordinate; Respectively represent the time-weighted coordinate means of the i-th surgical target point at the X coordinate, Y coordinate, and Z coordinate.

[0014] Further, step S4 specifically includes: based on the lens weighted three-coordinate parameters and weighted three-coordinate parameters of the surgical target point , calculate the navigation direction for the endoscope lens; The navigation direction is a vector ,satisfy: .

[0015] Furthermore, in step S5, the navigation information is weighted three-coordinate parameters from the center point of the lens field of view to the surgical target point. Arrow symbol.

[0016] The present invention also provides an intraoperative 2D3D real-time ultrasound combined with hysteroscopy to guide surgical navigation system, which is used to implement the intraoperative 2D3D real-time ultrasound combined with hysteroscopy to guide surgical navigation method, including hysteroscopic surgical equipment, intraoperative ultrasound equipment, a coordinate calculation module, and an AR navigation module; the coordinate calculation module is used to calculate the coordinate values ​​of the lens and the surgical target point; the AR navigation module is used to display navigation auxiliary information to the surgeon through AR technology within the surgical field of view according to the navigation direction.

[0017] The beneficial effects of the present invention are: (1) The present invention filters the real-time coordinate parameters measured by the intraoperative ultrasound device and regards the points with fluctuations less than a threshold as stationary points, indicating that no significant coordinate changes have occurred, thereby improving the accuracy of the calculation and avoiding errors caused by fluctuations in individual theoretical parameters.

[0018] (2) Compared with the technical method of manually setting a safety value for redundant navigation based directly on the fluctuation amount in the X, Y and Z directions, the present invention statistically analyzes the fluctuation amount in a specific time interval and uses the time of the stationary point as a weighting parameter to improve the navigation accuracy and increase the resistance to error influence and stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of the method flow of the present invention. DETAILED DESCRIPTION

[0020] In order to deepen the understanding of the present invention, the present invention will be further described in detail below in conjunction with examples. The examples are only used to explain the present invention and do not constitute a limitation on the protection scope of the present invention.

[0021] Embodiment 1 according to Figure 1 As shown, this embodiment provides a method for guiding surgical navigation by combining 2D3D real-time ultrasound with hysteroscopy during surgery, including the following steps: S1. Intraoperative ultrasound examination: During hysteroscopic surgery, use intraoperative ultrasound equipment to examine the surgical area and obtain the lens coordinates at any time. and the surgical target point coordinates , construct the lens coordinate set And the surgical target point coordinate set ,satisfy: ; ; ; ; in, It means that at any sampling time t, the lens X coordinate, lens Y coordinate and lens Z coordinate determined by the orthogonal three-axis coordinate system are respectively , and ; It means that the X coordinate, Y coordinate and Z coordinate of the i-th surgical target point at any sampling time t are determined by the orthogonal three-axis coordinate system. , and ; S2, coordinate filtering: taking into account the human pulse and body deformation, the lens coordinate set And the surgical target point coordinate set Filter to get the lens filter coordinate set And the surgical target point filtered coordinate set ; Specifically include: S21. For any two coordinate points at sampling times t and t+1, if their coordinates satisfy the following formula, the coordinates at sampling times t and t+1 are regarded as stationary points: ; in, represents the X coordinate of the lens or the X coordinate of the surgical target point at the sampling time t, , , , and Similarly; S22, establish a filter coordinate set; construct the coordinate parameters and duration of the continuous stationary points into a filter coordinate set, including: lens filter coordinate set And the surgical target point filter point coordinate set ,satisfy: ; ; in, and They represent the nth filter coordinate of the lens and the residence time of the lens at the coordinate respectively; and denote the mth filtered coordinate of the ith surgical target point and the residence time of the surgical target point at the coordinate, respectively; N and M denote the total number of filtered coordinates of the lens and the surgical target point, respectively; S3, construct the time-weighted coordinate mean; for the filtered coordinate set, use the dwell time as the weighted value to calculate the lens weighted three-coordinate parameters and weighted three-coordinate parameters of the surgical target point ,satisfy: ; ; ; ; in, Respectively represent the time-weighted coordinate mean of the lens at the X coordinate, Y coordinate, and Z coordinate; Respectively represent the time-weighted coordinate means of the i-th surgical target point at the X coordinate, Y coordinate, and Z coordinate; S4. Calculate navigation direction based on lens weighted three-coordinate parameters and weighted three-coordinate parameters of the surgical target point , calculate the navigation direction for the endoscope lens; S5, AR navigation: Based on the navigation direction of the endoscope lens, the navigation information is displayed in the endoscope field of view using AR technology; the navigation information is the weighted three-coordinate parameters pointing from the center point of the lens field of view to the surgical target point Arrow symbol.

[0022] Embodiment 2 The present embodiment provides an intraoperative 2D3D real-time ultrasound and hysteroscopy combined guided surgical navigation system, which is used to implement an intraoperative 2D3D real-time ultrasound and hysteroscopy combined guided surgical navigation method, including a hysteroscopic surgical device, an intraoperative ultrasound device, a coordinate calculation module, and an AR navigation module; the coordinate calculation module is used to calculate the coordinate values ​​of the lens and the surgical target point; the AR navigation module is used to display navigation auxiliary information to the surgeon through AR technology within the surgical field of view according to the navigation direction.

[0023] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for guiding surgical navigation by combining 2D3D real-time ultrasound with hysteroscopy during surgery, characterized in that: The following steps are involved: S1. Intraoperative ultrasound examination: During hysteroscopic surgery, the surgical area is examined using intraoperative ultrasound equipment to obtain a lens coordinate set and a surgical target point coordinate set; S2, coordinate filtering: taking into account the human pulse and body deformation, filtering the lens coordinate set and the surgical target point coordinate set, filtering out the coordinate points whose offset is less than the filtering threshold, but retaining the time information of the filtered coordinate points; S3, constructing a time-weighted coordinate mean; for the filtered coordinate set, using the dwell time as a weighted value, calculating the weighted values ​​of the lens coordinates and the surgical target point coordinates; S4, calculating the navigation direction; calculating the navigation direction of the endoscope lens based on the weighted three-coordinate parameters of the lens and the weighted three-coordinate parameters of the surgical target point; S5. Lens AR navigation: Based on the navigation direction of the endoscope lens, the navigation information is displayed in the endoscope field of view using AR technology.

2. The intraoperative 2D3D real-time ultrasound and hysteroscopy combined guided surgical navigation method according to claim 1, characterized in that: Step S1 specifically includes: During hysteroscopic surgery, intraoperative ultrasound equipment is used to examine the surgical area and obtain the lens coordinates at any time. and the coordinates of the surgical target point , construct the lens coordinate set And the surgical target point coordinate set ,satisfy: ； ； ； ； in, Indicates that at any sampling time t The X coordinate, Y coordinate and Z coordinate of the lens determined by the orthogonal three-axis coordinate system are , and ; Indicates i The surgical target point at any sampling time t The X coordinate, Y coordinate and Z coordinate of the surgical target point determined by the orthogonal three-axis coordinate system are respectively , and .

3. The intraoperative 2D3D real-time ultrasound and hysteroscopy combined guided surgical navigation method according to claim 2, characterized in that: Step S2 specifically includes: Taking into account the human pulse and body deformation, the lens coordinate set And the surgical target point coordinate set Perform filtering to obtain the lens filter coordinate set And the surgical target point filtered coordinate set ; Specifically include: S21. For any two coordinate points at sampling times t and t+1, if their coordinates satisfy the following formula, the coordinates at sampling times t and t+1 are regarded as stationary points: ； in, represents the X coordinate of the lens or the X coordinate of the surgical target point at the sampling time t, , , , and Similarly; is the filtering threshold; S22, establish a filter coordinate set; construct the coordinate parameters and duration of the continuous stationary points into a filter coordinate set, including: lens filter coordinate set And the surgical target point filter point coordinate set ,satisfy: ； ； in, and They represent the nth filter coordinate of the lens and the residence time of the lens at the coordinate respectively; and They represent the mth filtered coordinate of the ith surgical target point and the residence time of the surgical target point at the coordinate respectively; N and M represent the total number of filtered coordinates of the lens and the surgical target point respectively.

4. The intraoperative 2D3D real-time ultrasound and hysteroscopy combined guided surgical navigation method according to claim 3, characterized in that: Step S3 specifically includes: For the filter coordinate set, the lens weighted three-coordinate parameters are calculated using the dwell time as the weighted value. and weighted three-coordinate parameters of the surgical target point ,satisfy: ； ； ； ； in, Respectively represent the time-weighted coordinate mean of the lens at the X coordinate, Y coordinate, and Z coordinate; Respectively represent the time-weighted coordinate means of the i-th surgical target point at the X coordinate, Y coordinate, and Z coordinate.

5. The intraoperative 2D3D real-time ultrasound and hysteroscopy combined guided surgical navigation method according to claim 4, characterized in that: Step S4 specifically includes: Based on lens weighted three-coordinate parameters and weighted three-coordinate parameters of the surgical target point , calculate the navigation direction of the endoscope lens; the navigation direction is a vector ,satisfy: 。 6. The intraoperative 2D3D real-time ultrasound and hysteroscopy combined guided surgical navigation method according to claim 5, characterized in that: In step S5, the navigation information is the weighted three-coordinate parameters from the center point of the lens field of view to the surgical target point. Arrow symbol.

7. An intraoperative 2D3D real-time ultrasound combined with hysteroscopy guided surgical navigation system, used to implement an intraoperative 2D3D real-time ultrasound combined with hysteroscopy guided surgical navigation method according to any one of claims 1 to 6, characterized in that: It includes hysteroscopic surgical equipment, intraoperative ultrasound equipment, a coordinate calculation module, and an AR navigation module; the coordinate calculation module is used to calculate the coordinate values ​​of the lens and the surgical target point; the AR navigation module is used to display navigation auxiliary information to the surgeon through AR technology within the surgical field of view according to the navigation direction.

Images