Thoracoscope auxiliary intelligent navigation system and method for thoracic surgery

Abstract

The invention discloses a thoracoscope-assisted intelligent navigation system and method for thoracic surgery, and the method comprises the steps: recording the respiratory capacity of a patient and shooting a focus image in a breathing state, precisely calculating the position change amplitude and range of a focus in a breathing process, and drawing a dynamic graph for visual presentation; selecting an electromagnetic navigation system positioning method and performing preoperative planning according to lesion characteristics and positions, and adopting a transbronchial or transthoracic navigation positioning technology; wherein different anatomical regions of a human body are divided according to the focus position and the anatomical structure, a navigation path algorithm is planned for each anatomical region, parameters of the electromagnetic navigation system are adjusted according to the characteristics of tissues passed by the navigation path, and final positioning is confirmed; the television is combined with the thoracoscope to assist the thoracoscope, so that the focus is accurately marked and cut off; according to the method, the safety and success rate of the operation are improved, operation wounds are reduced, better treatment effect and experience are brought to patients, and the method conforms to the development trend of precision and intelligence of the medical technology.

Description

Technical Field

[0001] The present invention relates to the field of medical technologies, and particularly to a thoracoscopic-assisted intelligent navigation system and method for thoracic surgery. Background Art

[0002] Video-assisted thoracoscopic surgery (VATS), as the main technology in minimally invasive thoracic surgery, its core challenge lies in how to achieve precise instrument navigation and lesion localization in a narrow operating space. The traditional method relies on the surgeon's spatial imagination and two-dimensional image reconstruction, and there are problems such as visual operation misalignment, instrument blind spots, and a steep learning curve. In recent years, with the integration of computer vision, augmented reality (AR), and robotic technology, intelligent positioning methods have gradually emerged. Through preoperative CT / MRI three-dimensional modeling, intraoperative real-time image registration, and optical / electromagnetic tracking technology, a "virtual-reality" interaction system is constructed to assist the surgeon in quickly identifying anatomical landmarks, planning the surgical path, and achieving sub-millimeter spatial positioning of the instrument tip. Such technologies have significantly improved the accuracy of complex operations such as pulmonary nodule resection and mediastinal tumor dissection, while reducing the dependence on X-ray fluoroscopy, and have become an important direction for the intelligent development of thoracic surgery.

[0003] Chinese invention patent application No. 201910066672.5 discloses a video-assisted thoracoscopic positioning method, and the method includes: performing preoperative planning to obtain positioning information, where the positioning information includes: lesion point information and preoperative positioning information; selecting a preoperative positioning method according to the positioning information, where the preoperative positioning method includes: a transbronchial navigation positioning method or a transthoracic wall navigation positioning method; positioning the lesion according to the selected preoperative positioning method; and combining video-assisted thoracoscopy and using a positioning probe to mark the positioned lesion. This method provides different preoperative positioning methods, enabling the operator to more accurately position and mark the preoperative lesion according to the lesion information and their proficient method, so as to meet the positioning of various complex lesions while facilitating the operation.

[0004] There are obvious deficiencies in the navigation and positioning methods of existing thoracic surgery. On the one hand, the dynamic changes in the lesion position during the breathing process are not fully considered, resulting in inaccurate preoperative planning and affecting the accuracy of surgical operations. On the other hand, the electromagnetic navigation system positioning method lacks pertinence, does not conduct detailed planning based on the lesion position and anatomical structure characteristics, and the parameter settings cannot flexibly adapt to different tissues. In addition, there is a lack of effective tactile feedback during the surgical process, and it is difficult for doctors to accurately judge the contact situation between the instrument and the tissue. Summary of the Invention

[0005] This application provides a thoracoscopic-assisted intelligent navigation system and method for thoracic surgery, a thoracoscopic-assisted intelligent navigation method and system that consider respiratory dynamics, precisely plan and position, and provide tactile feedback.

[0006] The present application provides a thoracoscopic-assisted intelligent navigation method for thoracic surgery, and the method includes: S1, recording the respiratory volume of the patient and taking images of the lesion in the respiratory state; S2, calculating the amplitude and range of the position change of the lesion during respiration according to the obtained lesion images; S3, drawing a dynamic change diagram of the lesion during the respiratory movement according to the amplitude and range of the lesion position change; S4, selecting and preoperatively planning an electromagnetic navigation system positioning method according to the characteristics and position of the lesion; wherein, the electromagnetic navigation system positioning method includes a transbronchial navigation positioning method and a transthoracic wall navigation positioning method.

[0007] Preferably, in the S4, the electromagnetic navigation system positioning method further includes: S41, dividing different anatomical regions of the human body according to the position and anatomical structure characteristics of the lesion; S42, planning a navigation path algorithm for each anatomical site; S43, adjusting the parameter settings of the electromagnetic navigation system according to the tissue characteristics passed by the navigation path, and confirming the final positioning of the electromagnetic navigation system.

[0008] Preferably, in the S43, confirming the final positioning of the electromagnetic navigation system includes: S431, integrating a tactile feedback device into the instrument handle part of the electromagnetic navigation system, and establishing a mapping relationship between the electromagnetic field strength and the tactile feedback; S432, establishing a virtual contact relationship model between the instrument and the tissue based on the biomechanical characteristics of the human thoracic tissue; S433, designing a force feedback algorithm according to the change of the electromagnetic field strength sensed by the tactile feedback device and the virtual contact relationship model; S434, using a machine learning algorithm to analyze the electromagnetic field strength and tactile feedback data, and training a model to distinguish tissue types; S435, generating multi-modal tactile signals with different characteristics according to the recognition results; S436, determining the movement trend of the electromagnetic navigation system and the contact situation with the tissue through the multi-modal tactile signals.

[0009] Preferably, in the S436, determining the movement trend of the electromagnetic navigation system and the contact situation with the tissue further includes: S6a, adding continuous multi-modal tactile signals simulating the surgical process to the dynamic change diagram during the respiratory movement, and adjusting the image of the dynamic diagram of the lesion; S6b, constructing a mathematical model capable of predicting the respiratory dynamics of the lesion according to the multi-modal tactile signals; S6c, correct and adjust the image of the dynamic map of the lesion according to the mathematical model.

[0010] Preferably, in S2, calculating the amplitude and range of the position change of the lesion during respiration includes: using image registration technology to align the images at different respiratory phases to the end-expiration; extracting the coordinates of the center point of the lesion from the 4D-CT sequence , arranging them in chronological order as a set of trajectory points; where i represents the i-th phase; calculating the maximum displacements of the lesion in the inner-outer, front-back, and up-down three directions; the maximum displacement in the inner-outer direction: ; the maximum displacement in the front-back direction: ; the maximum displacement in the up-down direction: ; the total displacement amplitude: ; fitting the motion trajectory of the lesion and calculating the position at any respiratory phase.

[0011] Preferably, in S3, drawing the dynamic change map includes: inputting the registered 4D image sequence and the lesion coordinates into medical imaging software to generate a dynamic map; superimposing the lesion contour and the movement path of the center point on the dynamic map; marking the maximum displacement direction and the volume change of the lesion at the key phases on the dynamic map.

[0012] Preferably, in S43, it includes: determining the tissue regions passed by the navigation path; measuring the density of the tissues passed by the navigation path, and for each passed tissue region, calculating its average tissue density; analyzing the attenuation of electromagnetic signals in the tissues according to the tissue density ρ; combining with the electromagnetic field strength adjustment formula , calculating the electromagnetic field strength required for each tissue region; where E is the electromagnetic field strength and k is a constant; according to the conductivity σ and permeability μ of the tissue, using the formula to calculate the electromagnetic signal frequency; where f is the electromagnetic signal frequency and c is the speed of light; according to the above analysis results, adjusting the parameter settings of the electromagnetic navigation system.

[0013] Preferably, the designed force feedback algorithm includes: , where is the contact force when the instrument contacts the tissue, F is the vibration frequency, R is the resistance, , and are weight coefficients.

[0014] Preferably, S6c specifically includes: real-time collecting multi-modal tactile feedback signals and inputting them into the mathematical model constructed in S6b, using the prediction result of the mathematical model to correct the lesion dynamic map obtained in step S6a; adjusting the position and morphological information of the lesion in the dynamic map according to the predicted respiratory dynamic changes of the lesion.

[0015] The present application also provides a thoracoscopic-assisted intelligent navigation system for thoracic surgery, and the system includes: an acquisition module, a calculation module, a drawing module, a navigation positioning selection module, and a force feedback module.

[0016] The acquisition module is used to record the respiratory volume of the patient and capture the lesion image in the respiratory state. The calculation module is used to obtain the lesion image and calculate the amplitude and range of the position change of the lesion during the respiratory process. The drawing module is used to draw a dynamic change diagram of the lesion during the respiratory movement according to the amplitude and range of the lesion position change. The navigation positioning selection module is used to select the electromagnetic navigation system positioning method and perform preoperative planning according to the characteristics and position of the lesion. The force feedback module is used to design a force feedback algorithm according to the change of the electromagnetic field intensity sensed by the tactile feedback device and the virtual contact relationship model.

[0017] One or more technical solutions provided in the present application have at least the following technical effects or advantages: By recording the respiratory volume of the patient and capturing the lesion image in the respiratory state, the amplitude and range of the position change of the lesion during the respiratory process are accurately obtained, and then a dynamic diagram is drawn for intuitive display. Preoperative planning and positioning method selection are performed based on the characteristics and position of the lesion to ensure the accuracy of the surgical path. The transbronchial or transthoracic navigation positioning technology is adopted, combined with video-assisted thoracoscopy, to achieve accurate marking and resection of the lesion. This solution not only improves the safety and success rate of the surgery, but also reduces the surgical trauma, bringing better treatment effects and experiences to the patients, and reflecting the development trend of precision and intelligence of medical technology.

[0018] The anatomical region is divided according to the lesion position and anatomical structure characteristics, making the navigation more targeted. When planning the navigation path algorithm, the edge weights are determined by comprehensively considering the tissue characteristics, and the Dijkstra algorithm is used to accurately plan the path. The parameters of the electromagnetic navigation system are adjusted according to the tissue characteristics passed by the navigation path, such as adjusting the electromagnetic field intensity according to the tissue density, calculating the signal frequency according to the conductivity and magnetic permeability, and also considering the phase difference of multi-channel signals for phase adjustment. This series of operations improves the accuracy of the electromagnetic navigation, can better adapt to different anatomical regions and tissue characteristics, and provides more reliable and effective navigation support for the surgery.

[0019] By integrating a tactile feedback device into the instrument handle of an electromagnetic navigation system and establishing a mapping relationship between the electromagnetic field intensity and tactile feedback, the navigation information can be accurately and real-time feedback. Based on the biomechanical characteristics of human tissues, a virtual contact relationship model is constructed and a force feedback algorithm is designed to simulate the real tissue touch and enhance the operation realism. Machine learning is used to distinguish tissue types and generate multi-modal tactile signals, which can assist doctors in quickly identifying different tissues. By determining the movement trend of the electromagnetic navigation system and the contact situation with tissues through multi-modal tactile signals, the accuracy of the electromagnetic navigation system is improved, effectively enhancing the accuracy and safety of electromagnetic navigation in thoracic surgery and reducing the surgical risk.

[0020] By integrating multi-modal tactile signals into the dynamic diagram of respiratory movement changes to adjust the lesion image, the contact situation between the electromagnetic navigation and tissues can be intuitively presented. A prediction model is constructed using deep learning LSTM, and the multi-modal tactile signals are used as inputs to accurately predict the respiratory dynamics of the lesion. During actual surgery, the tactile signals are real-time collected and the lesion dynamic diagram is corrected by the model prediction, making the image highly consistent with the actual respiratory dynamics and enhancing the intuitiveness and accuracy of surgical navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic flow chart of a thoracoscopic-assisted intelligent navigation method for thoracic surgery according to an embodiment of the present invention; Figure 2 It is a structural block diagram of a thoracoscopic-assisted intelligent navigation system for thoracic surgery according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] To facilitate the understanding of the present invention, the present application will be described more comprehensively with reference to the relevant drawings; the preferred embodiments of the present invention are shown in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein; on the contrary, these embodiments are provided to make the disclosure of the present invention more thorough and comprehensive.

[0023] It should be noted that the terms "vertical", "horizontal", "upper", "lower", "left", "right" and similar expressions used herein are for illustrative purposes only and do not represent the only embodiments.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs; the terms used in the description of the present invention in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention; the term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0025] Embodiment 1: Figure 1It is a schematic flowchart of a thoracoscopic-assisted intelligent navigation method for thoracic surgery according to an embodiment of the present invention.

[0026] As Figure 1 shown, the thoracoscopic-assisted intelligent navigation method for thoracic surgery includes the following steps: S1, record the respiratory volume of the patient and capture the lesion images in the respiratory state.

[0027] Specifically, use a respiratory volume monitoring device to record the tidal volume (TV), inspiratory reserve volume (IRV), and expiratory reserve volume (ERV) of the patient in the natural breathing and deep breathing states. Use four-dimensional CT to continuously scan the patient in the free breathing state, covering the entire respiratory cycle (usually divided into the inspiratory phase, expiratory phase, and intermediate phase), obtain the lesion images in the respiratory state, perform three-dimensional reconstruction on the image sequence, and extract the lesion contour.

[0028] S2, calculate the amplitude and range of the position change of the lesion during respiration according to the obtained lesion images.

[0029] Specifically, calculating the amplitude and range of the position change of the lesion during respiration includes: Adopt image registration technology to align the images in different respiratory phases to the end of expiration.

[0030] Extract the coordinates of the lesion center point from the 4D-CT sequence , and arrange them in chronological order as a set of trajectory points; where i represents the i-th phase.

[0031] Calculate the maximum displacement of the lesion in the three directions of inside-outside, front-back, and up-down.

[0032] Among them, based on the standard anatomical position, according to the relative position of organs or structures, inside-outside is defined to describe the positional relationship between hollow organs. The one closer to the inner cavity is the inside, and the one farther from the inner cavity is the outside. Front-back, the one closer to the ventral side of the body is the front, and the one closer to the dorsal side of the body is the back. Up-down, the one closer to the head is the upper, and the one closer to the feet is the lower.

[0033] The maximum displacement in the inside-outside direction: ; The maximum displacement in the front-back direction: ; The maximum displacement in the up-down direction: ; The total displacement amplitude: .

[0034] Fit the motion trajectory of the lesion and calculate the position at any respiratory phase.

[0035] S3, draw a dynamic diagram of the change of the lesion during the respiratory movement according to the amplitude and range of the lesion position change.

[0036] Specifically, drawing the dynamic diagram of the change includes: Input the registered 4D image sequence and lesion coordinates into a medical imaging software (3DSlicer) to generate a dynamic image.

[0037] Overlay the lesion contour (semi-transparent coloring) and the movement path of the center point (curve connecting the positions at each time phase) on the dynamic image.

[0038] Mark the maximum displacement direction and the volume change of the lesion (such as calculating the volume difference) at the key time phases (end-inspiration, end-expiration) on the dynamic image.

[0039] S4. Select and preoperatively plan the electromagnetic navigation system positioning method according to the characteristics and location of the lesion.

[0040] Among them, the electromagnetic navigation system positioning methods include transbronchial navigation positioning method and transthoracic wall navigation positioning method. If the transbronchial navigation positioning method is selected, plan the path of the electromagnetic navigation bronchoscope system to ensure accurate arrival at the lesion area; if the transthoracic wall navigation positioning method is selected, plan the puncture needle insertion path of the transthoracic wall electromagnetic navigation system to ensure that the puncture needle can accurately reach the lesion. The preoperative planning is to determine the path of the electromagnetic navigation bronchoscope system and the puncture needle insertion path of the transthoracic wall electromagnetic navigation system.

[0041] S5. Locate the lesion according to the selected preoperative positioning method, and combine with video-assisted thoracoscopic surgery for marking and resection.

[0042] Specifically, for the transbronchial navigation positioning method, operate the electromagnetic navigation bronchoscope system to make the positioning wire with a positioning sensor and the extension catheter enter the lesion area together. With the help of the positioning sensor equipped on the positioning wire, accurately obtain the position information of the lesion. According to the depth of the positioning wire, accurately determine the anchoring position of the anchoring forceps. Use the anchoring forceps to firmly support the extension catheter at the lesion position. For transthoracic wall navigation, first obtain the registration matrix of the electromagnetic navigation bronchoscope according to the preoperative dynamic image. Based on this registration matrix, perform navigation operations, and at the same time use the peripheral ultrasound probe to obtain the image information of the lesion. Navigate the positioning wire with a sheath to near the lesion, then withdraw the positioning wire and insert the peripheral ultrasound probe for further confirmation. Import the pre-obtained puncture path and registration information into the transthoracic wall electromagnetic navigation system. According to the puncture path and registration information, accurately align the puncture needle with the target puncture point on the subject for puncture. Combine with video-assisted thoracoscopic surgery, use the positioning probe to enter through the thoracic cavity opening to mark the lesion, and perform the lesion resection surgery according to the marked points and marked range.

[0043] The technical solutions in the above embodiments of the present application at least have the following technical effects or advantages: By recording the patient's respiratory volume and taking images of the lesion in the respiratory state, the amplitude and range of the position change of the lesion during respiration are accurately obtained, and then a dynamic graph is drawn to visually display. Based on the characteristics and position of the lesion, preoperative planning and selection of positioning methods are carried out to ensure the accuracy of the surgical path. The transbronchial or transthoracic navigation positioning technology is adopted, combined with video-assisted thoracoscopy, to achieve precise marking and resection of the lesion. This solution not only improves the safety and success rate of the operation, but also reduces surgical trauma, bringing better treatment effects and experiences to patients, reflecting the trend of precision and intelligence development of medical technology.

[0044] Embodiment 2: In Embodiment 1, the conventional human anatomical structure division method is used to plan the path for the electromagnetic navigation system. Although it can meet the basic requirements, it has obvious limitations when facing complex thoracic surgical anatomical structures and tissue characteristics. The conventional division does not fully consider the tissue differences around the lesion and the influence on the propagation of electromagnetic signals, so the path planning is prone to deviation and the parameter adjustment is not precise enough. Due to the large differences in the anatomy and tissue characteristics of different patients, the unified method leads to insufficient navigation accuracy and unstable signals. To accurately adapt to individuals and improve the accuracy and reliability of navigation, it is necessary to optimize the path planning and parameter adjustment by comprehensively considering tissue characteristics, thus leading to Embodiment 2.

[0045] Therefore, the embodiments of the present application are optimized on the basis of the above embodiments.

[0046] In some embodiments, in step S4, the electromagnetic navigation system positioning method further includes: S41. Divide different anatomical regions of the human body according to the position and anatomical structure characteristics of the lesion.

[0047] Among them, the anatomical regions are divided according to the position and anatomical structure characteristics of the lesion, and the human body is divided into different anatomical regions.

[0048] For example, for lung surgery, the lungs can be divided into different regions such as the upper lobe, middle lobe, and lower lobe; for thoracic surgery, the distribution of surrounding structures such as the heart and large blood vessels can be further considered, and the chest can be divided into regions close to the heart, regions close to the large blood vessels, etc.

[0049] S42. Plan a navigation path algorithm for each anatomical part.

[0050] Specifically, the navigation path algorithm planning includes: Obtain detailed structural information and medical image data of the target anatomical region, including tissue distribution, organ boundaries, positions of blood vessels and nerves, etc.

[0051] Convert the medical image data into a three-dimensional grid model, and each grid unit represents a small anatomical space.

[0052] In a three-dimensional grid model, the center point of each grid cell is defined as a node. According to the adjacent relationship of the grid cells, the edges between the nodes are defined. If two grid cells are adjacent, there is an edge between their corresponding nodes.

[0053] The weight of an edge represents the cost of moving from one node to another. Considering the tissue characteristics of the anatomical region, such as tissue density, blood vessel distribution, etc., factors are used to determine the weight of the edge. For example, for a region with a relatively large tissue density, the weight of the edge can be set relatively high, indicating that it is more difficult to move in this region; for a region with obstacles such as blood vessels, the weight of the corresponding edge can be set to infinity, indicating that it is impassable.

[0054] The weight calculation formula can be designed according to the actual situation. The weight formula of this application is designed as: , where is the weight of the edge from node i to node j, is the average tissue density of the region passed by this edge, is a flag indicating whether this edge passes through obstacles such as blood vessels (1 for passing through, 0 for not passing through), and are weight coefficients that can be adjusted according to experiments and experience.

[0055] Design the Dijkstra navigation path algorithm, determine the starting point (such as the surgical incision position) and the target point (such as the center of the lesion) of the navigation path, initialize the distance array and the predecessor node array, create a set of unvisited nodes, etc. For the specific design of the navigation path of the Dijkstra algorithm, please refer to the prior art and will not be elaborated in this article.

[0056] S43. According to the tissue characteristics passed by the navigation path, adjust the parameter settings of the electromagnetic navigation system and confirm the final positioning of the electromagnetic navigation system.

[0057] Specifically, determine the tissue region passed by the navigation path. According to the navigation path planned in step S42, determine each anatomical region passed by the path. It can be mapped to the specific anatomical structure by analyzing the position in the three-dimensional grid model corresponding to the nodes on the path. For example, if a certain node on the path is in the three-dimensional grid model of the upper lobe of the lung, it can be determined that the tissue region corresponding to this node is the upper lobe of the lung.

[0058] Measure the density of the tissue passed by the navigation path. For each passed tissue region, calculate its average tissue density; where i represents the starting node, j represents the target node, and here it represents a certain tissue region passed by a section of the path.

[0059] Analyze the attenuation of electromagnetic signals in tissues according to the tissue density ρ. The greater the tissue density, the more severe the attenuation of electromagnetic signals, and it is necessary to appropriately increase the electromagnetic field strength to ensure the effective transmission of signals.

[0060] Combine with the electromagnetic field strength adjustment formula , and calculate the electromagnetic field strength required for each tissue region; where E is the electromagnetic field strength and k is a constant that can be adjusted according to the actual situation.

[0061] Obtain the conductivity σ and permeability μ of the tissues through which the navigation path passes. According to the conductivity σ and permeability μ of the tissues, use the formula to calculate the electromagnetic signal frequency; where f is the electromagnetic signal frequency and c is the speed of light.

[0062] According to the above analysis results, adjust the parameter settings of the electromagnetic navigation system, including the electromagnetic field strength, frequency, phase, etc., and input the adjusted parameters into the electromagnetic navigation system for parameter configuration.

[0063] It should be noted that in a multi-channel electromagnetic navigation system during surgery, it is necessary to consider the differences in the propagation paths of signals in different channels in tissues, analyze the phase differences of signals reaching the target position, and adjust the phases of signals in each channel according to the phase differences to ensure the coherent superposition of signals in each channel at the target position. The calculation of phase adjustment can be carried out according to the length of the signal propagation path and the electromagnetic characteristics of the tissues.

[0064] The technical solutions in the embodiments of the present application above have at least the following technical effects or advantages: Divide the anatomical regions according to the lesion location and anatomical structure characteristics, making the navigation more targeted. When planning the navigation path algorithm, comprehensively consider the tissue characteristics to determine the edge weights, and use the Dijkstra algorithm to accurately plan the path. Adjust the parameters of the electromagnetic navigation system according to the tissue characteristics passed by the navigation path, such as adjusting the electromagnetic field strength according to the tissue density, calculating the signal frequency according to the conductivity and permeability, etc., and also consider the phase differences of multi-channel signals for phase adjustment. This series of operations improves the accuracy of electromagnetic navigation, can better adapt to different anatomical regions and tissue characteristics, and provides more reliable and effective navigation support for surgery.

[0065] Embodiment 3: In Embodiment 2, the tactile feedback mechanism of the electromagnetic navigation system for thoracic surgery has limitations. Its feedback method is single, it is difficult to accurately simulate the tactile sensations of different thoracic tissues, and it is not accurate for doctors to judge the contact situation between the instrument and the tissue; there is a lack of an efficient tissue type recognition method, increasing the risk of misoperation; and the tactile and lesion dynamic map information are not integrated, making it difficult for doctors to comprehensively understand the navigation situation. To improve the accuracy and safety of the surgery, it is necessary to optimize and improve considering multiple factors, so Embodiment 3 is proposed.

[0066] Therefore, the embodiments of the present application are optimized to a certain extent based on the above embodiments.

[0067] In some embodiments, in step S43, confirming the final positioning of the electromagnetic navigation system includes: S431, integrating the tactile feedback device into the instrument handle part of the electromagnetic navigation system, and establishing a mapping relationship between the electromagnetic field strength and the tactile feedback.

[0068] Among them, establishing the mapping relationship between the electromagnetic field strength and the tactile feedback means measuring the vibration frequency F and the resistance R tactile feedback parameters generated by the tactile feedback device under different electromagnetic field strengths, and establishing a mapping relationship between the electromagnetic field strength B and the tactile feedback parameters. The mapping relationship can be divided into linear and non-linear relationships. According to the specific corresponding results after measurement, if the data shows linear characteristics, a linear formula F = aB + b, R = cB + d (a, b, c, d are constants) is constructed; if the data shows non-linear characteristics, a non-linear corresponding relationship is constructed by methods such as fitting.

[0069] S432, based on the biomechanical characteristics of the human thoracic tissue, establishing a virtual contact relationship model between the instrument and the tissue.

[0070] Specifically, based on parameters such as the elastic modulus E and Poisson's ratio ν of different types of tissues (such as lung tissue, blood vessels, bronchial wall, etc.), a virtual contact relationship model between the instrument and the tissue is established using the finite element analysis method, and the contact force when the instrument contacts the tissue is calculated. 。

[0071] S433, according to the change in the electromagnetic field strength sensed by the tactile feedback device and the virtual contact relationship model, designing a force feedback algorithm.

[0072] Among them, the force feedback algorithm , among which, , and are weight coefficients determined according to experiments, and are used to adjust the influence degree of different parameters on the feedback force.

[0073] S434, using machine learning algorithms to analyze the electromagnetic field strength and tactile feedback data, and training a model to distinguish tissue types.

[0074] Specifically, collecting the electromagnetic field strength B and the corresponding tactile feedback parameters, and labeling the corresponding tissue types (such as blood vessels, bronchial wall, lesions, etc.), and using a neural network to analyze and train these data to establish a tissue type recognition model.

[0075] S435, according to the recognition result, generating multi-modal tactile signals with different characteristics.

[0076] For example, when vascular tissue is recognized, the electromagnetic field intensity generates a vibration signal with a specific frequency and intensity as well as a resistance signal with a specific magnitude ; when diseased tissue is recognized, the electromagnetic field intensity generates another vibration signal with a different frequency , intensity and a resistance ; the multi-modal tactile signal can be expressed as , where B is the electromagnetic field intensity, F is the vibration frequency, A is the vibration intensity, and R is the resistance, and different tissue types correspond to different values of B, f, A, and R.

[0077] S436, determining the movement trend of the electromagnetic navigation system and the contact situation with the tissue through the multi-modal tactile signal.

[0078] The technical solutions in the above embodiments of the present application at least have the following technical effects or advantages: By integrating the tactile feedback device into the instrument handle of the electromagnetic navigation system and establishing the mapping relationship between the electromagnetic field intensity and the tactile feedback, the navigation information can be accurately and real-time feedback. Based on the biomechanical characteristics of human tissues, a virtual contact relationship model is constructed and a force feedback algorithm is designed to simulate the real tissue touch and enhance the operation realism. Using machine learning to distinguish tissue types and generate multi-modal tactile signals can assist doctors in quickly identifying different tissues. Determining the movement trend of the electromagnetic navigation system and the contact situation with the tissue through the multi-modal tactile signal improves the accuracy of the electromagnetic navigation system, effectively improves the accuracy and safety of electromagnetic navigation in thoracic surgery, and reduces the surgical risk.

[0079] Example 4: In Example 3, the thoracic surgery monitors the respiratory dynamics of the lesion by means of the mapping relationship between the force feedback and the electromagnetic signal, but there are limitations. On the one hand, a single mapping relationship is difficult to comprehensively and accurately capture the dynamic changes of the lesion under complex respiratory conditions; on the other hand, the dynamic images taken before the operation present limited information and cannot intuitively reflect the key interaction information such as the contact frequency, resistance, and force between the electromagnetic navigation and the tissue, increasing the surgical uncertainty. To more comprehensively and intuitively present the relevant information and improve the surgical accuracy and safety, Example 4 is specifically proposed.

[0080] Therefore, the embodiments of the present application are optimized to a certain extent on the basis of the above embodiments.

[0081] In some embodiments, in step S436, determining the movement trend of the electromagnetic navigation system and the contact situation with the tissue further includes: S6a, adding continuous multi-modal tactile signals simulating the surgical process to the dynamic images of the changes during the respiratory movement to adjust the images of the dynamic images of the lesion.

[0082] Specifically, using image processing and signal fusion technologies, multi-modal tactile feedback signals are added to the dynamic image of the lesion for simulation. Specifically, according to the characteristics of the tactile feedback information, the image of the lesion is adjusted on the dynamic image. For example, according to the magnitude of the vibration frequency F, the flicker frequency of the lesion image is adjusted to simulate the change in the contact frequency with the tissue when the electromagnetic navigation system moves; according to the magnitude of the resistance R, the transparency or color depth of the lesion image is adjusted to reflect the resistance situation when the electromagnetic navigation system contacts the tissue; according to the vibration intensity A, the change amplitude of the size of the lesion image is adjusted to reflect the strength of the contact force.

[0083] S6b. Construct a mathematical model that can predict the respiratory dynamics of the lesion based on multi-modal tactile signals.

[0084] Specifically, use the multi-modal tactile signals as input features and the respiratory dynamics of the lesion as the output target for model training.

[0085] Collect a large amount of data on the dynamic images of the lesion after fusing multi-modal tactile signals, as well as the corresponding actual respiratory dynamics data of the lesion. Extract effective features from the multi-modal tactile signals, such as the mean and variance of the vibration frequency, the peak value and change rate of the vibration intensity, the distribution characteristics of the resistance, etc. At the same time, extract features from the respiratory dynamics data of the lesion, such as the speed and acceleration of the position movement, the degree of morphological change, etc. Use the long short-term memory network LSTM in deep learning, with the multi-modal tactile signal features as input features and the respiratory dynamics features of the lesion as the output target for model training.

[0086] S6c. Correct and adjust the image of the dynamic image of the lesion according to the mathematical model.

[0087] Specifically, during the actual surgical process, multi-modal tactile feedback signals are collected in real time and input into the mathematical model constructed in step S6b. Using the prediction results of the mathematical model, the dynamic image of the lesion obtained in step S6a is corrected. According to the predicted respiratory dynamic changes of the lesion by the model, adjust the position, shape and other information of the lesion in the dynamic image to make it more consistent with the actual respiratory dynamics of the lesion.

[0088] The technical solutions in the embodiments of the present application described above have at least the following technical effects or advantages: By integrating multi-modal tactile signals into the dynamic image of the respiratory movement change to adjust the lesion image, the contact situation between the electromagnetic navigation and the tissue can be intuitively presented. Use deep learning LSTM to construct a prediction model, and accurately predict the respiratory dynamics of the lesion with multi-modal tactile signals as input. During the actual surgery, collect tactile signals in real time and correct the dynamic image of the lesion through model prediction, making the image highly consistent with the actual respiratory dynamics, enhancing the intuitiveness and accuracy of surgical navigation.

[0089] Furthermore, an embodiment of the present invention also provides a thoracoscopic-assisted intelligent navigation system for thoracic surgery.

[0090] Figure 2 It is a schematic structural diagram of the thoracoscopic-assisted intelligent navigation system for thoracic surgery according to an embodiment of the present invention.

[0091] As Figure 2 shown, the thoracoscopic-assisted intelligent navigation system for thoracic surgery includes: an acquisition module, a calculation module, a drawing module, a navigation positioning selection module, and a force feedback module.

[0092] The acquisition module is used to record the respiratory volume of the patient and capture the lesion image in the respiratory state. The calculation module is used to obtain the lesion image and calculate the position change amplitude and range of the lesion during the breathing process. The drawing module is used to draw a dynamic change diagram of the lesion during the breathing movement according to the position change amplitude and range of the lesion. The navigation positioning selection module is used to select the electromagnetic navigation system positioning method and perform preoperative planning according to the characteristics and position of the lesion. The force feedback module is used to design a force feedback algorithm according to the electromagnetic field strength change sensed by the tactile feedback device and the virtual contact relationship model.

[0093] It should be noted that other specific implementation contents of the thoracoscopic-assisted intelligent navigation system for thoracic surgery according to an embodiment of the present invention can refer to the above-mentioned thoracoscopic-assisted intelligent navigation method for thoracic surgery.

[0094] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A thoracoscopically assisted intelligent navigation method for thoracic surgery, characterized in that: The method comprises: S1, record the patient's respiratory volume and take images of the lesion under respiratory status; S2, calculating the amplitude and range of the position change of the lesion during the breathing process according to the acquired lesion image; S3, according to the amplitude and range of the lesion position change, draw a dynamic diagram of the lesion during respiratory movement; S4, select the electromagnetic navigation system positioning method and preoperative planning according to the characteristics and location of the lesion; among them, the electromagnetic navigation system positioning method includes transbronchial navigation positioning method and transthoracic wall navigation positioning method.

2. The thoracoscopically assisted intelligent navigation method for thoracic surgery according to claim 1, characterized in that: In S4, the electromagnetic navigation system positioning method further includes: S41, divide the human body into different anatomical regions according to the location of the lesion and the characteristics of the anatomical structure; S42, a navigation path planning algorithm for each anatomical part; S43, adjusting parameter settings of the electromagnetic navigation system according to characteristics of the tissue through which the navigation path passes, and confirming the final positioning of the electromagnetic navigation system.

3. The thoracoscopically assisted intelligent navigation method for thoracic surgery according to claim 2, characterized in that: In S43, confirming the final positioning of the electromagnetic navigation system includes: S431, integrating a tactile feedback device into the instrument handle portion of the electromagnetic navigation system, and establishing a mapping relationship between electromagnetic field intensity and tactile feedback; S432, based on the biomechanical properties of human chest tissue, establish a virtual contact relationship model between the instrument and the tissue; S433, designing a force feedback algorithm according to the change in electromagnetic field intensity sensed by the tactile feedback device and the virtual contact relationship model; S434, using machine learning algorithms to analyze electromagnetic field intensity and tactile feedback data and train models to distinguish tissue types; S435, generating a multimodal tactile signal with different characteristics according to the recognition result; S436, determining the movement trend of the electromagnetic navigation system and the contact status with the tissue through multimodal tactile signals.

4. The thoracoscopically assisted intelligent navigation method for thoracic surgery according to claim 3, characterized in that: In S436, determining the movement trend of the electromagnetic navigation system and the contact situation with the tissue further includes: S6a, adding continuous multimodal tactile signals simulating the surgical process to the dynamic image of changes during respiratory movement, and adjusting the image of the dynamic image of the lesion; S6b, construct a mathematical model that can predict the respiratory dynamics of the lesion based on multimodal tactile signals; S6c, image of the dynamic map of the lesion adjusted according to the mathematical model correction.

5. The thoracoscopically assisted intelligent navigation method for thoracic surgery according to claim 1, characterized in that: S2, calculating the amplitude and range of the position change of the lesion during breathing, includes: using image registration technology to align images of different respiratory phases to the end of exhalation; extracting the coordinates of the center point of the lesion from the 4D-CT sequence , arranged in chronological order as a trajectory point set; where i represents the i-th time phase; the maximum displacement of the lesion in the three directions of inside-outside, front-back, and top-bottom is calculated; the maximum displacement in the inside-outside direction is: ; Maximum displacement in the front-to-back direction: ; Maximum displacement in up and down directions: ; Total displacement amplitude: ; Fit the motion trajectory of the lesion and calculate its position at any respiratory phase.

6. The thoracoscopic-assisted intelligent navigation method for thoracic surgery according to claim 1, characterized in that: The S3, drawing a dynamic graph of changes, includes: inputting the registered 4D image sequence and lesion coordinates into medical imaging software to generate a dynamic graph; superimposing the lesion contour and the center point motion path in the dynamic graph; marking the maximum displacement direction and the lesion volume change in the key phase in the dynamic graph.

7. The thoracoscopically assisted intelligent navigation method for thoracic surgery according to claim 2, characterized in that: In the S43, it includes: determining the tissue area through which the navigation path passes; measuring the density of the tissue through which the navigation path passes, and for each tissue area passed, calculating its average tissue density; analyzing the attenuation of the electromagnetic signal in the tissue according to the tissue density ρ; combining the electromagnetic field strength adjustment formula , calculate the electromagnetic field strength required for each tissue area; where E is the electromagnetic field strength and k is a constant; according to the electrical conductivity σ and magnetic permeability μ of the tissue, use the formula Calculate the electromagnetic signal frequency; where f is the electromagnetic signal frequency and c is the speed of light; adjust the parameter settings of the electromagnetic navigation system according to the electromagnetic field adjustment formula and the electromagnetic signal frequency.

8. The thoracoscopically assisted intelligent navigation method for thoracic surgery according to claim 3, characterized in that: The designed force feedback algorithm includes: ,in, is the contact force between the instrument and the tissue, F is the vibration frequency, R is the resistance, , and is the weight coefficient.

9. The thoracoscopically assisted intelligent navigation method for thoracic surgery according to claim 4, characterized in that: The S6c specifically includes: collecting multimodal tactile feedback signals in real time and inputting them into the mathematical model constructed in S6b, and correcting the lesion dynamic map obtained in step S6a using the prediction results of the mathematical model; and adjusting the position and morphological information of the lesion in the dynamic map according to the dynamic changes of the lesion breathing predicted by the model.

10. A thoracoscopically assisted intelligent navigation system for thoracic surgery, applied to a thoracoscopically assisted intelligent navigation method for thoracic surgery as claimed in any one of claims 1 to 9, characterized in that: The system comprises: An acquisition module is used to record the patient's respiratory volume and take images of the lesion under the respiratory state; A calculation module, used to obtain lesion images and calculate the amplitude and range of position changes of the lesions during breathing; A drawing module is used to draw a dynamic diagram of the changes in the lesion during the respiratory movement according to the amplitude and range of the lesion position change; A navigation positioning selection module is used to select the positioning method of the electromagnetic navigation system and perform preoperative planning according to the characteristics and location of the lesion; The force feedback module is used to design a force feedback algorithm based on the changes in electromagnetic field intensity perceived by the tactile feedback device and the virtual contact relationship model.

Images