Operating instructions, devices, equipment, and storage media for flexible medical devices

Abstract

This invention provides a method, apparatus, device, and storage medium for providing operation prompts for a flexible surgical instrument. The method includes: establishing a finite element model of the flexible instrument; the finite element model of the flexible instrument includes multiple finite element nodes; determining the displacement of the flexible instrument based on the principle of minimum potential energy and target constraints; the target constraints represent the positional relationship between the finite element nodes in the finite element model of the flexible instrument and target patches in a three-dimensional model of a human body's natural cavities; the displacement of the flexible instrument includes the displacement of each finite element node; determining the force result of the flexible instrument based on its displacement; and outputting operation instruction information for the flexible instrument based on the determined displacement and force result; the operation instruction information is used to indicate the method of performing surgery on a human body's natural cavities using the flexible instrument. This method can effectively improve the efficiency and safety of surgery.

Description

Technical Field

[0001] This invention relates to the field of medical device technology, and in particular to a method, apparatus, device, and storage medium for providing operation prompts for a flexible medical device. Background Technology

[0002] In minimally invasive surgery, the spatial information of medical devices within the patient's body plays a crucial role in their accurate operation. However, many medical devices inevitably undergo shape changes after entering the body. Therefore, accurately determining the forces and displacements of flexible devices to provide precise information for the surgery is of great significance.

[0003] In related technologies, determining the force and displacement of a flexible instrument requires either manually assigning the contact force between the flexible instrument and the body's natural cavities or relying on collision detection methods to determine the contact position between the flexible instrument and the body's natural cavities. Furthermore, the contact point between the flexible instrument and the body's natural cavities does not change during the iterative process, resulting in a large error in the determined force and displacement of the flexible instrument, which increases the surgical risk for the patient. Summary of the Invention

[0004] To address the problems in the prior art, embodiments of the present invention provide an operation prompting method, apparatus, device, and storage medium for flexible medical devices.

[0005] Specifically, the embodiments of the present invention provide the following technical solutions:

[0006] In a first aspect, embodiments of the present invention provide an operation prompting method for a flexible medical device, comprising:

[0007] A finite element model of the flexible device is established; the finite element model of the flexible device includes multiple finite element nodes;

[0008] Based on the principle of minimum potential energy and target constraints, the displacement of the flexible device is determined; the target constraints are used to represent the positional relationship between the finite element nodes in the finite element model of the flexible device and the target surface in the three-dimensional model of the human body's natural cavities; the displacement of the flexible device includes the displacement of each finite element node.

[0009] The force result of the flexible instrument is determined based on its displacement.

[0010] Based on the determined displacement and force results of the flexible instrument, operation instruction information of the flexible instrument is output; the operation instruction information of the flexible instrument is used to indicate the method of performing surgery on the natural cavities of the human body through the flexible instrument.

[0011] Furthermore, determining the displacement of the flexible device based on the principle of minimum potential energy and target constraints includes:

[0012] The displacement of the flexible device is determined when the potential energy change of the flexible device is minimized and the positional relationship between the flexible device and the human body's natural cavity satisfies the target constraint condition; the target constraint condition is that the finite element nodes in the finite element model of the flexible device and the target surface in the three-dimensional model of the human body's natural cavity are not penetrable.

[0013] Furthermore, determining the displacement of the flexible device based on the principle of minimum potential energy and target constraints includes:

[0014] The displacement of the flexible instrument is determined using the following formula:

[0015]

[0016] or

[0017] Ax i =b i

[0018] The This represents the minimum change in potential energy of the flexible device; the Δx i The displacement of the flexible instrument from time i-1 to time i; the K(x) i-1 ) represents the stiffness matrix of the flexible device; the x i-1 The displacement of the finite element node at time i-1; F represents the target external force acting on the flexible device, which does not include the contact force between the flexible device and the natural cavity of the human body; or Represents the target constraint conditions; the The x represents the normal vector of the contact patch between the flexible device and the natural cavity of the human body; i The S represents the finite element node displacement at time i; i X0 represents the generalized coordinate position of the flexible device in its undeformed state; X0 represents any position on the plane corresponding to the three-dimensional model of the human body's natural cavities; Ax i =b i The first constraint condition is represented by A; the projection matrix is ​​represented by b. i Indicates the target position of the flexible device; the x i This represents the displacement of the finite element node at time i.

[0019] Furthermore, determining the force result of the flexible device based on its displacement includes:

[0020] The force result of the flexible device is determined based on the displacement of the flexible device and the stiffness matrix of the flexible device.

[0021] Furthermore, the method also includes:

[0022] The stiffness matrix of the flexible device is determined by the rotation matrix between the local coordinate system and the global coordinate system of the element in the finite element model.

[0023] Furthermore, after determining the displacement of the flexible device, the following also includes:

[0024] In the case where the natural human cavity is a narrow bifurcated cavity, based on the determined displacement of the flexible device, the positional relationship between the finite element nodes in the finite element model of the flexible device and each facet in the three-dimensional model of the natural human cavity is determined.

[0025] The target constraint is updated when at least one surface in the finite element model of the flexible device penetrates between a finite element node and multiple surfaces in the three-dimensional model of the human body's natural cavities.

[0026] Secondly, embodiments of the present invention also provide an operation prompting device for a flexible medical device, comprising:

[0027] A module is established to create a finite element model of the flexible device; the finite element model of the flexible device includes multiple finite element nodes.

[0028] The first determining module is used to determine the displacement of the flexible device based on the principle of minimum potential energy and target constraints; the target constraints are used to represent the constraints on the positional relationship between the finite element nodes in the finite element model of the flexible device and the target surface in the three-dimensional model of the human body's natural cavities; the displacement of the flexible device includes the displacement of each finite element node.

[0029] The second determining module is used to determine the force result of the flexible device based on the displacement of the flexible device;

[0030] The prompting module is used to output operation instruction information for the flexible instrument based on the determined displacement and force result of the flexible instrument; the operation instruction information for the flexible instrument is used to indicate the method of performing surgery on the natural cavities of the human body through the flexible instrument.

[0031] Thirdly, embodiments of the present invention also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the operation prompting method for the flexible device as described in the first aspect.

[0032] Fourthly, embodiments of the present invention also provide a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the operation prompting method for the flexible device as described in the first aspect.

[0033] Fifthly, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the operation prompting method for the flexible device as described in the first aspect.

[0034] The operation prompting method, device, equipment, and storage medium for flexible instruments provided in this invention are based on the constraints of the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the target surface in the three-dimensional model of the human body's natural cavities, and the principle of minimum potential energy. This determines the displacement of the flexible instrument corresponding to the minimum potential energy change, thus ensuring that the determined displacement satisfies the environment of the human body's confined cavity height, making the determined displacement more accurate. Consequently, surgeons can accurately operate the flexible instrument based on the determined displacement and operation prompts, improving surgical efficiency and safety. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0036] Figure 1 This is a flowchart illustrating the operation prompting method for a flexible device provided in an embodiment of the present invention;

[0037] Figures 2a-2b This is a schematic diagram of the flexible device provided in an embodiment of the present invention contacting the natural cavity of the human body;

[0038] Figure 3 This is a flowchart illustrating another operation prompting method for a flexible device provided in an embodiment of the present invention;

[0039] Figure 4 This is a schematic diagram of the operation prompting device for the flexible medical device provided in an embodiment of the present invention;

[0040] Figure 5 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0042] The method described in this invention can be applied to surgical scenarios based on medical devices, enabling operation prompts for flexible instruments and improving surgical efficiency and safety.

[0043] In related technologies, determining the force and displacement of a flexible instrument requires either manually assigning the contact force between the flexible instrument and the body's natural cavities or relying on collision detection methods to determine the contact position between the flexible instrument and the body's natural cavities. Furthermore, the contact point between the flexible instrument and the body's natural cavities does not change during the iterative process, resulting in a large error in the determined force and displacement of the flexible instrument, which increases the surgical risk for the patient.

[0044] The operation prompting method for flexible instruments in this invention is based on the constraints of the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the target surface in the three-dimensional model of the human body's natural cavities, as well as the principle of minimum potential energy. It determines the displacement of the flexible instrument corresponding to the minimum potential energy change, thus ensuring that the determined displacement satisfies the environment of the human body's confined cavity height, making the determined displacement more accurate. Consequently, based on the determined displacement and operation instructions of the flexible instrument, surgeons can accurately operate it, improving surgical efficiency and safety.

[0045] The following is combined Figures 1-5 The technical solution of the present invention will be described in detail with reference to specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0046] Figure 1 This is a flowchart illustrating an embodiment of the operation prompting method for a flexible device provided in this invention. Figure 1 As shown, the method provided in this embodiment includes:

[0047] Step 101: Establish the finite element model of the flexible device; the finite element model of the flexible device includes multiple finite element nodes;

[0048] Specifically, in minimally invasive surgery, the spatial information of medical devices within the patient's body plays a crucial role in their accurate operation. However, flexible instruments inevitably undergo shape changes after entering the body, thus requiring accurate determination of the forces and displacements acting on them. This provides surgeons with precise information to improve surgical efficiency and safety.

[0049] In this embodiment of the invention, a finite element model of the flexible device is first established. Optionally, the finite element model of the flexible device is a model of the flexible device established using the finite element analysis method. It is a combination of elements connected only at nodes, transmitting force only through nodes, and being constrained only at nodes. The finite element model of the flexible device includes multiple finite element elements and finite element nodes. By establishing the finite element model of the flexible device, the continuous geometric mechanism corresponding to the flexible device can be discretized into a finite number of flexible device finite element elements, and a finite number of flexible device finite element nodes can be set in each flexible device finite element element. Thus, the continuous body corresponding to the flexible device can be regarded as a collection of flexible device finite element elements connected only at the flexible device finite element nodes.

[0050] After establishing the finite element model of the flexible device, when determining the displacement and force of the flexible device, it can be regarded as a combination of many flexible device finite element elements and flexible device finite element nodes connected to each other. After determining the displacement and force of each sub-unit of the flexible device, the displacement and force of the flexible device can be determined. By simulating the deformation of the flexible device through a large number of flexible device finite element elements and finite element nodes, the difficulty of determining the displacement and force results of the flexible device is reduced.

[0051] Step 102: Based on the principle of minimum potential energy and target constraints, determine the displacement of the flexible device; the target constraints are used to represent the constraints on the positional relationship between the finite element nodes in the finite element model of the flexible device and the target surface in the three-dimensional model of the human body's natural cavities; the displacement of the flexible device includes the displacement of each finite element node.

[0052] Specifically, after establishing the finite element model of the flexible device, the displacement of the flexible device can be determined based on the principle of minimum potential energy and the target constraint conditions; optionally, the potential energy of the flexible device at a certain moment i-1 is expressed as follows:

[0053]

[0054] Where U is the total potential energy, x i-1 Let K(x) be the generalized coordinate column vector of the nodal displacement at time i-1, and F be the known total external force; the total external force F does not include the interaction force between the flexible device and the natural cavity of the human body; i-1 Let be the stiffness matrix of the flexible device. The potential energy of the flexible device at the next moment i is as follows:

[0055]

[0056]

[0057] Where Δx i Let x be the displacement of the flexible device from time i-1 to time i. i =x i-1 +Δx i C is related to Δx i An irrelevant constant term. Defined in relation to Δx. i The relevant unknown potential energy change ΔU is as follows:

[0058]

[0059] Based on the principle of minimum potential energy, and using the incremental method, given U... i-1 Need to find U i To find the minimum value, we only need to minimize ΔU. The displacement corresponding to the minimum potential energy change of the flexible instrument is the displacement Δx of the flexible instrument. i Optionally, the displacement Δx of the flexible device i This includes the displacement of each finite element node, which is the displacement Δx of the flexible instrument. i It is composed and determined by multidimensional finite element nodal displacements. Furthermore, when determining the displacement of a flexible device, it is necessary to consider the displacement Δx of the flexible device in a highly confined environment such as human body cavities. i It is subject to environmental constraints. That is, the embodiments of the present invention are based on the constraints of the positional relationship between the finite element nodes in the finite element model of the flexible device and the target surface in the three-dimensional model of the human body's natural cavity, and the principle of minimum potential energy. The displacement corresponding to the minimum potential energy change of the flexible device is determined, which is the displacement of the flexible device. This makes the determined displacement of the flexible device satisfy the environment of the human body cavity height restriction, and thus makes the determined displacement of the flexible device more accurate.

[0060] Step 103: Determine the force result of the flexible instrument based on its displacement;

[0061] Specifically, based on the constraint conditions of the positional relationship between the finite element nodes in the finite element model of the flexible device and the target surface in the three-dimensional model of the human body's natural cavities, and after determining the displacement of the flexible device, the force result of the flexible device can be determined based on the determined displacement. Optionally, the force result of the flexible device can be determined according to Hooke's theorem. Optionally, the determined force result of the flexible device includes the force result of each finite element node, that is, the determined force result of the flexible device is composed and determined based on the force result of multi-dimensional finite element nodes.

[0062] Step 104: Based on the determined displacement and force results of the flexible instrument, output the operation instruction information of the flexible instrument; the operation instruction information of the flexible instrument is used to indicate the method of performing surgery on the natural cavities of the human body through the flexible instrument.

[0063] Specifically, based on the constraint conditions of the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the target surface in the 3D model of the human body's natural cavities, and after determining the displacement and force results of the flexible instrument, operational instructions for the flexible instrument can be output based on these results. These instructions guide the surgeon on how to operate the flexible instrument to perform the next step in the surgery on the human body's natural cavities. The surgeon can then accurately operate the flexible instrument based on these instructions, improving the efficiency and safety of the surgery. Optionally, after determining the displacement and force results of the flexible instrument, these results can also be displayed on the screen for the surgeon's reference, thereby accurately determining the next surgical action and further improving the efficiency and safety of the surgery.

[0064] Existing technical solutions typically provide the force results of flexible instruments directly or rely on collision detection algorithms to determine the contact position between the flexible instrument and the body's natural cavities. Furthermore, the contact point between the flexible instrument and the body's natural cavities remains unchanged during the iteration process, leading to errors in the determined displacement and force results of the flexible instrument, resulting in insufficient accuracy and real-time performance. This also causes errors in the indicated information, affecting the efficiency and safety of the surgery. In this embodiment of the invention, by combining the determination of the displacement and force results of the flexible instrument with the constraint conditions of the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the target surface in the three-dimensional model of the body's natural cavities, the method does not rely on manually given contact forces between the flexible instrument and the body's natural cavities, nor does it rely on collision detection algorithms to determine the contact position. Therefore, the method of this embodiment does not introduce errors from different collision detection methods, nor does it assume that the contact point position between the flexible instrument and the body's natural cavities remains unchanged during the calculation of two time frames. This makes the determined displacement and force results of the flexible instrument in this embodiment of the invention more accurate.

[0065] On the other hand, in related technologies, relying on collision detection algorithms or simplified calculations in the contact space (such as the SOFA method) significantly reduces the speed of determining the displacement and force of the flexible device as the number of contact points between the flexible device and the human body's natural cavity increases. The number of contact points greatly affects the real-time performance and stability of determining the displacement and force of the flexible device. In the embodiments of this invention, the process of determining the displacement and force of the flexible device does not rely on manually given contact forces between the flexible device and the human body's natural cavity, nor does it rely on collision detection algorithms to determine the contact position between the flexible device and the human body's natural cavity. This allows the method of this embodiment to calculate the displacement and force of the flexible device in real time under different contact conditions between the flexible device and the human body's natural cavity, especially when there are many contact points. This improves the stability and practicality of determining the displacement and force of the flexible device, and avoids the problem in related technologies where the efficiency of determining the displacement of the flexible device decreases as the number of collision points between the flexible device and the human body's natural cavity increases.

[0066] Thirdly, in related technologies, when the relative motion speed between the flexible device and the human body's natural cavity is relatively high (relative to the surface scale of the human body's natural cavity model), the accuracy of the simulated displacement and force conditions of the flexible device is severely insufficient. The embodiments of this invention do not limit the relative motion speed between the flexible device and the human body's natural cavity, and do not restrict the relative motion speed between the two objects. This allows for a good simulation of the physical motion process of objects with high relative motion speeds, resulting in more accurate displacement and force results for the flexible device.

[0067] The method described above, based on the constraints of the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the target surface in the three-dimensional model of the human body's natural cavities, and the principle of minimum potential energy, determines the displacement of the flexible instrument corresponding to the minimum potential energy change. This ensures that the determined displacement of the flexible instrument satisfies the environment of the human body's cavity height limitation, making the determined displacement of the flexible instrument more accurate. Consequently, based on the determined displacement of the flexible instrument and the operation instructions of the flexible instrument, the surgeon can accurately operate the flexible instrument, improving the efficiency and safety of the surgery.

[0068] In one embodiment, the displacement of the flexible device is determined based on the principle of minimum potential energy and target constraints, including:

[0069] The displacement of the flexible device is determined when the potential energy change of the flexible device is minimized and the positional relationship between the flexible device and the human body's natural cavity satisfies the target constraint condition. The target constraint condition is that the finite element nodes in the finite element model of the flexible device and the target surface in the three-dimensional model of the human body's natural cavity cannot penetrate each other.

[0070] Specifically, since the height of the human body cavity is limited, when determining the displacement of the flexible instrument based on the principle of minimum potential energy, the determined displacement of the flexible instrument must satisfy the height of the natural human body cavity. That is, the displacement of the flexible instrument must be determined under the condition that the finite element nodes in the finite element model of the flexible instrument and the target surface in the 3D model of the natural human body cavity are not penetrable. This ensures that the determined displacement of the flexible instrument is within the natural human body cavity, and cannot be determined outside the natural human body cavity. Therefore, the determined displacement of the flexible instrument conforms to the actual surgical situation, and thus the determined displacement of the flexible instrument is more accurate. Optionally, the target surface is the surface in the 3D model of the natural human body cavity that is closest to the finite element nodes of the flexible instrument.

[0071] The method described in the above embodiments, when determining the displacement of the flexible instrument based on the principle of minimum potential energy, uses the impenetrable constraint condition between the finite element nodes in the finite element model of the flexible instrument and the target surface in the three-dimensional model of the human body's natural cavity. This ensures that the determined displacement of the flexible instrument satisfies the height of the human body's natural cavity, thus ensuring that the determined displacement of the flexible instrument is within the human body's natural cavity and cannot be determined outside the human body's natural cavity. This makes the determined displacement of the flexible instrument more accurate and more consistent with the actual surgical situation. Therefore, based on the determined displacement of the flexible instrument, the next surgical plan can be determined more accurately, improving the efficiency and safety of the surgery.

[0072] In one embodiment, the displacement of the flexible device is determined based on the principle of minimum potential energy and target constraints, including:

[0073] The displacement of the flexible instrument is determined using the following formula:

[0074]

[0075] or

[0076] Ax i =b i

[0077] Δx represents the minimum change in potential energy of a flexible device. i K(x) represents the displacement of the flexible instrument from time i-1 to time i; i-1 ) represents the stiffness matrix of the flexible instrument; x i-1 denoted as the finite element node displacement at time i-1; F represents the target external force acting on the flexible device, which does not include the contact force between the flexible device and the human body's natural cavities. or Indicate the objective constraints; The normal vector of the contact patch between the flexible medical device and the natural cavities of the human body; x i S represents the finite element nodal displacement at time i; i X represents the generalized coordinate position of a flexible instrument in its undeformed state; X0 represents any position on the plane corresponding to the three-dimensional model of a natural human cavity; Ax i =b i Denotes the first constraint condition; A denotes the projection matrix; b i Indicates the target location of the flexible instrument; x i This represents the displacement of the finite element node at time i.

[0078] Specifically, based on the constraints of the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the target surface in the three-dimensional model of the human body's natural cavities, and the principle of minimum potential energy, this embodiment of the invention determines the displacement of the flexible instrument under the condition of minimum potential energy change. This ensures that the determined displacement of the flexible instrument satisfies the environment of the human body cavity height restriction, making the determined displacement of the flexible instrument more accurate. Consequently, it can accurately provide the surgeon with operation prompts for the flexible instrument, improving the efficiency and safety of the surgery.

[0079] Optionally, compared to the static equilibrium method, one advantage of determining the displacement of the flexible instrument based on the principle of minimum potential energy in this embodiment of the invention is the simplified expression of the environmental constraints between the flexible instrument and the body's natural cavities. This improves the accuracy and efficiency of the determined displacement of the flexible instrument, thereby providing accurate operational prompts to the surgeon and enhancing the efficiency and safety of the surgery. Optionally, the displacement of the flexible instrument is determined using the following formula:

[0080]

[0081] or

[0082] Ax i =b i

[0083] Δx represents the minimum change in potential energy of a flexible device. i K(x) represents the displacement of the flexible instrument from time i-1 to time i; i-1 ) represents the stiffness matrix of the flexible instrument; x i-1 denoted as the finite element node displacement at time i-1; F represents the target external force acting on the flexible device, which does not include the contact force between the flexible device and the human body's natural cavities. or Indicate the objective constraints; The normal vector of the contact patch between the flexible medical device and the natural cavities of the human body; x i S represents the finite element nodal displacement at time i; i X represents the generalized coordinate position of a flexible instrument in its undeformed state; X0 represents any position on the plane corresponding to the three-dimensional model of a natural human cavity; Ax i =b i Denotes the first constraint condition; A denotes the projection matrix; b i Indicates the target location of the flexible instrument; x i Let Ax represent the finite element nodal displacement at time i. i =b i This means mapping the known actual position of the flexible instrument to the finite element model of the flexible instrument; This is used to determine the displacement of a flexible instrument, specifically by determining the displacement Δx corresponding to the minimum change in potential energy of the flexible instrument. i The displacement Δx of the flexible instrument i It is composed of and determined by multidimensional finite element nodal displacements.

[0084] Optionally, the 3D model of the human body's natural cavities, reconstructed from preoperative CT scans, consists of multiple facets in the virtual environment. The inequality constraint on the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the target facets in the 3D model of the human body's natural cavities simplifies the collision between the flexible instrument and the human body's natural cavities to a point-to-surface collision. The normal vector of the facet at the contact point between the flexible instrument and the human body's natural cavities is... For any point on the plane, at position X0, we have or The inequality direction of the positional relationship between finite element nodes in the finite element model of a flexible device and target patches in the 3D model of a natural human body cavity depends on the magnitude of the initial value, where S... i This represents the generalized coordinate position of the flexible device in its undeformed state. Optionally, or The finite element nodes in the finite element model of the flexible device and the target surface in the 3D model of the human body's natural cavities are impenetrable. An inequality relating the positional relationships between these nodes is incorporated as a constraint into the minimum potential energy principle for determining the displacement of the flexible device. Furthermore, an equation representing the known position of the flexible device obtained by the flexible robot through sensors or other algorithms, approximately Ax, is introduced. i =b iThe displacement of a flexible instrument can be accurately determined using the following formula, ensuring that the determined displacement satisfies the constraint condition of the height of the body's natural cavities, thus making the determined displacement of the flexible instrument more accurate:

[0085]

[0086] or

[0087] Ax i =b i

[0088] For example, such as Figure 2a and 2b As shown, Figure 2a This diagram illustrates the contact points and forces between the finite element model of the flexible device and the natural cavity model of the human body between two frames in an embodiment of the present invention. Figure 2b This diagram illustrates the contact points and forces between a finite element model of a flexible device and a natural human cavity model between two frames in the prior art. n1 to n6 represent the finite element nodes of the flexible device, X represents the contact points on the surface of the natural human cavity model, and F represents the contact forces. Figure 2a and 2bAs can be seen, in this embodiment of the invention, the contact point between the flexible device and the human body's natural cavity between two frames can be any position within the domain composed of multiple facets in the human body's natural cavity model. In contrast, existing technologies assume that the contact point between the flexible device and the human body's natural cavity remains constant between every two frames, with only the direction and magnitude of the force at the contact point changing, leading to estimation errors. This embodiment of the invention determines the displacement of the flexible device based on the finite element model of the flexible device and the principle of minimum potential energy. Its advantage over existing technologies is that this embodiment eliminates the assumption that the contact point between the flexible device and the human body's natural cavity is fixed. The contact point between the flexible device and the human body's natural cavity can change between two frames, whereas existing technologies assume that the contact point between the flexible device and the human body's natural cavity remains constant between adjacent frames. In practical applications, such as during the insertion of a flexible instrument into a natural cavity, the insertion speed is relatively fast, and the contact point between the flexible instrument and the human body's natural cavity moves a large distance within a frame, resulting in significant changes in the contact point position. This introduces considerable errors in existing technologies. Because existing technologies assume a fixed contact point between the flexible instrument and the human body's natural cavity, determining the displacement of the flexible instrument in these technologies imposes significant restrictions on the relative motion speed between the two. Excessive relative motion speed can lead to clipping and instability. This invention addresses this by establishing inequality constraints on the positional relationship between finite element nodes in the finite element model of the flexible instrument and target surfaces in the three-dimensional model of the human body's natural cavity. This constraint only ensures that the finite element nodes in the finite element model of the flexible instrument and the target surfaces in the three-dimensional model of the human body's natural cavity are not penetrable. The contact and force-bearing positions between the flexible instrument and the human body's natural cavity can be any position within the domain composed of multiple surfaces of the human body's natural cavity model, thereby enabling… Figure 2a The determined displacement and force results of the flexible device are compared with Figure 2b The displacement and force results of flexible devices determined in the prior art are more accurate.

[0089] In this embodiment of the invention, the displacement of the flexible device is determined by the constraints of the positional relationship between the finite element nodes in the finite element model of the flexible device and the target surface in the three-dimensional model of the human body's natural cavity, as well as the principle of minimum potential energy. This method does not rely on the contact force of the flexible device given by the user, nor does it rely on collision detection methods to determine the contact position between the flexible device and the human body's natural cavity. As a result, the method of this embodiment of the invention does not introduce errors from different collision detection methods, does not need to assume that the contact point position between the flexible device and the human body's natural cavity remains unchanged between two frames, and does not restrict the relative motion speed (relative to the surface scale of the human body's natural cavity model) when the flexible device and the human body's natural cavity are in contact. This greatly improves the accuracy of the determined displacement and force results of the flexible device. In other words, the embodiments of the present invention ensure the non-intrusiveness between the finite element model of the flexible device and the natural cavity of the human body by constraining the positional relationship between the finite element nodes in the finite element model of the flexible device and the target surface in the three-dimensional model of the natural cavity of the human body. It does not rely on collision detection methods, that is, it does not require prior assumption of the contact point and collision position between the flexible device and the natural cavity of the human body. The contact and collision positions between the flexible device and the natural cavity of the human body can be directly determined by the method of the embodiments of the present invention. Moreover, the contact and collision positions between the flexible device and the natural cavity of the human body vary between different frames, making the determined displacement and force results of the flexible device more accurate.

[0090] The method described above, based on the constraints of the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the target surface in the three-dimensional model of the human body's natural cavities, and the principle of minimum potential energy, determines the displacement of the flexible instrument corresponding to the minimum potential energy change. This ensures that the determined displacement of the flexible instrument satisfies the environment of the human body's cavity height limitation, making the determined displacement of the flexible instrument more accurate. Consequently, based on the determined displacement of the flexible instrument and the operation instructions of the flexible instrument, the surgeon can accurately operate the flexible instrument, improving the efficiency and safety of the surgery.

[0091] In one embodiment, determining the force result of the flexible device based on its displacement includes:

[0092] The force results of the flexible instrument are determined based on its displacement and stiffness matrix.

[0093] Specifically, a finite element model of the flexible device is established, and the displacement Δx of the flexible device is determined based on the principle of minimum potential energy and target constraints. i Then, the displacement Δx of the flexible device can be determined. i Determine the force results of the flexible device. Optionally, the force results of the flexible device can be determined according to the following formula:

[0094] f i =f i-1 +K(x i-1 )Δx i

[0095] Among them, f i The force results of the flexible device can be expressed as follows: Optionally, the force results of the flexible device are composed and determined based on the force results of multi-dimensional finite element nodes. The force results of the flexible device include the external force or its own weight experienced by the flexible device in contact with the human body's natural cavities and tissues; the f i-1 The Δx represents the force result of the flexible device at time i-1. i The displacement of the flexible device, the K(x) i-1 ) represents the stiffness matrix of a flexible instrument.

[0096] In one embodiment, the stiffness matrix of the flexible device is determined based on the rotation matrix between the local coordinate system and the global coordinate system of the element in the finite element model of the flexible device.

[0097] Specifically, the stiffness matrix of the flexible instrument is a semi-positive definite band-symmetric matrix, which is related to the physical properties of the flexible body and the magnitude of the deformation under large deformation. The efficiency of determining the displacement and force results of the flexible instrument depends on the stiffness matrix K(x) of the flexible instrument. i-1 Considering the dimensions and processing efficiency, optionally, the stiffness matrix of the flexible device is:

[0098]

[0099] Where T e It is the rotation matrix between the unit coordinate system and the global coordinate system, which is related to the displacement x of the flexible device. T needs to be recalculated in each frame. e The value, For element stiffness, a new stiffness matrix is ​​recalculated for each calculation under large deformation conditions. ∑ is a stiffness matrix assembly method that assembles element stiffness matrices into a global stiffness matrix. Optionally, in this embodiment of the invention, the displacement of the flexible instrument is determined by determining the displacement corresponding to the minimum potential energy change of the flexible instrument based on the minimum potential energy change principle. The stiffness matrix of the finite element model of the flexible instrument is redefined between each frame. By combining the minimum potential energy change of the flexible instrument with the stiffness matrix, the deformation and stress of the flexible instrument under large deformation conditions can be accurately simulated, achieving large deformation flexible instrument simulation. This provides accurate information on the displacement and stress results of the flexible instrument for surgery, improving surgical efficiency and safety.

[0100] The method described above determines the displacement of the flexible instrument by determining the displacement corresponding to the minimum potential energy change of the flexible instrument based on the principle of minimum potential energy. It also redetermines the stiffness matrix of the finite element model of the flexible instrument between each frame, thereby accurately simulating the deformation and stress of the flexible instrument under large deformation conditions. This provides doctors with accurate displacement and stress results of the flexible instrument, improving the efficiency and safety of the surgery.

[0101] In one embodiment, after determining the displacement of the flexible device, the method further includes:

[0102] In the case of narrow and bifurcated natural human cavities, the positional relationship between the finite element nodes in the finite element model of the flexible device and the various facets in the three-dimensional model of the natural human cavity is determined based on the determined displacement of the flexible device.

[0103] When at least one surface in the finite element model of a flexible device penetrates another surface in the 3D model of a human body's natural cavities, the target constraints are updated.

[0104] Specifically, when the human body's natural cavities are narrow and bifurcated, the narrowness and bifurcation of these cavities may prevent finite element nodes in the finite element model of the flexible device from penetrating the target surface in the 3D model of the human body's natural cavity. However, penetration may still exist between the finite element nodes in the finite element model of the flexible device and other surfaces in the 3D model of the human body's natural cavity. Therefore, if it is determined that at least one surface in the finite element model of the flexible device penetrates multiple surfaces in the 3D model of the human body's natural cavity, the positional constraints between the flexible device and the human body's natural cavity need to be updated; that is, based on each calculated node... The point penetration surface condition determines the inequality constraint conditions between the finite element nodes in the finite element model of the new flexible instrument and the facets in the 3D model of the human body's natural cavities. The convergence occurs when there is no more penetration between the finite element nodes of the flexible instrument and all facets of the human body's natural cavity model. This ensures that the determined displacement of the flexible instrument meets the actual surgical conditions, making the determined displacement and force results of the flexible instrument more accurate. In turn, it can provide doctors with accurate information on the displacement, force results, and operation prompts of the flexible instrument, ensuring that the flexible instrument can pass through narrow human cavities better. It is suitable for surgeries through natural human cavities such as the bronchus, improving the efficiency and safety of the surgery.

[0105] For example, such as Figure 3The operation prompt method for the flexible device shown first establishes a finite element model of the flexible device. Then, based on the principle of minimum potential energy and the constraints of the positional relationship between the finite element nodes in the finite element model of the flexible device and the target surface in the three-dimensional model of the human body's natural cavities, the displacement of the flexible device can be determined. After determining the displacement of the flexible device, the force result of the flexible device can be determined based on the stiffness matrix of the flexible device. The determined displacement and force results of the flexible device are then displayed, and the operation instruction information of the flexible device is output. Optionally, when the human body's natural cavities are narrow and bifurcated, if it is determined that at least one surface in the finite element model of the flexible instrument penetrates at least one surface in the 3D model of the human body's natural cavity, then the inequality constraints on the positional relationships between the finite element nodes of the flexible instrument's finite element model and the surfaces in the 3D model of the human body's natural cavity are updated until convergence occurs when no further penetration occurs between the finite element nodes of the flexible instrument and all surfaces in the human body's natural cavity model. By updating the constraints on the positional relationships between the finite element nodes of the flexible instrument's finite element model and the surfaces in the 3D model of the human body's natural cavity, the displacement and force results of the flexible instrument determined according to the updated constraints on the positional relationships between the finite element nodes of the flexible instrument's finite element model and the surfaces in the 3D model of the human body's natural cavity can ensure that the flexible instrument can pass through narrow cavities well, making it suitable for simulating surgical procedures in confined spaces such as the bronchus. Figure 3 The inequality constraint condition between the positional relationship between the finite element nodes in the finite element model of the updated flexible instrument (indicated by the black dashed arrow) and the facets in the 3D model of the human body's natural cavities is not a mandatory step, but rather an optimization step taken to determine the displacement and force results of the flexible instrument in the narrow and bifurcated human body's natural cavities. This allows the method in the embodiments of the present invention to more accurately determine the displacement and force results of the flexible instrument in the narrow and bifurcated human body's natural cavities, as well as the flexible instrument operation prompts, ensuring that the flexible instrument can pass through the narrow human body cavities more effectively, thus improving the efficiency and safety of the surgery.

[0106] The method described in the above embodiments, by updating the constraints on the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the facets in the three-dimensional model of the human body's natural cavities, can more accurately determine the displacement and force results of the flexible instrument in narrow and bifurcated human body cavities, as well as the operation prompts of the flexible instrument. This ensures that the flexible instrument can pass through narrow human body cavities more effectively, thereby improving the efficiency and safety of the surgery.

[0107] The operation prompting device for the flexible device provided by the present invention is described below. The operation prompting device for the flexible device described below can be referred to in correspondence with the operation prompting method for the flexible device described above.

[0108] Figure 4 This is a structural schematic diagram of the operation prompting device for the flexible medical device provided by the present invention. The operation prompting device for the flexible medical device provided in this embodiment includes:

[0109] Module 710 is used to create the finite element model of the flexible device; the finite element model of the flexible device includes multiple finite element nodes.

[0110] The first determining module 720 is used to determine the displacement of the flexible device based on the principle of minimum potential energy and target constraints. The target constraints are used to represent the constraints on the positional relationship between the finite element nodes in the finite element model of the flexible device and the target surface in the three-dimensional model of the human body's natural cavities. The displacement of the flexible device includes the displacement of each finite element node.

[0111] The second determining module 730 is used to determine the force result of the flexible device based on the displacement of the flexible device;

[0112] The prompting module 740 is used to output operation instruction information for the flexible instrument based on the determined displacement and force results of the flexible instrument; the operation instruction information for the flexible instrument is used to indicate the method of performing surgery on the natural cavities of the human body through the flexible instrument.

[0113] Optionally, the first determining module 720 is specifically used to: determine the displacement of the flexible device when the potential energy change of the flexible device is minimized and the positional relationship between the flexible device and the human body's natural cavity satisfies the target constraint condition; the target constraint condition is that the finite element nodes in the finite element model of the flexible device and the target surface in the three-dimensional model of the human body's natural cavity cannot penetrate each other.

[0114] Optionally, the first determining module 720 is specifically used to: determine the displacement of the flexible device using the following formula:

[0115]

[0116] or

[0117] Ax i =b i

[0118] Δx represents the minimum change in potential energy of a flexible device. i K(x) represents the displacement of the flexible instrument from time i-1 to time i; i-1 ) represents the stiffness matrix of the flexible instrument; x i-1 denoted as the finite element node displacement at time i-1; F represents the target external force acting on the flexible device, which does not include the contact force between the flexible device and the human body's natural cavities. or Indicate the objective constraints; The normal vector of the contact patch between the flexible medical device and the natural cavities of the human body; x i S represents the finite element nodal displacement at time i; i X represents the generalized coordinate position of a flexible instrument in its undeformed state; X0 represents any position on the plane corresponding to the three-dimensional model of a natural human cavity; Ax i =b i Denotes the first constraint condition; A denotes the projection matrix; b i Indicates the target location of the flexible instrument; x i This represents the displacement of the finite element node at time i.

[0119] Optionally, the second determining module 730 is specifically used to: determine the force result of the flexible device based on the displacement of the flexible device, including:

[0120] The force results of the flexible instrument are determined based on its displacement and stiffness matrix.

[0121] Optionally, the second determining module 730 is further configured to: determine the stiffness matrix of the flexible device based on the rotation matrix between the local coordinate system and the global coordinate system of the element in the finite element model of the flexible device.

[0122] Optionally, the second determining module 730 is further configured to: when the human body's natural cavity is a narrow bifurcated cavity, determine the positional relationship between the finite element nodes in the finite element model of the flexible device and each facet in the three-dimensional model of the human body's natural cavity based on the determined displacement of the flexible device.

[0123] When at least one surface in the finite element model of a flexible device penetrates another surface in the 3D model of a human body's natural cavities, the target constraints are updated.

[0124] The apparatus of this invention is used to execute the method in any of the foregoing method embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0125] Figure 5A schematic diagram of the physical structure of an electronic device is provided. This electronic device may include a processor 810, a communication interface 820, a memory 830, and a communication bus 840. The processor 810, communication interface 820, and memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute an operation prompting method for a flexible instrument. This method includes: establishing a finite element model of the flexible instrument; the finite element model of the flexible instrument includes multiple finite element nodes; determining the displacement of the flexible instrument based on the principle of minimum potential energy and target constraints; the target constraints represent the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the target surface in the three-dimensional model of the human body's natural cavities; the displacement of the flexible instrument includes the displacement of each finite element node; determining the force result of the flexible instrument based on the displacement of the flexible instrument; and outputting operation instruction information for the flexible instrument based on the determined displacement and force result of the flexible instrument; the operation instruction information of the flexible instrument is used to indicate the method of performing surgery on the human body's natural cavities using the flexible instrument.

[0126] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0127] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, which, when executed by a computer, enable the computer to execute the operation prompting method for the flexible device provided by the above methods, the method comprising: establishing a finite element model of the flexible device; the finite element model of the flexible device comprising multiple finite element nodes; determining the displacement of the flexible device based on the principle of minimum potential energy and target constraints; the target constraints representing the positional relationship between the finite element nodes in the finite element model of the flexible device and the target surface in the three-dimensional model of the human body's natural cavity; the displacement of the flexible device comprising the displacement of each finite element node; determining the force result of the flexible device based on the displacement of the flexible device; outputting operation instruction information of the flexible device based on the determined displacement and force result of the flexible device; the operation instruction information of the flexible device indicating the method of performing surgery on the human body's natural cavity through the flexible device.

[0128] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the operation prompting method for each of the above-mentioned flexible instruments. The method includes: establishing a finite element model of the flexible instrument; the finite element model of the flexible instrument includes multiple finite element nodes; determining the displacement of the flexible instrument based on the principle of minimum potential energy and target constraints; the target constraints represent constraints on the positional relationship between the finite element nodes in the finite element model of the flexible instrument and target patches in the three-dimensional model of the human body's natural cavities; the displacement of the flexible instrument includes the displacement of each finite element node; determining the force result of the flexible instrument based on the displacement of the flexible instrument; and outputting operation instruction information for the flexible instrument based on the determined displacement and force result of the flexible instrument; the operation instruction information for the flexible instrument is used to indicate the method of performing surgery on the human body's natural cavities using the flexible instrument.

[0129] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0130] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technologies; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for providing operation prompts for a flexible medical device, characterized in that, include: A finite element model of the flexible device is established; the finite element model of the flexible device includes multiple finite element nodes; Based on the principle of minimum potential energy and target constraints, the displacement of the flexible device is determined; The target constraint is used to represent the positional relationship between the finite element nodes in the finite element model of the flexible device and the target surface in the three-dimensional model of the human body's natural cavities. The displacement of the flexible device includes the displacement of each finite element node. The determination of the displacement of the flexible device based on the principle of minimum potential energy and target constraints includes: The displacement of the flexible instrument is determined using the following formula: ； ； ； The This represents the minimum change in potential energy of a flexible device; the... Indicating flexible devices Time to Displacement at time; the The stiffness matrix represents the stiffness of the flexible device; for The displacement of finite element nodes at any given time; F represents the target external force acting on the flexible device, which does not include the contact force between the flexible device and the natural cavities of the human body; Represents the target constraint conditions; the This represents the normal vector of the contact patch between the flexible device and the body's natural cavities; the... express The finite element nodal displacements of the flexible instrument at any given time; This represents the generalized coordinate position of a flexible device in its undeformed state; the... The three-dimensional model representing the natural cavities of the human body corresponds to any position on the plane; the The first constraint condition is represented by A; A represents the projection matrix; the... Indicates the target position of the flexible device; the Let i represent the finite element nodal displacement of the flexible device at time i; The force result of the flexible instrument is determined based on its displacement. Based on the determined displacement and force results of the flexible instrument, operation instruction information of the flexible instrument is output; the operation instruction information of the flexible instrument is used to indicate the method of performing surgery on the natural cavities of the human body through the flexible instrument.

2. The operation prompting method for the flexible device according to claim 1, characterized in that, The determination of the displacement of the flexible device based on the principle of minimum potential energy and target constraints includes: The displacement of the flexible device is determined when the potential energy change of the flexible device is minimized and the positional relationship between the flexible device and the human body's natural cavity satisfies the target constraint condition; the target constraint condition is that the finite element nodes in the finite element model of the flexible device and the target surface in the three-dimensional model of the human body's natural cavity are not penetrable.

3. The operation prompting method for the flexible device according to claim 1, characterized in that, The determination of the force result of the flexible device based on its displacement includes: The force result of the flexible device is determined based on the displacement of the flexible device and the stiffness matrix of the flexible device.

4. The operation prompting method for the flexible device according to claim 3, characterized in that, The method further includes: The stiffness matrix of the flexible device is determined by the rotation matrix between the local coordinate system and the global coordinate system of the element in the finite element model.

5. The operation prompting method for the flexible device according to claim 1, characterized in that, After determining the displacement of the flexible device, the following steps are also included: In the case where the natural human cavity is a narrow bifurcated cavity, based on the determined displacement of the flexible device, the positional relationship between the finite element nodes in the finite element model of the flexible device and each facet in the three-dimensional model of the natural human cavity is determined. The target constraint is updated when at least one surface in the finite element model of the flexible device penetrates between a finite element node and multiple surfaces in the three-dimensional model of the human body's natural cavities.

6. An operation prompting device for a flexible medical device, characterized in that, include: A module is established to create a finite element model of the flexible device; the finite element model of the flexible device includes multiple finite element nodes. The first determining module is used to determine the displacement of the flexible device based on the principle of minimum potential energy and target constraints. The target constraint is used to represent the positional relationship between the finite element nodes in the finite element model of the flexible device and the target surface in the three-dimensional model of the human body's natural cavities. The displacement of the flexible device includes the displacement of each finite element node. The determination of the displacement of the flexible device based on the principle of minimum potential energy and target constraints includes: The displacement of the flexible instrument is determined using the following formula: ； ； ； The This represents the minimum change in potential energy of a flexible device; the... Indicating flexible devices Time to Displacement at time; the The stiffness matrix represents the stiffness of the flexible device; for The displacement of finite element nodes at any given time; F represents the target external force acting on the flexible device, which does not include the contact force between the flexible device and the natural cavities of the human body; Represents the target constraint conditions; the This represents the normal vector of the contact patch between the flexible device and the body's natural cavities; the... express The finite element nodal displacements of the flexible instrument at any given time; This represents the generalized coordinate position of a flexible device in its undeformed state; the... The three-dimensional model representing the natural cavities of the human body corresponds to any position on the plane; the The first constraint condition is represented by A; A represents the projection matrix; the... Indicates the target position of the flexible device; the Let i represent the finite element nodal displacement of the flexible device at time i; The second determining module is used to determine the force result of the flexible device based on the displacement of the flexible device; The prompting module is used to output operation instruction information for the flexible instrument based on the determined displacement and force result of the flexible instrument; the operation instruction information for the flexible instrument is used to indicate the method of performing surgery on the natural cavities of the human body through the flexible instrument.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the operation prompting method for the flexible device as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the operation prompting method for the flexible device as described in any one of claims 1 to 5.

9. A computer program product having executable instructions stored thereon, characterized in that, When executed by the processor, the instruction causes the processor to implement the steps of the operation prompting method for the flexible device as described in any one of claims 1 to 5.

Images