Force feedback methods, devices, electronic equipment and storage media

Abstract

This invention discloses a force feedback method, device, electronic device, and storage medium. The method includes: determining a target surgical instrument; during a simulated surgery performed on a skin tissue model using the target surgical instrument, determining a first real-time position of the pierceable tissue entity at a target time point, and determining a first reference position of the moving entity on the tissue at a reference time point corresponding to the target time point; determining a second real-time position of the moving entity on the tissue at the target time point based on the first real-time position information and the first reference position information; determining a target feedback force of the target surgical instrument at the target time point based on the first and second real-time position information; and providing force feedback to the object operating the target surgical instrument based on the target feedback force. Based on this invention, force feedback-based simulated surgery can achieve a tactile interactive perception that more closely resembles actual surgery.

Description

Technical Field

[0001] This invention relates to the field of computer application technology, and in particular to a force feedback method, device, electronic device, and storage medium. Background Technology

[0002] It is now common practice to use models simulating human tissue to train relevant personnel before actual surgery. Highly realistic preoperative training can effectively reduce the risk of accidents during actual surgery.

[0003] In related technologies, force feedback for simulated surgical objects during simulated surgery is usually achieved through traditional force feedback models. However, phenomena such as sudden changes in force feedback and discontinuous force feedback often occur, resulting in a non-realistic tactile experience. In other words, simulated surgery based on traditional force feedback methods is difficult to achieve interactive perception that closely resembles actual surgery. Summary of the Invention

[0004] This invention provides a force feedback method, device, electronic device, and storage medium to solve the technical problem that simulated surgery based on traditional force feedback methods is difficult to achieve interactive perception that closely resembles actual surgery.

[0005] According to one aspect of the present invention, a force feedback method is provided, the method comprising:

[0006] Identify a target surgical instrument, wherein the target surgical instrument includes a tissue-penetrating entity and a tissue-moving entity connected by a spring;

[0007] When performing a target simulated surgery on a skin tissue model using a target surgical instrument, the first real-time position information of the puncturable tissue entity at the target time point is determined, and the first reference position information of the moving entity on the tissue at the reference time point corresponding to the target time point is determined.

[0008] The second real-time location information of the mobile entity on the organization at the target time point is determined based on the first real-time location information and the first reference location information.

[0009] The target feedback force of the target surgical instrument at the target time point is determined based on the first real-time location information and the second real-time location information.

[0010] Force feedback is provided to the object operating the target surgical instrument based on the target feedback force.

[0011] According to another aspect of the present invention, a force feedback device is provided, the device comprising:

[0012] A surgical instrument determination module is used to determine a target surgical instrument, wherein the target surgical instrument includes a tissue-penetrating entity and a tissue-moving entity connected by a spring;

[0013] The location information acquisition module is used to determine the first real-time location information of the puncturable tissue entity at the target time point when performing a target simulated surgery on a skin tissue model using a target surgical instrument, and to determine the first reference location information of the moving entity on the tissue at the reference time point corresponding to the target time point.

[0014] The location information calculation module is used to determine the second real-time location information of the mobile entity on the organization at the target time point based on the first real-time location information and the first reference location information;

[0015] A feedback force determination module is used to determine the target feedback force of the target surgical instrument at the target time point based on the first real-time position information and the second real-time position information;

[0016] The force feedback module is used to provide force feedback to the object operating the target surgical instrument based on the target feedback force.

[0017] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0018] At least one processor; and

[0019] A memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the force feedback method according to any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the force feedback method according to any embodiment of the present invention.

[0022] The technical solution of this invention involves determining a target surgical instrument, wherein the target surgical instrument includes a tissue-penetrating entity and a tissue-moving entity connected by a spring; when performing a target simulated surgery on a skin tissue model using the target surgical instrument, determining the first real-time position information of the tissue-penetrating entity at a target time point, and determining the first reference position information of the tissue-moving entity at a reference time point corresponding to the target time point; determining the second real-time position information of the tissue-moving entity at the target time point based on the first real-time position information and the first reference position information; determining the target feedback force of the target surgical instrument at the target time point based on the first real-time position information and the second real-time position information; and providing force feedback to the object operating the target surgical instrument based on the target feedback force. This achieves the effect of accurately and continuously calculating the force between the target surgical instrument and the skin tissue model during the target simulated surgery and feeding this force back to the object operating the target surgical instrument in real time. Based on this technical solution, the object can continuously perceive tactile stimulation during the target simulated surgery, achieving a technical effect of interactive perception that closely resembles actual surgery. This invention can be applied to simulated surgical training before actual surgery.

[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of a force feedback method provided according to Embodiment 1 of the present invention;

[0026] Figure 2 This is a scene diagram of a target simulated surgery performed on a skin tissue model using a target surgical instrument, according to an embodiment of the present invention.

[0027] Figure 3 These are example diagrams of the target surface under two different collision surface constraint information provided in embodiments of the present invention;

[0028] Figure 4 This is a flowchart of calculating second real-time location information according to an embodiment of the present invention;

[0029] Figure 5 This is a flowchart of a force feedback method provided according to Embodiment 2 of the present invention;

[0030] Figure 6 This is a comparison diagram of the second real-time position information update between the presence and absence of Coulomb friction in the same simulated surgical scenario provided by an embodiment of the present invention;

[0031] Figure 7 This is a schematic diagram of the structure of a force feedback device according to Embodiment 3 of the present invention;

[0032] Figure 8 This is a schematic diagram of the structure of an electronic device that implements the force feedback method of the present invention. Detailed Implementation

[0033] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0035] Example 1

[0036] Figure 1 The flowchart illustrates a force feedback method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where force feedback is performed using a force feedback model. The method can be executed by a force feedback device, which can be implemented in hardware and / or software and can be configured in a computer. Figure 1 As shown, the method includes:

[0037] S110. Determine the target surgical instrument, wherein the target surgical instrument includes a tissue-penetrating entity and a tissue-moving entity connected by a spring.

[0038] The target surgical instrument can be understood as a surgical instrument. In this embodiment of the invention, the target surgical instrument may include a tissue-penetrating entity and a tissue-moving entity connected by a spring, such as... Figure 2 As shown, Figure 2 This is a scene diagram illustrating a simulated surgical procedure performed on a skin tissue model using a target surgical instrument, according to an embodiment of the present invention. Figure 2 An entity at a proxy location can represent a mobile entity on the organization, and an entity at a physical location can represent a mobile entity on the organization.

[0039] In this embodiment of the invention, during the execution of a target simulated surgery on a skin tissue model using a target surgical instrument, the punctureable tissue entity can penetrate into the interior of the skin tissue model, and after the punctureable tissue entity penetrates into the interior of the skin tissue model, the position of the punctureable tissue entity no longer changes.

[0040] In this embodiment of the invention, the tissue-moving entity does not pierce the skin tissue model during the simulated surgery. After the position of the tissue-moving entity is fixed, the tissue-moving entity can slide along the target surface of the skin tissue model to perform the target simulated surgery.

[0041] In this embodiment of the invention, both the pierceable tissue entity and the tissue-moving entity can be a spherical entity.

[0042] S120. When performing a target simulated surgery on a skin tissue model using a target surgical instrument, determine the first real-time position information of the puncturable tissue entity at the target time point, and determine the first reference position information of the moving entity on the tissue at the reference time point corresponding to the target time point.

[0043] The skin tissue model can be understood as a physical model that simulates human skin tissue.

[0044] Optionally, the target simulated surgery can be a simulated surgery in which the punctureable tissue entity is inserted into the skin tissue model and the moving entity on the tissue is manipulated to slide on the target surface of the skin tissue model.

[0045] The target surface may be the insertion surface of the pierceable tissue entity.

[0046] In this embodiment of the invention, the target time point can be a real-time time point or the current time point. The reference time point can be the previous historical time point corresponding to the target time point. The time interval between the target time point and the reference time point can be related to the force feedback update frequency preset by this invention, and is not specifically limited here.

[0047] The first real-time location information can be the position coordinates of the pierceable tissue entity in the target coordinate system at the target time point. The target coordinate system can be a preset three-dimensional coordinate system, and correspondingly, the first real-time location information can be three-dimensional position coordinates.

[0048] The first reference position information may be the position coordinates of the moving entity on the organization in the target coordinate system at a reference time point. Related to the first real-time position information, the first real-time position information may also be three-dimensional position coordinates.

[0049] S130. Determine the second real-time location information of the mobile entity on the organization at the target time point based on the first real-time location information and the first reference location information.

[0050] In this embodiment of the invention, the first reference position information of the historical moving entity on the tissue can be stored during the previous force feedback cycle and can be directly obtained in real time. The second real-time position information of the moving entity on the tissue is difficult to obtain directly and needs to be calculated based on the first real-time position information and the first reference position information.

[0051] The second real-time location information can be the position coordinates of the moving entity on the organization in the target coordinate system at the target time point.

[0052] Optionally, determining the second real-time location information of the mobile entity on the organization at the target time point based on the first real-time location information and the first reference location information includes:

[0053] Determine the connecting line segment between the moving entity on the organization under the first real-time location information and the moving entity on the organization under the first reference location information;

[0054] In the case where there is an intersection between the connecting line segment and the target surface of the skin tissue model, the collision surface constraint information of the target surface is determined;

[0055] The second real-time position information of the moving entity on the tissue at the target time point is determined based on the collision surface constraint information and the first real-time position information.

[0056] Optionally, determining the second real-time location information of the mobile entity on the organization at the target time point based on the first real-time location information and the first reference location information further includes:

[0057] If there is an intersection between the connecting line segment and the target surface of the skin tissue model, the first real-time position information is used as the second real-time position information of the moving entity on the tissue at the target time point.

[0058] Wherein, the collision surface constraint information may be the contact information between the moving entity on the tissue and the target surface. In this embodiment of the invention, during the target simulation surgery, the target contact points between the moving entity on the tissue and the target surface may be one or more. The collision surface constraint information may be information related to the target contact points. The collision surface constraint information may also be information related to the attribute coefficients of the target surface. For example, as shown... Figure 3 As shown, Figure 3 These are example diagrams of the target surface under two different collision surface constraint information provided in embodiments of the present invention.

[0059] Figure 4 This is a flowchart illustrating the calculation of second real-time location information according to an embodiment of the present invention. Figure 4 As shown, optionally, the process of determining the second real-time location information of the mobile entity on the organization at the target time point based on the first real-time location information and the first reference location information can be as follows:

[0060] Before explaining the process of calculating the second real-time position information, the purpose of calculating the second real-time position information is briefly introduced. In each cycle of the present invention, the feedback force of the tactile feedback needs to be updated to update the real-time position of the moving entity on the tissue in order to ensure the continuity of the feedback force and achieve consistency between the tactile effect and the visual effect during the simulated surgery.

[0061] Optionally, the calculation and updating of the second real-time position information can be implemented using the Dynamic Proxy algorithm. Specifically, after the Device (representing the pierceable tissue entity) pierces into the skin tissue model (e.g., a kidney tissue model), the line segment connecting the reference position of the Proxy (representing the moving entity on the tissue) of the target surgical instrument at the previous update time point and the real-time position of the Device at the current time point is determined, and collision detection is performed (determining whether the connecting line segment intersects with the target surface); if a collision occurs (intersection exists), the second real-time position information is calculated and updated according to the collision surface constraints, i.e., the real-time position coordinates of the Proxy of the surgical instrument; if no collision occurs, the first real-time position information is used as the second real-time position information. In this invention, the real-time position of the Proxy in the nth tactile feedback loop is calculated based on the reference position of the Proxy in the (n-1)th tactile feedback loop.

[0062] Optionally, determining the second real-time position information of the moving entity on the tissue at the target time point based on the collision surface constraint information and the first real-time position information includes:

[0063] The second real-time position information of the moving entity on the tissue at the target time point is obtained by calculating the collision surface constraint information and the first real-time position information using the Lagrange multiplier method.

[0064] The Lagrange multiplier method can be understood as a method for finding the extrema of a multivariate function when its variables are constrained by one or more conditions.

[0065] Specifically, the calculation of the second real-time location information can be achieved based on the following objective formula:

[0066]

[0067] (A2x+B2y+C2z+D2)+l3(A3x+B3y+C3z+D3);

[0068]

[0069] Where, x p y p z p A represents the first real-time location information; x, y, z represent the second real-time location information; A i B i C i D i The values ​​represent the collision surface constraint information; l1, l2, and l3 represent the Lagrange operators; and L represents an intermediate value that can be used to determine the second real-time position information.

[0070] The construction of the above objective formula will be explained below:

[0071] When the spring constant of the spring connecting the penetrable tissue entity and the movable entity on the tissue is 1, the elastic potential energy can be calculated based on the following formula:

[0072]

[0073] Where, x p y p z p Let x represent the first real-time location information, x, y, and z represent the second real-time location information, and Q represent the elastic potential energy.

[0074] When the target surgical instrument collides with the skin tissue model, the following formula is constructed related to the collision surface (which may be the target surface) to further calculate the second real-time position information:

[0075] A n x+B n y+C n z+D n =0

[0076] Among them, A n B n C n D n The values ​​represent the collision surface constraint information, and x, y, and z represent the second real-time position information.

[0077] In summary, based on the principle that the smaller the potential energy, the more stable it is, the problem of solving the second real-time position information is transformed into the problem of constructing the Lagrange function and finding its extremum, so as to construct the above objective formula.

[0078] S140. Determine the target feedback force of the target surgical instrument at the target time point based on the first real-time location information and the second real-time location information.

[0079] The target feedback force can be understood as the interaction force between the target surgical instrument and the skin tissue model during the target simulated surgery. In this embodiment of the invention, the target feedback force can be fed back to the object being operated on by the target surgical instrument.

[0080] S150. Force feedback is provided to the object operating the target surgical instrument based on the target feedback force.

[0081] The operation object can be understood as the object that operates the target surgical instrument. In this embodiment of the invention, the operation object can be preset according to the needs of the scenario, and is not specifically limited here. Optionally, the operation object can be a user.

[0082] The technical solution of this invention involves determining a target surgical instrument, wherein the target surgical instrument includes a tissue-penetrating entity and a tissue-moving entity connected by a spring; when performing a target simulated surgery on a skin tissue model using the target surgical instrument, determining the first real-time position information of the tissue-penetrating entity at a target time point, and determining the first reference position information of the tissue-moving entity at a reference time point corresponding to the target time point; determining the second real-time position information of the tissue-moving entity at the target time point based on the first real-time position information and the first reference position information; determining the target feedback force of the target surgical instrument at the target time point based on the first real-time position information and the second real-time position information; and providing force feedback to the object operating the target surgical instrument based on the target feedback force. This achieves the effect of accurately and continuously calculating the force between the target surgical instrument and the skin tissue model during the target simulated surgery and feeding this force back to the object operating the target surgical instrument in real time. Based on this technical solution, the object can continuously perceive tactile stimulation during the target simulated surgery, achieving a technical effect of interactive perception that closely resembles actual surgery. This invention can be applied to simulated surgical training before actual surgery.

[0083] Example 2

[0084] Figure 5 This is a flowchart of a force feedback method provided in Embodiment 2 of the present invention. This embodiment refines the method described in the above embodiment for determining the target feedback force of the target surgical instrument at the target time point based on the first real-time position information and the second real-time position information. Figure 5 As shown, the method includes:

[0085] S210. Identify the target surgical instrument.

[0086] S220. When performing a target simulated surgery on a skin tissue model using a target surgical instrument, determine the first real-time position information of the puncturable tissue entity at the target time point, and determine the first reference position information of the moving entity on the tissue at the reference time point corresponding to the target time point.

[0087] S230. Determine the second real-time location information of the mobile entity on the organization at the target time point based on the first real-time location information and the first reference location information.

[0088] S240. Calculate the first feedback force based on the first real-time location information and the second real-time location information.

[0089] In this embodiment of the invention, the first feedback force can be a support force, such as... Figure 2 The F shown n The second feedback force can be Coulomb friction, such as... Figure 2 The F shown f The target feedback force is determined based on the first feedback force and the second feedback force, such as... Figure 2 The F shown user .

[0090] Optionally, calculating the first feedback force based on the first real-time location information and the second real-time location information includes:

[0091] The target depth at which the penetrable tissue entity penetrates the skin tissue model is determined based on the first real-time location information and the second real-time location information.

[0092] Determine the angle information between the target surgical instrument and the skin tissue model, and determine the tissue stiffness coefficient of the skin tissue model;

[0093] The first feedback force is determined based on the target depth, the included angle information, and the tissue stiffness coefficient.

[0094] The target depth can be the depth in a direction perpendicular to the target surface of the skin tissue model.

[0095] In this embodiment of the invention, the target surgical instrument may be inserted obliquely into the skin tissue model or perpendicularly into the skin tissue model. Clearly, there is an angle between the target surgical instrument and the skin tissue model. Optionally, the angle information may be the cosine of the angle between the target surgical instrument (the direction in which the moving entity on the tissue points to the pierceable tissue entity) and a direction perpendicular to the target surface.

[0096] The tissue stiffness coefficient can be a coefficient of the stiffness property of the skin tissue model. The tissue stiffness coefficients of different skin tissue models can be different or the same.

[0097] Specifically, the calculation of the first feedback force can be based on the following formula:

[0098] F n =kΔxcosθ=k(x t,Device -x t,Proxy cosθ

[0099] Among them, F n Let x represent the first feedback force, k represent the tissue stiffness coefficient, Δx represent the target depth, cosθ represent the included angle information, and x represent the first feedback force. t,Device Indicates the first real-time location information, x t,Proxy This indicates the second real-time location information.

[0100] S250. Determine the Coulomb friction coefficient corresponding to the target surface of the skin tissue model, and determine the second feedback force based on the first feedback force and the Coulomb friction coefficient.

[0101] The Coulomb friction coefficient can be related to the material type of the target surface. The Coulomb friction coefficient can be preset according to scenario requirements and is not specifically limited here.

[0102] Specifically, the calculation of the second feedback force can be based on the following formula:

[0103] F f =μF n

[0104] Among them, F f F represents the second feedback force, μ represents the Coulomb friction coefficient, and F represents the second feedback force. n This represents the first feedback force.

[0105] S260. Determine the target feedback force of the target surgical instrument at the target time point based on the first feedback force and the second feedback force.

[0106] Specifically, the calculation of the target feedback force can be based on the following formula:

[0107]

[0108] in, The target feedback force represents the vector. The first feedback force represents the vector. The second feedback force represents the vector.

[0109] S270. Force feedback is provided to the object operating the target surgical instrument based on the target feedback force.

[0110] The technical solution of this invention involves calculating a first feedback force based on the first and second real-time position information; determining the Coulomb friction coefficient corresponding to the target surface of the skin tissue model; determining a second feedback force based on the first feedback force and the Coulomb friction coefficient; and determining the target feedback force of the target surgical instrument at the target time point based on the first and second feedback forces. This invention uses Coulomb friction as part of the feedback force for real-time calculation and feedback, which can make the tactile perception of the feedback object more realistic. Figure 6 , Figure 6 This is a comparison diagram of the second real-time position information update between the presence and absence of Coulomb friction in the same simulated surgical scenario provided by an embodiment of the present invention.

[0111] The following is a general explanation of the force feedback method. This invention is essentially based on a target-based force feedback model; that is, the determination and real-time updating of position information and feedback force are based on this target-based force feedback model. Traditional force feedback models suffer from discontinuous forces generated when surgical instruments are in gaps (e.g., the instant a surgical instrument pierces a tissue model). Furthermore, when a moving entity slides along the surface of a skin tissue model, it typically generates abruptly changing feedback forces, resulting in an unrealistic tactile experience. Consequently, during simulated surgery (actual pre-operative training, where the surgical object can be a tissue model), the visual and tactile experiences cannot be highly aligned when using force feedback based on traditional force feedback models, making it difficult to achieve the expected pre-operative training results. The target-based force feedback model of this invention, by employing real-time updates of the position information of moving entities on the tissue and combining the calculation and feedback of Coulomb friction, can achieve continuous and accurate real-time force feedback. This improves the tactile simulation effect of pre-operative simulated surgery based on the target-based force feedback model, achieving the expected pre-operative training results.

[0112] This invention combines the force feedback model of the target with the surgical instruments of the surgical robot, which can realize the interactive fusion perception of visual perception and tactile perception in simulated surgery, providing an ideal preoperative training scenario.

[0113] Example 3

[0114] Figure 7 This is a schematic diagram of a force feedback device provided in Embodiment 3 of the present invention. Figure 7 As shown, the device includes: a surgical instrument determination module 310, a position information acquisition module 320, a position information calculation module 330, a feedback force determination module 340, and a force feedback module 350.

[0115] The system includes: a surgical instrument determination module 310 for determining a target surgical instrument, wherein the target surgical instrument includes a tissue-penetrating entity and a tissue-moving entity connected by a spring; a position information acquisition module 320 for determining, when performing a target simulated surgery on a skin tissue model using the target surgical instrument, a first real-time position of the tissue-penetrating entity at a target time point, and a first reference position of the tissue-moving entity at a reference time point corresponding to the target time point; a position information calculation module 330 for determining, based on the first real-time position information and the first reference position information, a second real-time position of the tissue-moving entity at the target time point; a feedback force determination module 340 for determining, based on the first real-time position information and the second real-time position information, a target feedback force of the target surgical instrument at the target time point; and a force feedback module 350 for providing force feedback to the object operating the target surgical instrument based on the target feedback force.

[0116] The technical solution of this invention involves determining a target surgical instrument, wherein the target surgical instrument includes a tissue-penetrating entity and a tissue-moving entity connected by a spring; when performing a target simulated surgery on a skin tissue model using the target surgical instrument, determining the first real-time position information of the tissue-penetrating entity at a target time point, and determining the first reference position information of the tissue-moving entity at a reference time point corresponding to the target time point; determining the second real-time position information of the tissue-moving entity at the target time point based on the first real-time position information and the first reference position information; determining the target feedback force of the target surgical instrument at the target time point based on the first real-time position information and the second real-time position information; and providing force feedback to the object operating the target surgical instrument based on the target feedback force. This achieves the effect of accurately and continuously calculating the force between the target surgical instrument and the skin tissue model during the target simulated surgery and feeding this force back to the object operating the target surgical instrument in real time. Based on this technical solution, the object can continuously perceive tactile stimulation during the target simulated surgery, achieving a technical effect of interactive perception that closely resembles actual surgery. This invention can be applied to simulated surgical training before actual surgery.

[0117] Optionally, the target simulated surgery can be a simulated surgery in which the punctureable tissue entity is inserted into the skin tissue model and the moving entity on the tissue is manipulated to slide on the target surface of the skin tissue model.

[0118] Optionally, the location information calculation module 330 includes: an entity connection unit, a constraint information determination unit, and a first calculation unit;

[0119] The entity connection unit is used to determine the connection line segment between the organization-moving entity under the first real-time location information and the organization-moving entity under the first reference location information.

[0120] The constraint information determination unit is used to determine the collision surface constraint information of the target surface when there is an intersection between the connecting line segment and the target surface of the skin tissue model.

[0121] The first calculation unit is used to determine the second real-time position information of the moving entity on the tissue at the target time point based on the collision surface constraint information and the first real-time position information.

[0122] Optionally, the location information calculation module 330 further includes: a second calculation unit, used to use the first real-time location information as the second real-time location information of the moving entity on the tissue at the target time point when there is an intersection between the connecting line segment and the target surface of the skin tissue model.

[0123] Optionally, the first computing unit is specifically used for:

[0124] The second real-time position information of the moving entity on the tissue at the target time point is obtained by calculating the collision surface constraint information and the first real-time position information using the Lagrange multiplier method.

[0125] Optionally, the feedback force determination module 340 includes: a first feedback force determination unit, a second feedback force determination unit, and a target feedback force determination unit;

[0126] The first feedback force determination unit is used to calculate the first feedback force based on the first real-time location information and the second real-time location information.

[0127] The second feedback force determining unit is used to determine the Coulomb friction coefficient corresponding to the target surface of the skin tissue model, and to determine the second feedback force based on the first feedback force and the Coulomb friction coefficient.

[0128] The target feedback force determination unit is used to determine the target feedback force of the target surgical instrument at the target time point based on the first feedback force and the second feedback force.

[0129] Optionally, the first feedback force determining unit is specifically used for:

[0130] The target depth at which the penetrable tissue entity penetrates the skin tissue model is determined based on the first real-time location information and the second real-time location information.

[0131] Determine the angle information between the target surgical instrument and the skin tissue model, and determine the tissue stiffness coefficient of the skin tissue model;

[0132] The first feedback force is determined based on the target depth, the included angle information, and the tissue stiffness coefficient.

[0133] The force feedback device provided in the embodiments of the present invention can execute the force feedback method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method.

[0134] Example 4

[0135] Figure 8 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0136] like Figure 8 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0137] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0138] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as force feedback methods.

[0139] In some embodiments, the force feedback method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the force feedback method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the force feedback method by any other suitable means (e.g., by means of firmware).

[0140] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0141] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0142] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0143] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0144] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0145] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0146] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0147] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A force feedback method, characterized by, The method comprises: determining a target surgical instrument, wherein the target surgical instrument comprises a tissue-penetrable entity and a tissue-surface-moving entity connected by a spring; when a target simulation surgery is performed on a skin tissue model by the target surgical instrument, determining first real-time position information of the tissue-penetrable entity at a target time point, and determining first reference position information of the tissue-surface-moving entity at a reference time point corresponding to the target time point; determining second real-time position information of the tissue-surface-moving entity at the target time point according to the first real-time position information and the first reference position information; determining a target feedback force of the target surgical instrument at the target time point according to the first real-time position information and the second real-time position information; performing force feedback on an operating object operating the target surgical instrument based on the target feedback force; wherein the determining of the second real-time position information of the tissue-surface-moving entity at the target time point according to the first real-time position information and the first reference position information comprises: determining a connecting line segment between the tissue-surface-moving entity under the first real-time position information and the tissue-surface-moving entity under the first reference position information; when there is an intersection between the connecting line segment and a target surface of the skin tissue model, determining collision surface constraint information of the target surface; determining the second real-time position information of the tissue-surface-moving entity at the target time point according to the collision surface constraint information and the first real-time position information; wherein the determining of the target feedback force of the target surgical instrument at the target time point according to the first real-time position information and the second real-time position information comprises: calculating a first feedback force according to the first real-time position information and the second real-time position information; determining a Coulomb friction coefficient corresponding to the target surface of the skin tissue model, and determining a second feedback force according to the first feedback force and the Coulomb friction coefficient; determining the target feedback force of the target surgical instrument at the target time point according to the first feedback force and the second feedback force.

2. The method of claim 1, wherein, The target simulation surgery can be a simulation surgery of penetrating the tissue-penetrable entity into the skin tissue model and sliding the tissue-surface-moving entity on the target surface of the skin tissue model.

3. The method of claim 1, wherein, The determining of the second real-time position information of the tissue-surface-moving entity at the target time point according to the collision surface constraint information and the first real-time position information comprises: calculating the collision surface constraint information and the first real-time position information by the Lagrange multiplier method to obtain the second real-time position information of the tissue-surface-moving entity at the target time point.

4. The method of claim 1, wherein, The calculating of the first feedback force according to the first real-time position information and the second real-time position information comprises: determining a target depth of the tissue-penetrable entity penetrating into the skin tissue model according to the first real-time position information and the second real-time position information; determining angle information between the target surgical instrument and the skin tissue model, and determining a tissue stiffness coefficient of the skin tissue model. Determine a first feedback force according to the target depth, the included angle information and the tissue stiffness coefficient.

5. A force feedback device, characterized by Comprise: A surgical instrument determination module is configured to determine a target surgical instrument, wherein the target surgical instrument comprises a tissue-penetrable entity and a tissue-moving entity connected by a spring; A position information acquisition module is configured to determine first real-time position information of the tissue-penetrable entity at a target time point and first reference position information of the tissue-moving entity at a corresponding reference time point of the target time point when a target simulation surgery is performed on a skin tissue model by the target surgical instrument; A position information calculation module is configured to determine second real-time position information of the tissue-moving entity at the target time point according to the first real-time position information and the first reference position information; A feedback force determination module is configured to determine a target feedback force of the target surgical instrument at the target time point according to the first real-time position information and the second real-time position information; A force feedback module is configured to perform force feedback on an operating object operating the target surgical instrument based on the target feedback force; The position information calculation module comprises an entity connecting unit, a constraint information determination unit and a first calculation unit; The entity connecting unit is configured to determine a connecting line segment between the tissue-moving entity under the first real-time position information and the tissue-moving entity under the first reference position information; The constraint information determination unit is configured to determine collision surface constraint information of a target surface of the skin tissue model in the case that there is an intersection between the connecting line segment and the target surface; The first calculation unit is configured to determine the second real-time position information of the tissue-moving entity at the target time point according to the collision surface constraint information and the first real-time position information; The feedback force determination module comprises a first feedback force determination unit, a second feedback force determination unit and a target feedback force determination unit; The first feedback force determination unit is configured to calculate a first feedback force according to the first real-time position information and the second real-time position information; The second feedback force determination unit is configured to determine a Coulomb friction coefficient corresponding to the target surface of the skin tissue model and determine a second feedback force according to the first feedback force and the Coulomb friction coefficient; The target feedback force determination unit is configured to determine a target feedback force of the target surgical instrument at the target time point according to the first feedback force and the second feedback force.

6. An electronic device, comprising: The electronic device comprises: At least one processor; and A memory connected in communication with the at least one processor; wherein The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the force feedback method of any one of claims 1-4.

7. A computer readable storage medium characterized by The computer readable storage medium stores computer instructions for enabling the processor to execute the force feedback method of any one of claims 1-4 when executed.

Images