Force feedback method and device, electronic equipment and storage medium

Abstract

The invention discloses a force feedback method and device, electronic equipment and a storage medium. The method comprises the following steps: determining a target surgical instrument; when a target simulation operation is executed on the skin tissue model through the target surgical instrument, first real-time position information of a pierceable tissue entity at a target time point is determined, and first reference position information of a moving entity on the tissue at a reference time point corresponding to the target time point is determined; determining second real-time position information of the moving entity on the organization at the target time point according to the first real-time position information and the first reference position information; determining a target feedback force of the target surgical instrument at the target time point according to the first real-time position information and the second real-time position information; and performing force feedback on an operation object operating the target surgical instrument based on the target feedback force. Based on the technical scheme of the invention, the simulation operation based on force feedback can be closer to tactile interaction perception of an actual operation.

Description

Technical Field

[0001] The present invention relates to the field of computer application technology, and in particular to a force feedback method, device, electronic equipment and storage medium. Background Art

[0002] Before the actual operation, it is now very common to use models that simulate human tissue to provide preoperative training for technical personnel in related fields. Highly simulated preoperative training can effectively reduce the risk of accidents during the actual operation.

[0003] In related technologies, force feedback for simulated surgical operation objects during simulated surgery is usually achieved through traditional force feedback models, but non-realistic tactile experiences such as sudden changes in force feedback and discontinuous force feedback usually occur. In other words, simulated surgery based on traditional force feedback methods is difficult to achieve interactive perception close to actual surgery. Summary of the invention

[0004] The present invention provides a force feedback method, device, electronic device and storage medium to solve the technical problem that simulated surgery based on traditional force feedback methods is difficult to achieve interactive perception close to actual surgery.

[0005] According to one aspect of the present invention, a force feedback method is provided, the method comprising:

[0006] Determining a target surgical instrument, wherein the target surgical instrument includes a tissue-penetrable entity and a tissue-moving entity connected by a spring;

[0007] When performing a target simulated surgery on the skin tissue model by using a target surgical instrument, determining first real-time position information of the pierceable tissue entity at a target time point, and determining first reference position information of the moving entity on the tissue at a reference time point corresponding to the target time point;

[0008] Determine second real-time location information of the mobile entity in the organization at the target time point according to the first real-time location information and the first reference location information;

[0009] Determining a target feedback force of the target surgical instrument at the target time point according to the first real-time position information and the second real-time position information;

[0010] Force feedback is performed on an operating object operating the target surgical instrument based on the target feedback force.

[0011] According to another aspect of the present invention, there is provided a force feedback device, the device comprising:

[0012] A surgical instrument determination module, used for determining a target surgical instrument, wherein the target surgical instrument comprises a tissue-penetrable entity and a tissue-moving entity connected by a spring;

[0013] A position information acquisition module, used to determine the first real-time position information of the pierceable tissue entity at a target time point, and determine the first reference position information of the moving entity on the tissue at a reference time point corresponding to the target time point when performing a target simulated surgery on the skin tissue model through a target surgical instrument;

[0014] A position information calculation module, used to determine the second real-time position information of the mobile entity in the organization at the target time point according to the first real-time position information and the first reference position information;

[0015] A feedback force determination module, configured to determine a target feedback force of the target surgical instrument at the target time point according to the first real-time position information and the second real-time position information;

[0016] A force feedback module is used to provide force feedback to an operating object operating the target surgical instrument based on the target feedback force.

[0017] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0018] at least one processor; and

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the force feedback method described in any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the force feedback method described in any embodiment of the present invention when executed.

[0022] The technical solution of the embodiment of the present invention is to determine the target surgical instrument, wherein the target surgical instrument includes a pierceable tissue entity and a movable entity on the tissue connected by a spring; when the target simulated surgery is performed on the skin tissue model by the target surgical instrument, determine the first real-time position information of the pierceable tissue entity at the target time point, and determine the first reference position information of the movable entity on the tissue at the reference time point corresponding to the target time point; determine the second real-time position information of the movable entity on the tissue at the target time point according to the first real-time position information and the first reference position information; determine the target feedback force of the target surgical instrument at the target time point according to the first real-time position information and the second real-time position information; and perform force feedback on the operating object of the target surgical instrument based on the target feedback force. The effect of accurately and continuously calculating the force between the target surgical instrument and the skin tissue model during the target simulated surgery and feeding back the force to the operating object of the target surgical instrument in real time is achieved. Based on the technical solution of the present invention, the operating object can continuously perceive tactile stimulation during the target simulated surgery, so as to achieve the technical effect of interactive perception of the target simulated surgery close to the actual surgery. The present invention can be applied to simulated surgery training before the actual surgery.

[0023] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 is a flow chart of a force feedback method provided according to Embodiment 1 of the present invention;

[0026] Figure 2 A scene diagram of performing a target simulated surgery on a skin tissue model by using a target surgical instrument provided according to an embodiment of the present invention;

[0027] Figure 3 is an example diagram of a target surface under two different collision surface constraint information provided according to an embodiment of the present invention;

[0028] Figure 4 is a flow chart of calculating second real-time location information provided according to an embodiment of the present invention;

[0029] Figure 5 is a flow chart of a force feedback method provided according to Embodiment 2 of the present invention;

[0030] Figure 6 is a comparison diagram of second real-time position information updates between the presence of Coulomb friction and the absence of Coulomb friction in the same simulated surgical scenario provided by an embodiment of the present invention;

[0031] Figure 7 is a structural schematic diagram of a force feedback device provided according to Embodiment 3 of the present invention;

[0032] Figure 8 It is a schematic diagram of the structure of an electronic device for implementing the force feedback method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] Embodiment 1

[0036] Figure 1 A flow chart of a force feedback method is provided for the first embodiment of the present invention. This embodiment is applicable to the case where force feedback is performed through a force feedback model. The method can be performed by a force feedback device. The force feedback device can be implemented in the form of hardware and / or software. The force feedback device can be configured in a computer. Figure 1 As shown, the method includes:

[0037] S110. Determine a target surgical instrument, wherein the target surgical instrument includes a tissue-penetrable entity and a tissue-moving entity connected by a spring.

[0038] The target surgical instrument may be understood as a surgical instrument. In an embodiment of the present invention, the target surgical instrument may include a tissue-penetrating entity and a tissue-moving entity connected by a spring, such as Figure 2 As shown, Figure 2 is a scene diagram of performing a target simulated surgery on a skin tissue model using a target surgical instrument according to an embodiment of the present invention. Figure 2 The entity at the proxy location can represent the mobile entity in the organization, and the entity at the actual location can represent the mobile entity in the organization.

[0039] In an embodiment of the present invention, during the target simulated surgery performed on a skin tissue model using a target surgical instrument, the pierceable tissue entity can penetrate into the interior of the skin tissue model, and after the pierceable tissue entity penetrates into the interior of the skin tissue model, the position of the pierceable tissue entity no longer changes.

[0040] In an embodiment of the present invention, the moving entity on the tissue does not penetrate into the skin tissue model during the simulated surgery. After the position of the moving entity on the tissue is fixed, the moving entity on the tissue can slide along the target surface of the skin tissue model to perform the target simulated surgery.

[0041] In the embodiment of the present invention, both the entity capable of penetrating tissue and the entity movable on tissue can be a spherical entity.

[0042] S120, when performing a target simulated surgery on a skin tissue model using a target surgical instrument, determining the first real-time position information of the pierceable tissue entity at a target time point, and determining the first reference position information of the moving entity on the tissue at a reference time point corresponding to the target time point.

[0043] The skin tissue model can be understood as a physical model that simulates human skin tissue.

[0044] Optionally, the target simulated surgery may be a simulated surgery of inserting the pierceable tissue entity into the skin tissue model and operating the movable entity on the tissue to slide on the target surface of the skin tissue model.

[0045] The target surface may be a penetration surface of the penetrable tissue entity.

[0046] In the embodiment of the present invention, the target time point may be a real-time time point, or a current time point. The reference time point may be a previous historical time point corresponding to the target time point. The time interval between the target time point and the reference time point may be related to the update frequency of the force feedback preset in the present invention, which is not specifically limited here.

[0047] The first real-time position information may be the position coordinates of the pierceable tissue entity in a target coordinate system at a target time point, wherein the target coordinate system may be a preset three-dimensional coordinate system, and correspondingly, the first real-time position information may be a three-dimensional position coordinate.

[0048] The first reference position information may be the position coordinates of the mobile entity on the tissue in the target coordinate system at a reference time point. Related to the first real-time position information, the first real-time position information may also be three-dimensional position coordinates.

[0049] S130. Determine second real-time location information of the mobile entity in the organization at the target time point according to the first real-time location information and the first reference location information.

[0050] In an embodiment of the present invention, the first reference position information of the history of the mobile entity on the organization may be stored in the last cycle of force feedback and can be directly obtained in real time. The second real-time position information of the mobile entity on the organization is difficult to obtain directly and needs to be obtained by position calculation based on the first real-time position information and the first reference position information.

[0051] The second real-time location information may be the location coordinates of the mobile entity in the organization in the target coordinate system at the target time point.

[0052] Optionally, determining the second real-time location information of the mobile entity in the organization at the target time point according to the first real-time location information and the first reference location information includes:

[0053] Determine a connecting line segment between the organizational mobile entity under the first real-time location information and the organizational mobile entity under the first reference location information;

[0054] In the case where there is an intersection between the connecting line segment and the target surface of the skin tissue model, determining collision surface constraint information of the target surface;

[0055] The second real-time position information of the mobile entity on the organization at the target time point is determined according to the collision surface constraint information and the first real-time position information.

[0056] Optionally, the determining, according to the first real-time location information and the first reference location information, second real-time location information of the mobile entity in the organization at the target time point further includes:

[0057] In the case where there is an intersection between the connecting line segment and the target surface of the skin tissue model, the first real-time position information is used as the second real-time position information of the moving entity on the tissue at the target time point.

[0058] The collision surface constraint information may be contact information between the mobile entity on the tissue and the target surface. In an embodiment of the present invention, in the target simulation surgery, the target contact point between the mobile entity on the tissue and the target surface may be one or more. The collision surface constraint information may be information related to the target contact point. The collision surface constraint information may also be information related to the attribute coefficient of the target surface. For example, Figure 3 As shown, Figure 3 3 is an example diagram of a target surface under two different collision surface constraint information provided according to an embodiment of the present invention.

[0059] Figure 4 FIG. 4 is a flow chart of calculating the second real-time location information according to an embodiment of the present invention. Figure 4 As shown, optionally, the process of determining the second real-time location information of the mobile entity in the organization at the target time point according to the first real-time location information and the first reference location information may be as follows:

[0060] Before explaining the process of calculating the second real-time position information, the purpose of calculating the second real-time position information is briefly introduced. The present invention needs to update the real-time position of the moving entity on the tissue each time the tactile feedback force is fed back in a cycle to ensure the continuity of the feedback force fed back, so as to achieve consistency of the tactile and visual effects during the simulated surgery.

[0061] Optionally, the calculation and update of the second real-time position information can be realized by the Dynamic Proxy algorithm. Specifically, when the Device (characterizing the pierceable tissue entity) penetrates into the skin tissue model (e.g., kidney tissue model), the connecting line segment between the reference position of the Proxy (characterizing the moving entity on the tissue) of the target surgical instrument at the previous update time point and the real-time position of the Device at the current time point is determined, and a collision detection is performed (determining whether there is an intersection between the connecting line segment and the target surface); if a collision occurs (there is an intersection), the second real-time position information is calculated and updated according to the collision surface constraint, that is, the real-time position coordinates of the Proxy of the surgical instrument. If no collision occurs, the first real-time position information is used as the second real-time position information. The present invention calculates the real-time position of the Proxy in the n-1th tactile feedback cycle based on the reference position of the Proxy in the n-1th tactile feedback cycle.

[0062] Optionally, determining the second real-time position information of the mobile entity on the organization at the target time point according to the collision surface constraint information and the first real-time position information includes:

[0063] The collision surface constraint information and the first real-time position information are calculated by Lagrange multiplier method to obtain the second real-time position information of the mobile entity on the tissue at the target time point.

[0064] The Lagrange multiplier method can be understood as a method for finding the extreme value of a multivariate function when its variables are constrained by one or more conditions.

[0065] Specifically, the calculation of the second real-time location information can be implemented based on the following target formula:

[0066]

[0067] (A2x+B2y+C2z+D2)+l3(A3x+B3y+C3z+D3);

[0068]

[0069] Among them, x p ,y p 、z p represents the first real-time location information; x, y, z represent the second real-time location information; A i , B i , C i , D i represents the collision surface constraint information; l1, l2, l3 represent Lagrangian operators; L represents a calculated intermediate value, which can be used to determine the second real-time position information.

[0070] The construction of the above target formula is explained as follows:

[0071] When the elastic coefficient of the spring connecting the pierceable tissue entity and the movable entity on the tissue is 1, the elastic potential energy calculation can be implemented based on the following formula:

[0072]

[0073] Among them, x p ,y p 、z p represents the first real-time position information, x, y, z represent the second real-time position information, and Q represents elastic potential energy.

[0074] When the target surgical instrument collides with the skin tissue model, the following formula related to the collision surface (which may be the target surface) is constructed to further realize the calculation of the second real-time position information:

[0075] A n x+B n y+C n z+D n =0

[0076] Among them, A n , B n , C n , D n represents the collision surface constraint information, and x, y, and z represent the second real-time position information.

[0077] In summary, based on the principle that the smaller the potential energy, the more stable it is, the problem of solving the second real-time position information is transformed into the problem of constructing a Lagrangian function to find the extreme value, so as to construct the above-mentioned target formula.

[0078] S140. Determine a target feedback force of the target surgical instrument at the target time point according to the first real-time position information and the second real-time position information.

[0079] The target feedback force may be understood as the force between the target surgical instrument and the skin tissue model during the target simulated surgery. In an embodiment of the present invention, the target feedback force may be fed back to the operation object of the target surgical instrument.

[0080] S150: Performing force feedback on an operating object operating the target surgical instrument based on the target feedback force.

[0081] The operation object may be understood as an object that operates the target surgical instrument. In the embodiment of the present invention, the operation object may be preset according to the scene requirements, and is not specifically limited here. Optionally, the operation object may be a user.

[0082] The technical solution of the embodiment of the present invention is to determine the target surgical instrument, wherein the target surgical instrument includes a pierceable tissue entity and a movable entity on the tissue connected by a spring; when the target simulated surgery is performed on the skin tissue model by the target surgical instrument, determine the first real-time position information of the pierceable tissue entity at the target time point, and determine the first reference position information of the movable entity on the tissue at the reference time point corresponding to the target time point; determine the second real-time position information of the movable entity on the tissue at the target time point according to the first real-time position information and the first reference position information; determine the target feedback force of the target surgical instrument at the target time point according to the first real-time position information and the second real-time position information; and perform force feedback on the operating object of the target surgical instrument based on the target feedback force. The effect of accurately and continuously calculating the force between the target surgical instrument and the skin tissue model during the target simulated surgery and feeding back the force to the operating object of the target surgical instrument in real time is achieved. Based on the technical solution of the present invention, the operating object can continuously perceive tactile stimulation during the target simulated surgery, so as to achieve the technical effect of interactive perception of the target simulated surgery close to the actual surgery. The present invention can be applied to simulated surgery training before the actual surgery.

[0083] Embodiment 2

[0084] Figure 5 This is a flow chart of a force feedback method provided in Embodiment 2 of the present invention. This embodiment refines the method for determining the target feedback force of the target surgical instrument at the target time point according to the first real-time position information and the second real-time position information in the above embodiment. Figure 5 As shown, the method includes:

[0085] S210: Determine the target surgical instrument.

[0086] S220, when performing a target simulated surgery on a skin tissue model using a target surgical instrument, determining the first real-time position information of the pierceable tissue entity at a target time point, and determining the first reference position information of the moving entity on the tissue at a reference time point corresponding to the target time point.

[0087] S230. Determine second real-time location information of the mobile entity in the organization at the target time point based on the first real-time location information and the first reference location information.

[0088] S240: Calculate a first feedback force according to the first real-time position information and the second real-time position information.

[0089] In an embodiment of the present invention, the first feedback force may be a supporting force, such as Figure 2 F shown in n The second feedback force may be a Coulomb friction force, such as Figure 2 F shown in f The target feedback force is determined based on the first feedback force and the second feedback force, such as Figure 2 F shown in user .

[0090] Optionally, calculating a first feedback force according to the first real-time position information and the second real-time position information includes:

[0091] Determining a target depth for the pierceable tissue entity to penetrate the skin tissue model according to the first real-time position information and the second real-time position information;

[0092] Determining the angle information between the target surgical instrument and the skin tissue model, and determining the tissue stiffness coefficient of the skin tissue model;

[0093] A first feedback force is determined according to the target depth, the angle information and the tissue stiffness coefficient.

[0094] The target depth may be a depth in a direction perpendicular to the target surface of the skin tissue model.

[0095] In an embodiment of the present invention, the target surgical instrument may be inserted into the skin tissue model obliquely or vertically. Obviously, there is an angle between the target surgical instrument and the skin tissue model. Optionally, the angle information may be the cosine value of the angle between the target surgical instrument (the moving entity on the tissue pointing to the direction in which the tissue entity can be inserted) and the direction perpendicular to the target surface.

[0096] The tissue stiffness coefficient may be a coefficient of a stiffness property of the skin tissue model. The tissue stiffness coefficients of different skin tissue models may be different or the same.

[0097] Specifically, the calculation of the first feedback force can be implemented based on the following formula:

[0098] F n = kΔxcosθ = k(x t,Device -x t,Proxy )cosθ

[0099] Among them, F n represents the first feedback force, k represents the tissue stiffness coefficient, Δx represents the target depth, cosθ represents the angle information, and x t,Device Indicates the first real-time location information, x t,Proxy Indicates the second real-time location information.

[0100] S250, determining the Coulomb friction coefficient corresponding to the target surface of the skin tissue model, and determining a second feedback force according to the first feedback force and the Coulomb friction coefficient.

[0101] The Coulomb friction coefficient may be related to the material type of the target surface. The Coulomb friction coefficient may be preset according to scene requirements and is not specifically limited here.

[0102] Specifically, the calculation of the second feedback force can be implemented based on the following formula:

[0103] F f =μF n

[0104] Among them, F f represents the second feedback force, μ represents the Coulomb friction coefficient, F n represents the first feedback force.

[0105] S260. Determine a target feedback force of the target surgical instrument at the target time point according to the first feedback force and the second feedback force.

[0106] Specifically, the calculation of the target feedback force can be implemented based on the following formula:

[0107]

[0108] in, represents the target feedback force as a vector, represents the first feedback force of the vector, Denote the second feedback force representing a vector.

[0109] S270: Perform force feedback on the operating object operating the target surgical instrument based on the target feedback force.

[0110] The technical solution of the embodiment of the present invention calculates the first feedback force according to the first real-time position information and the second real-time position information; determines the Coulomb friction coefficient corresponding to the target surface of the skin tissue model, determines the second feedback force according to the first feedback force and the Coulomb friction coefficient; determines the target feedback force of the target surgical instrument at the target time point according to the first feedback force and the second feedback force. The present invention uses the Coulomb friction force as part of the feedback force for real-time calculation and feedback, which can make the tactile perception of the feedback object of the feedback force more realistic. Figure 6 , Figure 6 It is a comparison diagram of the second real-time position information update between the presence of Coulomb friction and the absence of Coulomb friction in the same simulated surgery scene provided by an embodiment of the present invention.

[0111] As follows, the force feedback method is explained as a whole. The present invention is essentially implemented based on a force feedback model of a target, that is, the determination and real-time update of the position information and feedback force are implemented based on the force feedback model of the target. The traditional force feedback model has the phenomenon that the force generated when the surgical instrument is in the gap (for example, the moment when the surgical instrument pierces the tissue model) is discontinuous, and when the moving entity slides along the surface of the skin tissue model, it usually feedbacks a sudden change of feedback force, that is, it produces a non-real tactile experience. As a result, during the process of simulated surgery (actual preoperative training, the surgical object can be a tissue model), when force feedback is performed based on the traditional force feedback model, the visual experience and the tactile experience cannot be highly matched, and it is difficult to achieve the expected effect of preoperative training. The target force feedback model of the present invention adopts a method of updating the position information of the moving entity on the tissue in real time and combines the calculation and feedback of the Coulomb friction force, which can achieve continuous and accurate real-time force feedback, so as to improve the tactile simulation effect of the preoperative simulated surgery based on the target force feedback model, and achieve the expected effect of preoperative training.

[0112] The present invention combines the force feedback model of the target with the surgical instruments of the surgical robot, which can realize the interactive fusion perception of visual perception and tactile perception of simulated surgery, and provides an ideal preoperative training scenario.

[0113] Embodiment 3

[0114] Figure 7 This is a schematic diagram of the structure of a force feedback device provided by Embodiment 3 of the present invention. Figure 7 As shown, the device includes: a surgical instrument determination module 310 , a position information acquisition module 320 , a position information calculation module 330 , a feedback force determination module 340 and a force feedback module 350 .

[0115] Among them, the surgical instrument determination module 310 is used to determine the target surgical instrument, wherein the target surgical instrument includes a pierceable tissue entity and a moving entity on the tissue connected by a spring; the position information acquisition module 320 is used to determine the first real-time position information of the pierceable tissue entity at the target time point, and determine the first reference position information of the moving entity on the tissue at the reference time point corresponding to the target time point when performing a target simulated surgery on a skin tissue model through the target surgical instrument; the position information calculation module 330 is used to determine the second real-time position information of the moving entity on the tissue at the target time point based on the first real-time position information and the first reference position information; the feedback force determination module 340 is used to determine the target feedback force of the target surgical instrument at the target time point based on the first real-time position information and the second real-time position information; the force feedback module 350 is used to perform force feedback on the operating object operating the target surgical instrument based on the target feedback force.

[0116] The technical solution of the embodiment of the present invention is to determine the target surgical instrument, wherein the target surgical instrument includes a pierceable tissue entity and a movable entity on the tissue connected by a spring; when the target simulated surgery is performed on the skin tissue model by the target surgical instrument, determine the first real-time position information of the pierceable tissue entity at the target time point, and determine the first reference position information of the movable entity on the tissue at the reference time point corresponding to the target time point; determine the second real-time position information of the movable entity on the tissue at the target time point according to the first real-time position information and the first reference position information; determine the target feedback force of the target surgical instrument at the target time point according to the first real-time position information and the second real-time position information; and perform force feedback on the operating object of the target surgical instrument based on the target feedback force. The effect of accurately and continuously calculating the force between the target surgical instrument and the skin tissue model during the target simulated surgery and feeding back the force to the operating object of the target surgical instrument in real time is achieved. Based on the technical solution of the present invention, the operating object can continuously perceive tactile stimulation during the target simulated surgery, so as to achieve the technical effect of interactive perception of the target simulated surgery close to the actual surgery. The present invention can be applied to simulated surgery training before the actual surgery.

[0117] Optionally, the target simulated surgery may be a simulated surgery of inserting the pierceable tissue entity into the skin tissue model and operating the movable entity on the tissue to slide on the target surface of the skin tissue model.

[0118] Optionally, the position information calculation module 330 includes: an entity connection unit, a constraint information determination unit and a first calculation unit;

[0119] Wherein, the entity connection unit is used to determine a connection line segment between the organizational mobile entity under the first real-time location information and the organizational mobile entity under the first reference location information;

[0120] The constraint information determination unit is used to determine the collision surface constraint information of the target surface when there is an intersection between the connecting line segment and the target surface of the skin tissue model;

[0121] The first calculation unit is used to determine the second real-time position information of the mobile entity on the organization at the target time point according to the collision surface constraint information and the first real-time position information.

[0122] Optionally, the position information calculation module 330 also includes: a second calculation unit, used to use the first real-time position information as the second real-time position information of the moving entity on the tissue at the target time point when there is an intersection between the connecting line segment and the target surface of the skin tissue model.

[0123] Optionally, the first computing unit is specifically configured to:

[0124] The collision surface constraint information and the first real-time position information are calculated by Lagrange multiplier method to obtain the second real-time position information of the mobile entity on the tissue at the target time point.

[0125] Optionally, the feedback force determination module 340 includes: a first feedback force determination unit, a second feedback force determination unit and a target feedback force determination unit;

[0126] Wherein, the first feedback force determination unit is used to calculate the first feedback force according to the first real-time position information and the second real-time position information;

[0127] The second feedback force determining unit is used to determine the Coulomb friction coefficient corresponding to the target surface of the skin tissue model, and determine the second feedback force according to the first feedback force and the Coulomb friction coefficient;

[0128] The target feedback force determination unit is used to determine the target feedback force of the target surgical instrument at the target time point according to the first feedback force and the second feedback force.

[0129] Optionally, the first feedback force determining unit is specifically configured to:

[0130] Determining a target depth for the pierceable tissue entity to penetrate the skin tissue model according to the first real-time position information and the second real-time position information;

[0131] Determining the angle information between the target surgical instrument and the skin tissue model, and determining the tissue stiffness coefficient of the skin tissue model;

[0132] A first feedback force is determined according to the target depth, the angle information and the tissue stiffness coefficient.

[0133] The force feedback device provided in the embodiment of the present invention can execute the force feedback method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0134] Embodiment 4

[0135] Figure 8 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0136] like Figure 8 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0137] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0138] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a force feedback method.

[0139] In some embodiments, the force feedback method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the force feedback method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the force feedback method in any other appropriate manner (e.g., by means of firmware).

[0140] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0141] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0142] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0143] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0144] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0145] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0146] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0147] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A force feedback method, characterized in that: include: Determining a target surgical instrument, wherein the target surgical instrument includes a tissue-penetrable entity and a tissue-moving entity connected by a spring; When performing a target simulated surgery on the skin tissue model by using a target surgical instrument, determining first real-time position information of the pierceable tissue entity at a target time point, and determining first reference position information of the moving entity on the tissue at a reference time point corresponding to the target time point; Determine second real-time location information of the mobile entity in the organization at the target time point according to the first real-time location information and the first reference location information; Determining a target feedback force of the target surgical instrument at the target time point according to the first real-time position information and the second real-time position information; Force feedback is performed on an operating object operating the target surgical instrument based on the target feedback force.

2. The method according to claim 1, characterized in that: The target simulated surgery may be a simulated surgery in which the pierceable tissue entity is inserted into the skin tissue model and the movable entity on the tissue is operated to slide on the target surface of the skin tissue model.

3. The method according to claim 1, characterized in that The determining, according to the first real-time location information and the first reference location information, the second real-time location information of the mobile entity in the organization at the target time point includes: Determine a connecting line segment between the organizational mobile entity under the first real-time location information and the organizational mobile entity under the first reference location information; In the case where there is an intersection between the connecting line segment and the target surface of the skin tissue model, determining collision surface constraint information of the target surface; The second real-time position information of the mobile entity on the organization at the target time point is determined according to the collision surface constraint information and the first real-time position information.

4. The method according to claim 3, characterized in that: Also includes: In the case where there is an intersection between the connecting line segment and the target surface of the skin tissue model, the first real-time position information is used as the second real-time position information of the moving entity on the tissue at the target time point.

5. The method according to claim 3, characterized in that: The determining, according to the collision surface constraint information and the first real-time position information, the second real-time position information of the mobile entity on the organization at the target time point comprises: The collision surface constraint information and the first real-time position information are calculated by Lagrange multiplier method to obtain the second real-time position information of the mobile entity on the tissue at the target time point.

6. The method according to claim 1, characterized in that The step of determining the target feedback force of the target surgical instrument at the target time point according to the first real-time position information and the second real-time position information includes: Calculate a first feedback force according to the first real-time position information and the second real-time position information; Determine the Coulomb friction coefficient corresponding to the target surface of the skin tissue model, and determine the second feedback force according to the first feedback force and the Coulomb friction coefficient; A target feedback force of the target surgical instrument at the target time point is determined according to the first feedback force and the second feedback force.

7. The method according to claim 6, characterized in that The calculating the first feedback force according to the first real-time position information and the second real-time position information includes: Determining a target depth for the pierceable tissue entity to penetrate the skin tissue model according to the first real-time position information and the second real-time position information; Determining the angle information between the target surgical instrument and the skin tissue model, and determining the tissue stiffness coefficient of the skin tissue model; A first feedback force is determined according to the target depth, the angle information and the tissue stiffness coefficient.

8. A force feedback device, characterized in that: include: A surgical instrument determination module, used for determining a target surgical instrument, wherein the target surgical instrument comprises a tissue-penetrable entity and a tissue-moving entity connected by a spring; A position information acquisition module, used to determine the first real-time position information of the pierceable tissue entity at a target time point, and determine the first reference position information of the moving entity on the tissue at a reference time point corresponding to the target time point when performing a target simulated surgery on the skin tissue model through a target surgical instrument; A position information calculation module, used to determine the second real-time position information of the mobile entity in the organization at the target time point according to the first real-time position information and the first reference position information; A feedback force determination module, configured to determine a target feedback force of the target surgical instrument at the target time point according to the first real-time position information and the second real-time position information; A force feedback module is used to provide force feedback to an operating object operating the target surgical instrument based on the target feedback force.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the force feedback method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the force feedback method according to any one of claims 1 to 7 when executed.

Images