Device control device, storage medium and program product

Abstract

The embodiments of the present application disclose an instrument control device, a storage medium and a program product. Among them, the device performs the following steps: during the movement of the instrument, a corresponding first endoscopic image is obtained; based on the first endoscopic image, the movement of the instrument is determined; if the movement meets the update conditions, the first total friction force of the instrument is determined; based on the first total friction force of the instrument, the target driving force is determined; based on the target driving force, the instrument is controlled. In this way, the movement of the instrument is determined by image recognition, and when the instrument undergoes a certain movement change, the corresponding first total friction force of the instrument can be determined in time, and the driving force can be updated in time, thereby providing a more accurate driving force for the instrument and realizing precise control of the instrument. Moreover, the first total friction force of the instrument can characterize the conveying condition of the instrument in the working channel, so that a more accurate driving force can be provided for the instrument based on the first total friction force of the instrument, further realizing precise control of the instrument.

Description

Technical Field

[0001] The present application relates to the field of medical device technology, and more specifically, to an equipment control device, a storage medium and a program product. Background Art

[0002] Natural cavities such as the digestive tract and respiratory tract are common sites for the occurrence of human diseases. Since the lesions are inside the natural cavities of the human body, they need to be examined by endoscopy or treated surgically.

[0003] Traditional endoscopic examinations or surgeries require people to use both hands to hold the scope, operate knobs, and manually transport the scope to complete the operation. At the same time, different types of instruments need to be used in accordance with the intraoperative procedures. The use of instruments is cumbersome and requires the coordinated operation of multiple people.

[0004] With the development of robot-assisted technology, the application of robots and digital technology in hospitals has become more prominent. The driving mechanism can provide driving force for the instrument to achieve control of the instrument. However, the instrument is long and there are differences in the natural cavities of the human body, which leads to inaccurate control of the instrument. Summary of the invention

[0005] In view of this, the embodiments of the present application disclose an instrument control method, device and storage medium to provide a more accurate driving force for the instrument and achieve precise control of the instrument.

[0006] The technical solutions provided by the embodiments of this application are as follows:

[0007] In a first aspect, an embodiment of the present application provides an apparatus control method, the method comprising:

[0008] During the movement of the instrument, acquiring a corresponding first endoscopic image;

[0009] determining movement of the instrument based on the first endoscopic image;

[0010] If the movement condition satisfies the update condition, determining a first total transport friction force of the instrument; the first total transport friction force represents the friction force corresponding to the outlet of the working channel of the endoscope when the instrument is transported;

[0011] determining a target driving force based on the first conveying total friction force;

[0012] The device is controlled based on the target driving force.

[0013] In a possible implementation, determining the movement of the instrument based on the first endoscopic image includes:

[0014] Determine an exit length based on the first endoscopic image, where the exit length is the distance from the head end of the instrument to the exit of the working channel;

[0015] Correspondingly, if the movement condition satisfies the update condition, determining the first total friction force of the conveying of the device includes:

[0016] If the out-of-lens length is greater than the distance threshold, the first transport total friction force is determined.

[0017] In a possible implementation manner, determining the first total conveying friction force includes:

[0018] Performing discretization processing on the motor control model corresponding to the device to obtain the motor control model after discretization processing;

[0019] The first total transport friction force is determined based on the motor control model after the discretization process and the first driving current corresponding to the device.

[0020] In a possible implementation, the method further includes:

[0021] Processing the first driving current by using a weight corresponding to the first driving current and a recursive average filtering method to obtain a second driving current;

[0022] Accordingly, determining the first total conveying friction force based on the motor control model after the discrete processing and the first driving current corresponding to the device includes:

[0023] The first total transport friction force is determined based on the discretized motor control model and the second drive current.

[0024] In a possible implementation manner, determining the target driving force based on the first total conveying friction force includes:

[0025] Determining a first angle feature based on the first conveying total friction force, wherein the first angle feature represents a curvature of the working channel;

[0026] Based on the first angle feature, a target driving force is determined.

[0027] In a possible implementation manner, determining the target driving force based on the first angle feature includes:

[0028] determining a target input force based on a target output force corresponding to an actuator of the device;

[0029] The target driving force is determined based on the target input force and the first angle characteristic.

[0030] In a possible implementation manner, the actuator is a clamp-type actuator, and the target input force is determined based on the target output force corresponding to the actuator of the instrument, including:

[0031] The target input force is determined based on the connecting rod length of the clamp actuator, the target closing angle corresponding to the clamp actuator, and the target output force.

[0032] In a possible implementation, the process of the apparatus moving includes: the apparatus is transported along a first direction;

[0033] Accordingly, the method further includes:

[0034] Controlling the device to transport along a first direction based on a first transport command;

[0035] determining a motion compensation value based on an actual conveying length and a target conveying length corresponding to the first conveying command;

[0036] generating a second transport command based on the motion compensation value;

[0037] The device is controlled to transport along a second direction based on a second transport command, wherein the second direction is opposite to the first direction.

[0038] In a second aspect, an embodiment of the present application provides an apparatus control device, the apparatus comprising:

[0039] An acquisition module, used for acquiring a corresponding first endoscopic image during the movement of the instrument;

[0040] a determination module, configured to determine the movement of the instrument based on the first endoscopic image;

[0041] The determination module is further used to determine a first total friction force of the instrument when the movement condition satisfies an update condition; the first total friction force of the instrument represents a friction force corresponding to the outlet of the working channel of the endoscope when the instrument is transported;

[0042] a determination module, further configured to determine a target driving force based on the first total conveying friction force;

[0043] A control module is used to control the device based on the target driving force.

[0044] The device control apparatus has the function of implementing the device control method in the first aspect or any possible implementation of the first aspect. The function can be implemented by hardware or by hardware executing corresponding software, and the hardware or software includes one or more modules corresponding to the above functions.

[0045] The beneficial effects shown in this aspect are similar to those of the first aspect, as shown in the first aspect, and will not be repeated here.

[0046] In a third aspect, an embodiment of the present application provides an apparatus control device, comprising:

[0047] A memory for storing instructions;

[0048] A processor is used to execute the instructions in the memory to perform the device control method described in any one of the first aspects above.

[0049] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on a computer, enables the computer to execute the instrument control method described in any one of the first aspects above.

[0050] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when running on a terminal device, enables the terminal device to execute the instrument control method described in any one of the first aspects above.

[0051] In a sixth aspect, the present application provides a chip system, which includes a processor for supporting the device to implement the functions involved in the above aspects, for example, sending or processing the data and / or information involved in the above methods. In one possible design, the chip system also includes a memory, which is used to store program instructions and data necessary for the device. The chip system can be composed of chips, or it can include chips and other discrete devices.

[0052] In the seventh aspect, an embodiment of the present application provides a chip, comprising one or more interface circuits and one or more processors; the interface circuit is used to receive signals from a memory of an electronic device and send signals to the processor, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device executes the instrument control method in the first aspect or any optional embodiment of the first aspect.

[0053] Based on the above technical solution, this application has the following beneficial effects:

[0054] The embodiment of the present application discloses an instrument control method, device and storage medium. Among them, the method includes: obtaining the corresponding first endoscopic image during the movement of the instrument; determining the movement of the instrument based on the first endoscopic image; if the movement meets the update condition, determining the first total friction force of the instrument; determining the target driving force based on the first total friction force; and controlling the instrument based on the target driving force. It can be seen that in the embodiment of the present application, the movement of the instrument is determined by image recognition, so that when the instrument undergoes a certain movement change, the corresponding first total friction force of transportation can be determined in time, and the driving force can be updated in time, so that a more accurate driving force can be provided for the instrument, and precise control of the instrument can be achieved. Moreover, the first total friction force of transportation represents the friction force corresponding to the outlet of the working channel of the instrument delivered to the endoscope, so that the first total friction force of transportation can represent the transportation of the instrument in the working channel, so that a more accurate driving force can be provided for the instrument based on the first total friction force of transportation, and precise control of the instrument can be further achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on the disclosed drawings.

[0056] Figure 1 A schematic diagram of the structure of an endoscope disclosed in an embodiment of the present application;

[0057] Figure 2 A schematic diagram of the structure of a device disclosed in an embodiment of the present application;

[0058] Figure 3 A schematic diagram of the structure of a driving mechanism disclosed in an embodiment of the present application;

[0059] Figure 4 A schematic diagram of a conveying state of a device disclosed in an embodiment of the present application;

[0060] Figure 5 A flowchart of a device control method disclosed in an embodiment of the present application;

[0061] Figure 6 A schematic diagram of an endoscope entering the digestive tract disclosed in an embodiment of the present application;

[0062] Figure 7 A flowchart of another device control method disclosed in an embodiment of the present application;

[0063] Figure 8 A schematic diagram of an endoscope field of view and an instrument outlet disclosed in an embodiment of the present application;

[0064] Fig. 9 A schematic diagram of a tendon sheath drive disclosed in an embodiment of the present application;

[0065] Fig.10 A schematic diagram of another tendon sheath drive disclosed in an embodiment of the present application;

[0066] Fig.11 A schematic diagram of a clamp-type actuator extending out of a working channel disclosed in an embodiment of the present application;

[0067] Fig.12 A schematic diagram of a force analysis of a clamp-type actuator disclosed in an embodiment of the present application;

[0068] Fig.13 This is a schematic diagram of the structure of an instrument control device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0069] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0070] See also Figure 1 , is a schematic diagram of the structure of an endoscope disclosed in an embodiment of the present application. The endoscope 10 includes: an operating part 101, an insertion part 102, a bending section 103 and a head end 104. The operating part 101 includes: a workstation interface 1011, a small dial knob 1013, a large dial knob 1012, a handle 1014, a function button 1015, a suction valve button 1016, a water vapor valve button 1017, etc. There is an entrance 1051 of the working channel 105 below the handle 1014, and the working channel 105 passes through the insertion part 102, the bending section 103 and the head end 104, and reaches the outlet 1052 of the working channel. Among them, the workstation interface 1011 is used to dock with the workstation and read the image of the visual field under the microscope. The head end 104 is the rigid part of the front end of the endoscope, and the end face of the head end 104 is provided with a camera and an outlet 1052 of the working channel. The bending end 103 can generally be bent in four directions.

[0071] See also Figure 2 , is a schematic diagram of the structure of an instrument disclosed in an embodiment of the present application, wherein the instrument 20 is a biopsy forceps, which adopts a disc-type solution. The biopsy forceps includes: an actuator 201, an instrument insertion tube 202, an instrument outlet 203, etc. Among them, the instrument 20 generally has multiple degrees of freedom such as instrument delivery, actuator opening and closing, and actuator rotation. The instrument delivery can be achieved by controlling the joint J1, the jaw opening and closing can be achieved by controlling the joint J2, and the instrument rotation can be achieved by controlling the joint J3.

[0072] See also Figure 3 , is a schematic diagram of the structure of a driving mechanism disclosed in an embodiment of the present application, and the driving structure 30 includes: a rotating body 301, an encoder 302, a servo motor 303, a mechanical transmission part 304, a fixed bracket 305, etc. Among them, the rotating body 301 has a driving motor, a driver and a printed circuit board with corresponding functions. The driver controls the movement of the driving motor so that the driver drives the actuator to open and close and / or rotate. The encoder 302 obtains the rotation angle of the rotating body 301, and according to the inherent internal rotation angle of the device 20, the conveying length is converted according to the ratio of the rotation angle to the length.

[0073] See also Figure 4 , which is a schematic diagram of a conveying state of an instrument disclosed in an embodiment of the present application. The guide tube 40 serves as an introduction channel for the instrument 20, and the instrument 20 is connected to the entrance 1051 of the working channel of the endoscope 10 through the guide tube 40. The driving mechanism 30 can provide driving force to the instrument 20 to realize the conveying operation of the instrument 20 and the rotation or opening and closing of the actuator 201.

[0074] See also Figure 5 , a flow chart of a device control method disclosed in an embodiment of the present application, the method comprising:

[0075] S501. During the movement of the instrument, obtain a corresponding first endoscopic image.

[0076] It should be noted that the instrument movement here refers to the related movement after the instrument is delivered to the outlet of the working channel. The instrument movement here may include at least one of the following: continued delivery of the instrument, actuator closing operation, actuator rotation operation, etc.

[0077] In the embodiment of the present application, at least one first endoscopic image can be acquired by a camera of an endoscope, and the at least one first endoscopic image should be understood as one first endoscopic image or multiple first endoscopic images.

[0078] See also Figure 6 , which is a schematic diagram of an endoscope entering the digestive tract disclosed in an embodiment of the present application. In actual use, the doctor will operate the endoscope to enter the human body and aim at the lesion area. Due to the complex structure of the human digestive tract, the insertion part and the bending section of the endoscope will have curved parts in the body. Thereafter, based on the delivery command, the instrument hose is controlled to pass through the guide tube to enter the working channel of the endoscope and be transported in the working channel. Among them, the delivery command can be triggered by the doctor through an operating handle or an interactive interface, and the servo motor can be controlled to generate the target driving force based on the delivery command.

[0079] S502: Determine the movement of the instrument based on the first endoscopic image.

[0080] In the embodiment of the present application, the content of the first endoscopic image can be identified through image recognition to determine the movement of the instrument, wherein the movement of the instrument may include: whether the instrument moves, the distance the instrument moves, and the like.

[0081] S503: If the movement condition satisfies the update condition, determine the first total conveying friction force of the device.

[0082] Among them, the first total conveying friction force represents the friction force corresponding to the conveying of the instrument to the outlet of the working channel of the endoscope, that is, the friction force corresponding to the conveying of the instrument from the entrance of the working channel to the outlet of the working channel, and can represent the conveying condition of the instrument in the working channel.

[0083] It should be noted that before determining the first total friction force of the instrument, the instrument needs to be transported to the outlet of the working channel. In the embodiment of the present application, the servo motor can be controlled to generate a driving force to drive the mechanical transmission member to drive the rotating body to move, thereby driving the inner and outer shells of the instrument to rotate relative to each other, sending out the instrument hose inside the instrument, and realizing the transportation of the instrument in the working channel of the endoscope to transport the instrument to the outlet of the working channel. Among them, the instrument hose includes an actuator and an instrument insertion tube.

[0084] S504: Determine a target driving force based on the first total conveying friction force.

[0085] S505: Control the device based on the target driving force.

[0086] It can be seen that in the embodiment of the present application, the movement of the instrument is determined by image recognition, so that when the instrument undergoes a certain movement change, the corresponding first total friction force of transportation can be determined in time, and the driving force can be updated in time, so as to provide a more accurate driving force for the instrument and achieve precise control of the instrument. Moreover, the first total friction force of transportation represents the friction force corresponding to the outlet of the instrument delivered to the working channel of the endoscope, so that the first total friction force of transportation can represent the transportation of the instrument in the working channel, so that based on the first total friction force of transportation, a more accurate driving force can be provided for the instrument, and further precise control of the instrument can be achieved.

[0087] See also Figure 7 , is a flow chart of another device control method disclosed in an embodiment of the present application, Figure 7 exist Figure 5 Based on the corresponding disclosed device control method, the device control method is explained in more detail. Figure 7 The device control methods provided include:

[0088] S701. Control the device to be transported along the working channel based on the third transport command.

[0089] The third delivery command in the embodiment of the present application can be triggered by the physician through an operating handle or an interactive interface, and the third delivery command is used to deliver the instrument to the outlet of the working channel.

[0090] S702: Acquire a corresponding second endoscopic image during the process of conveying the instrument along the working channel.

[0091] In the embodiment of the present application, at least one second endoscopic image can be acquired by the camera of the endoscope, and the at least one second endoscopic image should be understood as one second endoscopic image or multiple second endoscopic images.

[0092] S703: When the instrument feature corresponding to the instrument is identified in the second endoscopic image, the conveyance of the instrument is stopped.

[0093] Among them, the instrument feature is the head end feature of the instrument close to the outlet of the working channel.

[0094] It can be understood that when the instrument characteristics corresponding to the instrument are identified, it can be determined that the instrument has been delivered to the outlet of the working channel, the delivery of the instrument can be stopped, and the driving current before stopping the delivery can be obtained so that the second total delivery friction force can be determined subsequently using the driving current before stopping the delivery.

[0095] In a possible implementation method, in an embodiment of the present application, a region of interest can be determined from the second endoscopic image based on the relative position of the outlet of the working channel and the camera; the second endoscopic image is divided into regions of interest using the region of interest to obtain a regional image; the regional image is smoothed, grayed, and binarized to obtain a processed regional image; the processed regional image is edge extracted to obtain a fitting contour; based on the type of instrument to which the instrument belongs, a standard contour corresponding to the instrument is determined; the fitting contour and the standard contour are matched to obtain a first matching result; the first matching result is used to characterize whether the instrument feature is recognized in the second endoscopic image.

[0096] S704: Determine the second total conveying friction force of the device.

[0097] It should be noted that the second total transport friction force is the friction force corresponding to when the head end of the instrument is close to the outlet of the working channel. The second total transport friction force also represents the friction force corresponding to the outlet of the working channel of the instrument when it is transported to the endoscope, and can also represent the transport condition of the instrument in the working channel.

[0098] It can be understood that the method for determining the second total conveying friction force is similar to the method for determining the first total conveying friction force. In the embodiment of the present application, S704 may include: discretizing the motor control model corresponding to the device to obtain the motor control model after discrete processing; and determining the second total conveying friction force based on the motor control model after discrete processing and the third driving current corresponding to the device.

[0099] In the embodiment of the present application, the second angle feature can also be determined based on the second total friction force, so that the physician can understand the curvature of the current working channel based on the second angle feature, so as to facilitate the subsequent triggering of the corresponding instrument control command through the operating handle or the interactive interface. It can be understood that the method for determining the second angle feature is similar to the method for determining the first angle feature.

[0100] S705. During the movement of the instrument, obtain a corresponding first endoscopic image.

[0101] It should be noted that S705 in the embodiment of the present application is similar to S501 in the above embodiment, and the same parts will not be repeated here.

[0102] The process of the instrument movement in the embodiment of the present application may include: the instrument is transported along a first direction. Accordingly, in the embodiment of the present application, after the instrument is stopped from being transported in S703, a first transport command may be received; the instrument is controlled to be transported along the first direction based on the first transport command; and during the process of the instrument being transported along the first direction, a corresponding first endoscopic image is acquired through the camera of the endoscope.

[0103] S706: Determine the movement of the instrument based on the first endoscopic image.

[0104] It should be noted that S706 in the embodiment of the present application is similar to S502 in the above embodiment, and the same parts will not be repeated here.

[0105] The movement condition in the implementation of the present application may include the exit length, that is, S706 may include: determining the exit length of the instrument based on the first endoscopic image, and the exit length is the distance from the head end of the instrument to the exit of the working channel.

[0106] See also Figure 8 , is a schematic diagram of an endoscope field of view and an instrument outlet disclosed in an embodiment of the present application. The camera includes a lens optical center and an optical module. The lens optical center is the camera focus. Figure 8Where H2 and H3 are the pixel lengths of different exit lengths of the instrument under the endoscope field of view, L1 represents the distance from the virtual imaging plane to the end face of the head end, L2 and L3 represent the distance from the head end of the instrument to the virtual imaging plane, Lc represents the distance from the camera focus to the end face of the head end, and h represents the relative distance between the working channel and the camera focus. It should be noted that, given the relative position of the working channel outlet and the camera, the virtual imaging plane of the camera imaging under monocular vision and the camera focus position are fixed, that is, Lc, L1, and h are fixed. Among them, when the instrument is at L2, the exit length is L2+L1, L2 =H2 *L1 / (h- H2); when the instrument is at L3, the exit length is L3+L1, L3=H3*L1 / (h- H3).

[0107] S707: If the movement condition satisfies the update condition, determine the first total transport friction force of the device.

[0108] It can be understood that the first total transport friction and the second total transport friction are the total transport frictions corresponding to the device in different time periods. If the movement condition meets the update condition, the total transport friction needs to be updated from the second total transport friction to the first total transport friction.

[0109] It should be noted that S707 in the embodiment of the present application is similar to S503 in the above embodiment, and the same parts will not be repeated here.

[0110] In a possible implementation manner, S707 in the embodiment of the present application may include:

[0111] Step a1: Discretize the motor control model corresponding to the device to obtain the motor control model after discrete processing.

[0112] The motor control model corresponding to the device in the embodiment of the present application can be expressed as follows:

[0113] ;

[0114] ;

[0115] in, represents the electromagnetic torque of the motor, J represents the moment of inertia, that is, the inertia from the servo motor to the joint J1, represents the angular velocity of the driving mechanism, represents the external load, i.e., the pushing force of the instrument or the resistance encountered during the instrument delivery process, B represents the damping coefficient, represents the torque constant of the servo motor, and I represents the driving current of the servo motor.

[0116] In the embodiment of the present application, the calculus term in the motor control model can be discretized to obtain . Where k represents the kth transmission cycle, T s It represents the delivery cycle. The specific time length of the delivery cycle can be set according to actual needs, and the embodiment of the present application does not limit this.

[0117] By combining the above motor control model and the calculus terms after discrete processing, the motor control model after discrete processing can be obtained.

[0118] Step a2: Determine a first total conveying friction force based on the motor control model after discretization and a first driving current corresponding to the device.

[0119] In the embodiment of the present application, a force expression can be obtained based on the motor control model after discretization, and the force expression is as follows:

[0120] ;

[0121] Among them, T L (k) represents the force of the device in the kth delivery cycle, T em (k) represents the electromagnetic torque of the motor corresponding to the kth conveying cycle of the device, w m (k) represents the angular velocity of the device at the kth conveying cycle, w m (k-1) represents the angular velocity of the device corresponding to the k-1th conveying cycle, and n is a positive integer.

[0122] See also Fig. 9 , is a schematic diagram of a tendon sheath drive disclosed in an embodiment of the present application, Fig. 9 S(1) in the equation represents the conveying position corresponding to the last moment of the first conveying cycle, and F0 represents the total conveying friction. Indicates the angle change corresponding to the first conveying cycle, represents the angle change corresponding to the kth conveying cycle, F1 represents the driving force corresponding to the first conveying cycle, and F k-1 represents the driving force of the device in the k-1th conveying cycle, F k represents the driving force of the device in the kth conveying cycle, F n It indicates the driving force corresponding to the instrument in the nth delivery cycle, that is, the driving force corresponding to the instrument in the last delivery cycle, and can also be understood as the driving force corresponding to the last delivery distance of the instrument. It should be noted that the tendon sheath drive is composed of a sheath and a tendon. The sheath is a sheath tube, and the tendon is a flexible driving rod, such as a rope, nickel-titanium wire, etc. The tendon pushes or pulls inside the sheath. Among them, the tendon can be understood as a part of the instrument hose.

[0123] Combined with the above Fig. 9 Based on the content of and the above force expression, the calculation formula for the force corresponding to the device in the last delivery cycle can be obtained as follows:

[0124] ;

[0125] Among them, T L(n) represents the force corresponding to the device in the nth conveying cycle, that is, the force corresponding to the nth conveying distance, I(n) represents the driving current corresponding to the device in the nth conveying cycle, that is, the driving current corresponding to the nth conveying distance, w m (n) represents the angular velocity of the device in the nth conveying cycle, that is, the angular velocity corresponding to the nth conveying distance, w m (n-1) represents the angular velocity of the device corresponding to the n-1th conveying cycle.

[0126] Combined with the force corresponding to the last conveying cycle, the calculation formula for the total conveying friction force can be obtained as follows:

[0127] ;

[0128] Among them, F0 represents the total friction force of transportation, T L(n) It represents the force on the device in the nth conveying cycle, and R represents the ratio of the conveying length to the rotation angle of the rotating body.

[0129] It is understandable that in the embodiment of the present application, the first driving current, the first angular velocity and the second angular velocity can be substituted into In the calculation, T L(n) , and then T L(n) Substitution In , F0 is calculated, that is, the first total conveying friction force is obtained. Among them, the first driving current is the driving current corresponding to the device in the nth conveying cycle, which can be understood as the driving current corresponding to the conveying cycle in which the device is located when the movement condition meets the update condition, or the driving current corresponding to the last conveying cycle in which the device is located when the movement condition meets the update condition; the first angular velocity is the angular velocity corresponding to the device in the nth conveying cycle, which can be understood as the first angular velocity is the angular velocity corresponding to the conveying cycle in which the device is located when the movement condition meets the update condition; the second angular velocity is the angular velocity corresponding to the device in the n-1th conveying cycle, which can be understood as the second angular velocity is the angular velocity corresponding to the previous conveying cycle in which the device is located when the movement condition meets the update condition.

[0130] In one possible implementation, the nth conveying cycle in the embodiment of the present application includes multiple moments, each moment corresponds to a first drive current, that is, there are multiple first drive currents. The embodiment of the present application can use multiple weights corresponding to multiple first drive currents and a recursive average filtering method to process multiple first drive currents to obtain a second drive current; then determine the first conveying total friction force based on the motor control model and the second drive current after discrete processing. In this way, using a specific method to filter the current value can reduce the impact of noise while ensuring sufficient sensitivity, obtain a more accurate drive current, and thus determine a more accurate first conveying total friction force.

[0131] In the embodiment of the present application, the second driving current can be obtained by the following calculation formula:

[0132] ;

[0133] ;

[0134] Wherein, I represents the second driving current, I(i), I(i-1)…I(im) represent the first driving currents corresponding to the multiple moments in the last transmission cycle, and k i , k i-1 …k i-m represents the weight corresponding to the first driving current, where i, j, and m are positive integers.

[0135] It should be noted that the embodiment of the present application is based on the recursive averaging filtering method, and the driving current at different times is multiplied by different weights to obtain a weighted recursive averaging filtering method. This solves the shortcoming of low sensitivity of the recursive averaging filtering method, and improves the performance of the filter by changing the weights to obtain a more accurate driving current.

[0136] In a possible implementation, the calculation formula of the second driving current in the embodiment of the present application can be , that is, the second driving current is determined by taking four first driving currents corresponding to four moments in the last transmission cycle. In this way, the influence of noise can be reduced while ensuring sufficient sensitivity.

[0137] In one possible manner, S707 in the embodiment of the present application may include: if the length of the camera is greater than the distance threshold, determine the first total friction of the transport. Among them, the distance threshold can be set to a specific value according to the actual situation, and the embodiment of the present application does not limit this. In this way, based on the length of the camera, the change in the moving distance of the device can be accurately determined, and combined with the distance threshold, it can be accurately determined whether it is necessary to determine the first total friction of the transport, that is, determine whether it is necessary to update the driving force, so that when the driving force needs to be updated, the driving force can be updated in time, providing a more accurate driving force for the device, and realizing precise control of the device.

[0138] S708: Determine a target driving force based on the first total conveying friction force.

[0139] In a possible implementation manner, S708 in the embodiment of the present application may include:

[0140] Step b1: determining a first angle characteristic based on the first total conveying friction force.

[0141] The first angle feature represents the curvature of the working channel.

[0142] See also Fig.10 , is a schematic diagram of another tendon sheath drive disclosed in an embodiment of the present application, Fig.10 The figure shows the tendon sheath drive under the ideal arc state. Fig.10 In represents the arc angle, L represents the arc length, represents the input force, Indicates the output force.

[0143] The relationship between input force and output force can be expressed as ,in, represents the Coulomb friction coefficient. The output force can be obtained , correspondingly, the friction model can be expressed as .

[0144] Please continue to refer to 9. In the embodiment of the present application, the force action is mutual. Take a certain chord length along the curve. Assuming that the chord length is very short, according to the relationship between the input force and the output force, the force condition corresponding to the first section of the conveying distance can be obtained as follows: ;

[0145] Then the force corresponding to the next conveying distance is: ;

[0146] Then the force corresponding to the kth conveying distance is: ;

[0147] By multiplying the corresponding forces of n transport distances on the transport path of the device, we can get The conveying path includes the path corresponding to conveying the device from the entrance of the working channel to the exit of the working channel. It can be understood that the conveying distance corresponds to the conveying cycle, that is, the device will be conveyed along the working channel for the kth conveying distance in the kth conveying cycle.

[0148] based on , the angle characteristic of the conveying path can be obtained as It should be noted that the trajectory of the driving wire that controls the opening and closing freedom of the actuator of the device in this embodiment is basically the same as the shape of the working channel, so the path angle feature inside the device is also .

[0149] In the embodiment of the present application, the last section of the conveying distance is a straight line, so the driving force through the straight line can be obtained by pre-testing, that is, F can be obtained by pre-measurement. n It is understandable that in the embodiment of the present application, the first total friction force F0 calculated in S707 and the driving force F corresponding to the last conveying distance obtained in advance can be used as n Substitution In the example above, the first angle feature is obtained.

[0150] In the embodiment of the present application, the bending angle characteristic of the instrument hose is estimated by the stress condition of the instrument hose, so that a more accurate first angle characteristic can be obtained.

[0151] Step b2: Determine the target driving force based on the first angle feature.

[0152] In the embodiment of the present application, a control command triggered by a doctor can be received, and the target output force corresponding to the actuator of the instrument can be determined based on the control command, the target input force can be determined based on the target output force, and the target driving force can be determined based on the target input force and the first angle feature. In this way, the target driving force that better meets the control requirements of the instrument can be determined in combination with the target output force, thereby achieving precise control of the instrument.

[0153] In a possible implementation, the control command may be a gear control command, and different gear control commands may correspond to different target output forces. For example, the control gears include the first gear, the second gear, and the third gear. If the gear control command indicates the first gear, the corresponding target output force is 2N, if the gear control command indicates the second gear, the corresponding target output force is 3N, and if the gear control command indicates the third gear, the corresponding target output force is 4N. It can be understood that the above is only an exemplary description and should not be understood as a limitation on the embodiments of the present application.

[0154] In the embodiments of the present application, the closing of the forceps actuator can be completed at an ideal position, the target output force can be set, and then the control gear can be set accordingly to facilitate the doctor's operation; the target output force can also be set based on the maximum safe closing force and the minimum effective closing force, and then the control gear can be set accordingly, and so on.

[0155] See also Fig.11 and Fig.12 , Fig.11 This is a schematic diagram of a clamp-type actuator extending out of a working channel disclosed in an embodiment of the present application. Fig.12 This is a schematic diagram of a force analysis of a clamp-type actuator disclosed in an embodiment of the present application. Fig.11 and Fig.12 The clamp-type actuator is specifically a four-bar clamp-type actuator. Fig.11 The tendon driven by the tendon sheath is the driving wire, and a clamp-type actuator is provided at the end of the tendon sheath drive. The clamp-type actuator extends out of the working channel so that the lesion position can be inspected or surgically treated accordingly through the clamp-type actuator. Fig.11 and Fig.12 In represents the target input force, represents the target output force, represents the target internal angle, represents the target closing angle, , , Indicates the connecting rod length of the clamp actuator. represents the length of the first connecting rod, represents the length of the second connecting rod, represents the length of the third connecting rod, represents the component force at the node, Representation and Forces in opposite directions.

[0156] In the embodiment of the present application, if the actuator of the instrument is a four-link clamp actuator, the target input force can be determined based on the connecting rod length of the clamp actuator, the target closing angle corresponding to the clamp actuator, and the target output force. In this way, a more accurate target input force can be determined based on the structural characteristics and target parameters of the actuator, thereby achieving precise control of the actuator of the instrument.

[0157] Combination Fig.12 To illustrate, the force component at the jaw node of the four-link clamp actuator is , , and The direction is opposite to that of the triangle. ,get . Given in the examples of this application and ,and , from which it can be deduced that the calculation formula of the target input force can be expressed as follows:

[0158] .

[0159] Accordingly, the calculation formula for controlling the driving force of the motor in the embodiment of the present application can be expressed as follows:

[0160] ;

[0161] in, Indicates the driving force of the goal. It indicates the friction coefficient between the opening and closing drive wire of the instrument and the inner sheath of the instrument hose, that is, the friction coefficient between the tendon and the sheath. It represents the angle characteristic value of the sheath inside the instrument hose, that is, the first angle characteristic.

[0162] It can be seen that in the embodiment of the present application, the bending angle characteristics of the instrument hose are estimated by the force condition of the instrument hose, and the changed bending angle characteristics are determined in combination with the first endoscopic image, so as to determine a more accurate target driving force.

[0163] S709, controlling the device based on the target driving force.

[0164] In the embodiment of the present application, the actuator of the device can be controlled based on the target driving force, such as performing closing operations.

[0165] S710: Determine a motion compensation value based on an actual conveying length and a target conveying length corresponding to the first conveying command.

[0166] The process of the device movement includes: the device is transported along a first direction, and the first direction is the direction corresponding to the device continuing to transport forward. In the embodiment of the present application, the motion compensation value can be determined based on the difference between the actual transport length and the target transport length.

[0167] In the embodiment of the present application, the first conveying command may be triggered multiple times, and the device moves a certain step displacement each time, and then determines the actual displacement corresponding to the multiple first conveying commands, that is, determines the actual conveying length; or it may be moved at a given speed within a period of time from the triggering of the first conveying command to the end of the first conveying command, and then determines the actual displacement corresponding to the first conveying command, that is, determines the actual conveying length. It is understandable that the actual conveying length can be determined by using the method for determining the length of the out-of-lens shot.

[0168] S711 . Generate a second transport command based on the motion compensation value.

[0169] In the embodiment of the present application, when the instrument needs to switch direction, such as when the actuator and / or the instrument hose needs to be retracted into the working channel, or when the actuator and / or the instrument hose needs to be retracted into the interior of the instrument, etc., the physician will trigger a second delivery command. At this time, you can try to add a motion compensation value, that is, generate a second delivery command based on the motion compensation value, thereby eliminating the hysteresis of the force of the instrument delivery switching and the influence of motion backlash.

[0170] S712: Control the device to transport along the second direction based on the second transport command.

[0171] The second direction is opposite to the first direction.

[0172] It can be seen that in the embodiment of the present application, the length of the device out of the camera is determined by image recognition, and when the length of the device out of the camera exceeds the distance threshold, the driving force is updated in time, so that a more accurate driving force can be provided for the device, and precise control of the device can be achieved. Moreover, the driving force is determined according to the angular velocity characteristics corresponding to the conveying path and the structural characteristics of the actuator, so that a more accurate driving force can be provided for the actuator of the device, and precise control of the actuator can be further achieved.

[0173] See also Fig.13 , a schematic diagram of the structure of an apparatus control device disclosed in an embodiment of the present application, the device comprises:

[0174] An acquisition module 1301 is used to acquire a corresponding first endoscopic image during the movement of the instrument;

[0175] A determination module 1302, configured to determine the movement of the instrument based on the first endoscopic image;

[0176] The determination module 1302 is further configured to determine a first total friction force of the instrument when the movement condition satisfies an update condition; the first total friction force of the instrument represents a friction force corresponding to the outlet of the working channel of the endoscope when the instrument is transported;

[0177] The determination module 1302 is further configured to determine a target driving force based on the first total conveying friction force;

[0178] The control module 1303 is used to control the device based on the target driving force.

[0179] It can be seen that in the embodiment of the present application, the movement of the instrument is determined by image recognition, so that when the instrument undergoes a certain movement change, the corresponding first total friction force of transportation can be determined in time, and the driving force can be updated in time, so as to provide a more accurate driving force for the instrument and achieve precise control of the instrument. Moreover, the first total friction force of transportation represents the friction force corresponding to the outlet of the instrument delivered to the working channel of the endoscope, so that the first total friction force of transportation can represent the transportation of the instrument in the working channel, so that based on the first total friction force of transportation, a more accurate driving force can be provided for the instrument, and further precise control of the instrument can be achieved.

[0180] In a possible implementation manner, the determination module 1302 in the instrument control device provided in the embodiment of the present application is specifically configured to determine the exit length based on the first endoscopic image, where the exit length is the distance from the head end of the instrument to the exit of the working channel;

[0181] The determination module 1302 is further configured to determine the first total conveying friction force if the out-of-lens length is greater than a distance threshold.

[0182] In a possible implementation, the determination module 1302 in the instrument control device provided in the embodiment of the present application includes:

[0183] A processing unit, used for performing discrete processing on the motor control model corresponding to the device to obtain the motor control model after discrete processing;

[0184] The friction force determination unit is used to determine the first total conveying friction force based on the motor control model after the discrete processing and the first driving current corresponding to the device.

[0185] In a possible implementation, the device control device provided in the embodiment of the present application further includes:

[0186] A filtering module, configured to process the first driving current by using a weight corresponding to the first driving current and a recursive average filtering method to obtain a second driving current;

[0187] Correspondingly, the friction force determination unit is specifically configured to determine the first total conveying friction force based on the motor control model after the discrete processing and the second driving current.

[0188] In a possible implementation, the determination module 1302 in the instrument control device provided in the embodiment of the present application includes:

[0189] an angle determination unit, configured to determine a first angle feature based on the first conveying total friction force, wherein the first angle feature represents a curvature of the working channel;

[0190] The driving force determination unit is used to determine a target driving force based on the first angle feature.

[0191] In a possible implementation manner, the driving force determination unit in the instrument control device provided in the embodiment of the present application includes:

[0192] an input force determination subunit, configured to determine a target input force based on a target output force corresponding to an actuator of the apparatus;

[0193] The driving force determination subunit is used to determine the target driving force based on the target input force and the first angle feature.

[0194] In a possible implementation manner, the actuator in the instrument control device provided in the embodiment of the present application is a clamp-type actuator;

[0195] Correspondingly, the input force determination subunit is specifically used to determine the target input force based on the connecting rod length of the clamp actuator, the target closing angle corresponding to the clamp actuator, and the target output force.

[0196] In a possible implementation, the process of the movement of the instrument in the instrument control device provided in the embodiment of the present application includes: the instrument is transported along a first direction;

[0197] Correspondingly, the control module 1303 is further used to control the device to transport along the first direction based on the first transport command;

[0198] The determination module 1302 is further configured to determine a motion compensation value based on an actual conveying length and a target conveying length corresponding to the first conveying command;

[0199] The device further includes: a generating module, configured to generate a second delivery command based on the motion compensation value;

[0200] The control module 1303 is further configured to control the device to transport along a second direction based on a second transport command, where the second direction is opposite to the first direction.

[0201] Furthermore, the embodiment of the present application also provides an instrument control device, comprising:

[0202] A memory for storing instructions;

[0203] A processor is used to execute instructions in the memory to perform any implementation method of the above-mentioned device control method.

[0204] Furthermore, an embodiment of the present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes any one of the implementation methods of the above-mentioned device control method.

[0205] Furthermore, an embodiment of the present application also provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute any one of the implementation methods of the above-mentioned device control method.

[0206] It can be known from the description of the above implementation mode that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for enabling a computer device (which can be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0207] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0208] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0209] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0210] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An apparatus control device, characterized in that: The device comprises: A memory for storing instructions; A processor, configured to execute the instructions in the memory to perform the following steps: During the movement of the instrument, acquiring a corresponding first endoscopic image; Determine an exit length based on the first endoscopic image, where the exit length is the distance from the head end of the instrument to the exit of the working channel of the endoscope; If the exit length is greater than the distance threshold, a first total transport friction force is determined; the first total transport friction force represents the friction force corresponding to the transport of the device to the outlet of the working channel; Based on the first total conveying friction force, determining a first angle feature; the first angle feature represents the curvature of the working channel; determining a target input force based on a target output force corresponding to an actuator of the device; determining a target driving force based on the target input force and the first angle characteristic; The device is controlled based on the target driving force.

2. The device according to claim 1, characterized in that The determining of the first total conveying friction force comprises: Performing discretization processing on the motor control model corresponding to the device to obtain the motor control model after discretization processing; The first total transport friction force is determined based on the motor control model after the discretization process and the first driving current corresponding to the device.

3. The device according to claim 2, characterized in that The processor is further configured to execute the instructions in the memory to perform the following steps: Processing the first driving current by using a weight corresponding to the first driving current and a recursive average filtering method to obtain a second driving current; Accordingly, determining the first total conveying friction force based on the motor control model after the discrete processing and the first driving current corresponding to the device includes: The first total transport friction force is determined based on the discretized motor control model and the second drive current.

4. The device according to claim 1, characterized in that The actuator is a clamp-type actuator, and the target input force is determined based on the target output force corresponding to the actuator of the instrument, including: The target input force is determined based on the connecting rod length of the clamp actuator, the target closing angle corresponding to the clamp actuator, and the target output force.

5. The device according to claim 1, characterized in that The process of the apparatus moving includes: the apparatus is transported along a first direction; Accordingly, the processor is further configured to execute the instructions in the memory to perform the following steps: Controlling the device to transport along a first direction based on a first transport command; determining a motion compensation value based on an actual conveying length and a target conveying length corresponding to the first conveying command; generating a second transport command based on the motion compensation value; The device is controlled to transport along a second direction based on a second transport command, wherein the second direction is opposite to the first direction.

6. An apparatus control device, characterized in that: The device comprises: An acquisition module, used for acquiring a corresponding first endoscopic image during the movement of the instrument; A determination module, configured to determine an exit length based on the first endoscopic image, wherein the exit length is a distance from the head end of the instrument to an exit of a working channel of the endoscope; The determination module is further used to determine a first total transport friction force if the exit length is greater than a distance threshold; the first total transport friction force represents the friction force corresponding to the device being transported to the outlet of the working channel; The determination module is further used to determine a first angle feature based on the first total conveying friction force; the first angle feature represents the curvature of the working channel; The determination module is further used to determine the target input force based on the target output force corresponding to the actuator of the device; a determination module, further configured to determine a target driving force based on the target input force and the first angle feature; A control module is used to control the device based on the target driving force.

7. A computer-readable storage medium, characterized in that: The invention comprises instructions which, when executed on a computer, cause the computer to perform the following steps: During the movement of the instrument, acquiring a corresponding first endoscopic image; Determine an exit length based on the first endoscopic image, where the exit length is the distance from the head end of the instrument to the exit of the working channel of the endoscope; If the exit length is greater than the distance threshold, a first total transport friction force is determined; the first total transport friction force represents the friction force corresponding to the transport of the device to the outlet of the working channel; Based on the first total conveying friction force, determining a first angle feature; the first angle feature represents the curvature of the working channel; determining a target input force based on a target output force corresponding to an actuator of the device; determining a target driving force based on the target input force and the first angle characteristic; The device is controlled based on the target driving force.

8. A computer program product, characterized in that When the computer program product is run on a terminal device, the terminal device is caused to execute the following steps: During the movement of the instrument, acquiring a corresponding first endoscopic image; Determine an exit length based on the first endoscopic image, where the exit length is the distance from the head end of the instrument to the exit of the working channel of the endoscope; If the exit length is greater than the distance threshold, a first total transport friction force is determined; the first total transport friction force represents the friction force corresponding to the transport of the device to the outlet of the working channel; Based on the first total conveying friction force, determining a first angle feature; the first angle feature represents the curvature of the working channel; determining a target input force based on a target output force corresponding to an actuator of the device; determining a target driving force based on the target input force and the first angle characteristic; The device is controlled based on the target driving force.

Images