Endoscope field of view adjustment method, control system, and readable storage medium

Abstract

The application provides an endoscope field of view adjustment method, a control system and a readable storage medium. The endoscope field of view adjustment method comprises the following steps: obtaining a mode selection instruction, the mode selection instruction corresponding to at least two preset modes; switching to the corresponding preset mode; based on the switched preset mode, setting and kinematics calculation of a virtual control object, and converting displacement input of a control master hand into different endoscope movements. In this way, on the one hand, the number of endoscope field of view adjustment modes is increased, which can adapt to the operation habits of different operators; at the same time, based on the virtual control object, the control of the endoscope field of view is more intuitive, which is convenient to use; and the problems of single endoscope field of view adjustment mode, non-intuitive and high threshold to use in the prior art are solved.

Description

Technical Field

[0001] This invention relates to the field of medical device technology, and in particular to an endoscopic field of view adjustment method, control system, and readable storage medium. Background Technology

[0002] Single-port laparoscopic surgery is a minimally invasive surgical technique that uses modern imaging technology and high-tech surgical instruments to perform complex intracavitary surgeries through a cannula or tiny incision.

[0003] In single-port laparoscopic surgery, the adjustment of the endoscopic field of view is crucial to the success rate and efficiency of the entire procedure. Currently, the adjustment methods for the endoscopic field of view are relatively limited, making it difficult to adapt to the different operating habits of various surgeons. The adjustment methods are also not intuitive enough, do not meet the surgeon's expectations, and require considerable practice for adaptation, resulting in a high learning curve. Therefore, it is necessary to improve the methods for adjusting the endoscopic field of view to facilitate surgical procedures.

[0004] In summary, existing technologies suffer from problems such as limited and unintuitive methods for adjusting the endoscopic field of view, and a high learning curve. Summary of the Invention

[0005] This invention provides an endoscope field of view adjustment method, control system, and readable storage medium to solve the problems of the existing endoscope field of view adjustment methods being singular, unintuitive, and having a high learning curve.

[0006] To address the aforementioned technical problems, according to a first aspect of the present invention, an endoscope field of view adjustment method is provided, the endoscope field of view adjustment method comprising the following steps: acquiring a mode selection instruction, the mode selection instruction corresponding to at least two preset modes; switching to the corresponding preset mode; acquiring the displacement of a first joint, the first joint being subordinate to a control master hand; calculating the expected pose of a virtual control object based on the displacement of the first joint; calculating the target pose of a virtual endoscope based on the expected pose of the virtual control object; determining the target pose of the endoscope based on the target pose of the virtual endoscope; and calculating and controlling the displacement of a second joint, the second joint being subordinate to the endoscope, based on the target pose of the endoscope.

[0007] The virtual control object, the algorithm for calculating the expected pose of the virtual control object, and the algorithm for calculating the target pose of the virtual endoscope are all related to the selected preset mode.

[0008] Optionally, the step of determining the target pose of the endoscope based on the target pose of the virtual endoscope specifically includes: constructing the kinematic geometry of the endoscope and determining the motion boundary of the endoscope; if the target pose of the virtual endoscope is within the motion boundary, setting the target pose of the virtual endoscope as the target pose of the endoscope; otherwise, setting the pose within the motion boundary that is closest to the target pose of the virtual endoscope as the target pose of the endoscope.

[0009] Optionally, the endoscopic field of view adjustment method further includes: if the target pose of the virtual endoscope is not within the motion boundary, performing force feedback based on at least a portion of the first joint or restricting the motion degrees of freedom of at least a portion of the first joint.

[0010] Optionally, the preset mode includes a two-hand drag control mode; when switching to the two-hand drag control mode, the virtual control object is the midpoint between the ends of the two control hands. The displacement of the virtual endoscope in the three translational directions is consistent with or proportional to the displacement of the virtual control object in the three translational directions.

[0011] Optionally, the endoscope has more than three degrees of freedom of motion. When switching to the two-hand drag control mode, the step of calculating and controlling the displacement of the second joint based on the target pose of the endoscope includes: determining the displacement of the endoscope in each degree of freedom of motion based on an optimization algorithm; wherein, the objective function is the sum of the terminal evaluation index and the dynamic evaluation index; the terminal evaluation index is an indicator of the degree of overlap between the target pose and the actual pose of the endoscope, and the higher the degree of overlap, the lower the index value; the dynamic evaluation index is the total momentum during the motion process; the optimization direction of the optimization algorithm is to make the objective function as small as possible.

[0012] Optionally, when switching to the two-hand drag control mode, the endoscopic field of view adjustment method further includes the following steps: after a first preset time point, locking at least a portion of the motion degrees of freedom in the first joint, such that the change in the relative distance between the ends of the two control hands is within a first preset range.

[0013] The first preset time point is when the user switches to the two-hand drag control mode, receives a control command that represents the intention to lock, or receives a control command that reflects that the user has begun to adjust the field of view.

[0014] Optionally, the preset mode includes a one-handed matching control mode; when switching to the one-handed matching control mode, the virtual control object coincides with the end of one of the control master hands.

[0015] Optionally, the endoscope field of view adjustment method further includes: when switching to the single-hand matching control mode, adjusting the posture of the end of the control hand so that the posture of the end of the control hand is consistent with the current posture of the endoscope.

[0016] When switching to the single-handed matching control mode, the step of calculating the target pose of the virtual endoscope based on the expected pose of the virtual control object is as follows: the displacement of the virtual endoscope in the three translational directions is consistent with or proportional to the displacement of the virtual control object in the three translational directions; the virtual endoscope does not respond to the displacement of the virtual control object in the corresponding rotational direction in the non-degree-of-freedom rotational direction; the displacement of the virtual endoscope in the degree-of-freedom rotational direction is consistent with the displacement of the virtual control object in the corresponding rotational direction.

[0017] Optionally, when switching to the single-hand matching control mode, the step of calculating and controlling the displacement of the second joint based on the target pose of the endoscope includes: calculating the displacement of the second joint based on the target pose of the endoscope using the inverse kinematic matrix.

[0018] Optionally, when switching to the single-hand matching control mode, the endoscopic field of view adjustment method further includes the following steps: after a second preset time point, locking the other first joints of the end of the control hand, except for the first joint, so that the relative position of the end of the control hand and the console base is locked.

[0019] The second preset time point is when the user switches to the single-handed matching control mode and receives a control command that represents the intention to lock, or when the user receives a control command that reflects that the user has begun to adjust the field of vision.

[0020] Optionally, the preset mode includes a virtual rigid body control mode; when switching to the virtual rigid body control mode, the virtual control object is a virtual rigid body, which has two rod-shaped bodies that overlap with the grip of the control hand.

[0021] The virtual rigid body also has a field of view observation point overlap portion, the center of which coincides with the center of the ends of the two control hands. When switching to the virtual rigid body control mode, the step of calculating the target pose of the virtual endoscope based on the expected pose of the virtual control object is as follows: calculate the target pose of the virtual endoscope, such that the difference between the pose movement trajectory of the observed object of the virtual endoscope relative to the virtual endoscope and the pose movement trajectory of the field of view observation point overlap portion relative to the console base is within a second preset range.

[0022] Optionally, the step of calculating the target pose of the virtual endoscope, such that the difference between the pose movement trajectory of the observed object of the virtual endoscope and the pose movement trajectory of the overlapping part of the field of view observation point is within a second preset range, includes:

[0023] When switching to the virtual rigid body control mode: a virtual rigid body coordinate system is generated, which is fixed to the virtual rigid body; a monitor coordinate system is generated, which is fixed to the lens of the virtual endoscope; the first pose of the grip of the control hand in the virtual rigid body coordinate system is recorded; the second pose of the monitor coordinate system in the virtual rigid body coordinate system is recorded.

[0024] After switching to the virtual rigid body control mode: the motion trajectory of the grip of the main control hand relative to the console base is calculated based on the displacement of the first joint of the main control hand; the pose of the grip of the main control hand in the monitor coordinate system is calculated based on the motion trajectory; alternative poses of the virtual rigid body coordinate system in the monitor coordinate system are generated, and the alternative virtual rigid body coordinate system is optimized so that the pose of the grip of the main control hand described in the alternative virtual rigid body coordinate system has the highest degree of overlap with the first pose; the optimized alternative virtual rigid body coordinate system is set as the virtual rigid body coordinate system, and the third pose of the monitor coordinate system in the virtual rigid body coordinate system is recorded; the target pose of the virtual endoscope is calculated based on the difference between the third pose and the second pose.

[0025] Optionally, when switching to the virtual rigid body control mode, the step of calculating and controlling the displacement of the second joint based on the target pose of the endoscope includes: calculating the displacement of the second joint based on the target pose of the endoscope using the inverse kinematic matrix.

[0026] Optionally, when switching to the virtual rigid body control mode, the endoscopic field of view adjustment method further includes the following steps: after a third preset time point, locking at least a portion of the motion degrees of freedom in the first joint, so that the deviation between the grips of the two control hands and the rod-shaped body is within a third preset range.

[0027] The third preset time point is when the virtual rigid body control mode is switched to, when a control command representing the locking intention is received, or when a control command reflecting that the user has begun to adjust the field of view is received.

[0028] To address the aforementioned technical problems, according to a second aspect of the present invention, a control system is provided, comprising two control hands, a processor, and an endoscope, wherein the processor is configured to execute the aforementioned endoscope field of view adjustment method based on input instructions acquired by the control hands to control the endoscope, and the degrees of freedom of movement of the endoscope are adaptively set.

[0029] To address the aforementioned technical problems, according to a third aspect of the present invention, a readable storage medium is provided, on which a program is stored, wherein when the program is executed, the aforementioned endoscopic field of view adjustment method is performed.

[0030] Compared with existing technologies, the present invention provides an endoscopic field of view adjustment method, control system, and readable storage medium. The endoscopic field of view adjustment method includes the following steps: acquiring a mode selection command, the mode selection command corresponding to at least two preset modes; switching to the corresponding preset mode; and setting and kinematically calculating a virtual control object based on the switched preset mode, converting the displacement input of the controlling hand into different endoscopic movements. This configuration, on the one hand, increases the number of endoscopic field of view adjustment methods, adapting to the operating habits of different operators; on the other hand, based on the virtual control object, it makes the control of the endoscopic field of view more intuitive and easier to learn; thus solving the problems of limited and unintuitive endoscopic field of view adjustment methods and high learning curves in existing technologies. Attached Figure Description

[0031] Those skilled in the art will understand that the accompanying drawings are provided to better understand the invention and do not constitute any limitation on the scope of the invention. Wherein:

[0032] Figure 1 This is a flowchart illustrating an endoscopic field of view adjustment method according to an embodiment of the present invention;

[0033] Figure 2 This is a schematic diagram of a control system according to an embodiment of the present invention;

[0034] Figure 3 This is a schematic diagram of the structure of a snake bone endoscope according to an embodiment of the present invention;

[0035] Figure 4 This is a schematic diagram illustrating the difference in position between a virtual endoscope and an endoscope according to an embodiment of the present invention;

[0036] Figure 5 This is a control flowchart of a control system according to an embodiment of the present invention;

[0037] Figure 6 This is a flowchart of the main hand motion control module according to an embodiment of the present invention;

[0038] Figure 7This is a schematic diagram of a virtual control object in a two-hand drag control mode according to an embodiment of the present invention;

[0039] Figure 8 This is a schematic diagram illustrating the calculation of the objective function of an optimization algorithm according to an embodiment of the present invention;

[0040] Figure 9 This is a schematic diagram showing that part of the first joint is locked according to an embodiment of the present invention;

[0041] Figure 10 This is a schematic diagram of a virtual control object in virtual rigid body control mode according to an embodiment of the present invention;

[0042] Figure 11 This is a schematic diagram of the coordinate systems in a virtual rigid body control mode according to an embodiment of the present invention. Detailed Implementation

[0043] To make the objectives, advantages, and features of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the drawings are all in a very simplified form and are not drawn to scale, and are only used to facilitate and clarify the explanation of the embodiments of this invention. Furthermore, the structures shown in the drawings are often part of the actual structures. In particular, different figures may emphasize different aspects and may sometimes use different scales.

[0044] As used in this invention, the singular forms “a,” “an,” and “the” include plural objects; the term “or” is generally used to mean “and / or”; the term “a number” is generally used to mean “at least one”; the term “at least two” is generally used to mean “two or more”; furthermore, the terms “first,” “second,” and “third” are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, features defined with "first," "second," and "third" may explicitly or implicitly include one or at least two of those features. The term "proximal" typically refers to the end closer to the operator, and the term "distal" typically refers to the end closer to the patient. "One end" and "the other end," as well as "proximal" and "distal," generally refer to two corresponding parts, including not only endpoints. The terms "installed," "connected," and "linked" should be interpreted broadly. For example, they can be fixed connections, detachable connections, or integral connections; they can be mechanical connections or electrical connections; they can be direct connections or indirect connections through an intermediate medium; they can be internal connections between two elements or interactions between two elements. Furthermore, as used in this invention, the placement of one element on another element generally only indicates a connection, coupling, cooperation, or transmission relationship between the two elements, and the connection, coupling, cooperation, or transmission between the two elements can be direct or indirect through an intermediate element. It should not be construed as indicating or implying a spatial positional relationship between the two elements, i.e., one element can be located arbitrarily inside, outside, above, below, or to one side of another element, unless otherwise explicitly stated. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0045] The core idea of ​​this invention is to provide an endoscope field of view adjustment method, control system and readable storage medium to solve the problems of the existing endoscope field of view adjustment methods being single, unintuitive and having a high learning threshold.

[0046] The following description refers to the accompanying drawings.

[0047] Please refer to Figure 1 This embodiment provides a method for adjusting the endoscopic field of view, which includes the following steps:

[0048] S10 obtains a mode selection command, which corresponds to at least two preset modes.

[0049] S20 switches to the corresponding preset mode.

[0050] S30 acquires the displacement of the first joint, which is subordinate to the control master.

[0051] S40 calculates the expected pose of the virtual control object based on the displacement of the first joint.

[0052] The S50 calculates the target pose of the virtual endoscope based on the expected pose of the virtual control object.

[0053] The S60 determines the target pose of the endoscope based on the target pose of the virtual endoscope.

[0054] Additionally, the S80 calculates and controls the displacement of the second joint based on the target pose of the endoscope, and the second joint is subordinate to the endoscope.

[0055] Among them, the virtual control object, the algorithm for calculating the expected pose of the virtual control object, and the algorithm for calculating the target pose of the virtual endoscope are related to the selected preset mode.

[0056] In step S10, the mode selection command can originate from either the user or the upstream control algorithm. For example, the mode selection command can be obtained from the user interface, or it can be obtained from the upstream control algorithm via a specific communication interface.

[0057] In step S30, both "joint" and "displacement" should be interpreted broadly. In some embodiments, "joint" refers to a mechanical structure that can only rotate; in other embodiments, "joint" can also refer to a mechanical structure that can perform translational or combined motion. "Displacement" should also be understood as translational displacement, rotational displacement, or combined displacement, depending on the specific mechanical structure. The following content in this specification will also follow this understanding. Because the surgeon manipulates the end effector of the controlling hand, the movement of the end effector causes the movement of the corresponding first joint, resulting in displacement of at least a portion of the first joint. This displacement is read by a sensor and input to the corresponding processor. In step S30, "acquiring the displacement of the first joint" can also be understood as acquiring "the measured value of the displacement of the first joint."

[0058] In step S40, by setting a virtual control object, the control target of this embodiment is transformed from the displacement of several independent actuators into a virtual control object. As will be explained later, this virtual control object corresponds to a specific real object. This configuration makes the operator's operation more targeted and facilitates beginners' understanding and prediction of the actual effects of their current operation, thereby solving problems in the prior art. The virtual control object should be understood as follows: broadly speaking, the movement of the virtual control object has a simple mapping relationship with the movement of the controlling hand. Based on this, the present invention has the advantage of being intuitive and easy to use. Specifically, its whole or a part thereof has only a linear relationship with the movement of the controlling hand.

[0059] Steps S50 and S60 serve two purposes. First, they utilize a specific algorithm to reverse-engineer intuitive, operator-friendly operations into the expected pose of the endoscope, facilitating understanding and correction by developers. Second, they use a virtual endoscope as a bridge to sever the connection between the input commands and the endoscope, creating conditions for the intervention of safety mechanisms. Specific details of step S60 will be described later.

[0060] Step S80 ultimately converts the expected pose of the endoscope into the displacement of each actuator, thereby achieving the control objective.

[0061] For ease of understanding, the control system implementing this method is as follows: Figure 2 As shown, the single-arm, single-port surgical system consists of three subsystems: the surgeon's console 1, the patient carriage 2, and the imaging carriage 3. In clinical practice, the surgeon inserts multiple instruments into the abdominal cavity through a single incision to perform the surgery. The surgeon uses the main hand on the surgeon's console to control the position and orientation of the endoscope at the patient end to obtain the optimal surgical field of view.

[0062] The endoscope is mounted on the patient trolley 2. In one embodiment, please refer to... Figure 3 The endoscope is a snake-bone endoscope 4. The bending joint of the snake-bone endoscope 4 comprises two snake-bone units, connected by a parallel joint 41 and a pitching / tilting joint 42. This allows the snake-bone endoscope to have more degrees of freedom, making it more flexible than traditional surgical endoscopes and providing the surgeon with a wider field of view. However, this embodiment can obviously also be used in conjunction with endoscopes of other structures. For ease of understanding, the snake-bone endoscope 4 will be used as an example in the following description, but this embodiment is not limited to snake-bone endoscope 4.

[0063] Furthermore, step S60, which determines the target pose of the endoscope based on the target pose of the virtual endoscope, specifically includes:

[0064] S601 constructs the kinematic geometry of the endoscope and determines the motion boundary of the endoscope.

[0065] S602 If the target pose of the virtual endoscope is within the motion boundary, set the target pose of the virtual endoscope as the target pose of the endoscope; otherwise, set the pose within the motion boundary that is closest to the target pose of the virtual endoscope as the target pose of the endoscope.

[0066] The "closest" setting can be configured according to actual needs. For example, the translation distance between two poses, the combined translation and rotation displacement of two poses, or the displacement projection of two poses along a set measurement direction can be used as indicators to determine the closest.

[0067] Understandably, in step S601, other conditions can be added to determine the movement boundaries, such as some organs that the endoscope should avoid in the patient's body cavity.

[0068] Please refer to Figure 4 , Figure 4 This demonstrates the difference in position between the virtual endoscope 51 and the endoscope 52 when the virtual endoscope 51 moves beyond the motion boundary 53.

[0069] Based on the length and angle limits of each joint of the snake-bone endoscope, a corresponding virtual solid geometry is constructed to define the motion boundary of the snake-bone endoscope. In each control cycle, the motion boundary of the endoscope's current position is detected. When it is detected that the endoscope has reached the boundary, the tendency of the endoscope to continue moving outward from the boundary is restricted, causing it to stop at the boundary.

[0070] If the control hand still tends to move outward from the boundary, the endoscope will select the point on the boundary closest to the desired position as the target position, allowing it to slide on the boundary and exert boundary force on the control hand to guide the surgeon to move into the working area.

[0071] The specific implementation process is as follows: Assume a virtual endoscope with the same configuration as a snake endoscope. When the motion command issued by the master end exceeds the motion boundary of the slave end, the slave end automatically generates a virtual endoscope corresponding to the over-limit command. The torque of each joint of the virtual endoscope is calculated based on the current pose of the virtual endoscope, and the torque on the end of the virtual endoscope is obtained.

[0072] Please continue to refer to this. Figure 1 The endoscopic field of view adjustment method further includes: S70 If the target pose of the virtual endoscope is not within the motion boundary, force feedback is provided based on at least a portion of the first joint, or the motion degrees of freedom of at least a portion of the first joint are restricted. "At least a portion of the motion degrees of freedom of the first joint" should be understood as the motion degrees of freedom in each joint being fully or partially restricted; movement in one direction can be restricted, but movement in the opposite direction is not restricted. Force feedback should be understood as follows: when the control hand moves beyond the boundary, the control hand simultaneously experiences a restoring force pointing inwards from the boundary. If the control hand is no longer subjected to other forces (e.g., the operator releases their grip), the control hand can slowly return to the boundary.

[0073] Assuming the control input of the controlling hand is ψ, the desired motion velocity of the virtual endoscope satisfies the following equation of motion:

[0074]

[0075] Where J is the Jacobian matrix, q virtualLet q be the desired joint angle of the virtual endoscope. Of course, it is not excluded that in certain embodiments the endoscope has translational capabilities, therefore q virtual This can also be broadly understood as the generalized composite displacement of the actuator of the virtual endoscope. For ease of understanding, the calculations below will be performed according to the desired joint angle.

[0076] By solving the above equation, the desired velocity of the virtual endoscope moving to the desired position can be obtained. The desired joint angle q can be obtained by integration. virtual From the equation:

[0077] τ virtual =-K(q) endo -q virtual )

[0078] The torque τ acting on the virtual endoscope can be obtained. virtual Where K represents the relevant control parameters set in the endoscope-based driving algorithm, and q endo This refers to the joint angle within the motion boundary, which is the angle mentioned earlier that is closest to the expected angle.

[0079] The force F acting on the control input ψ The torque acting on the virtual endoscope satisfies the following equation:

[0080] F ψ =J T τ virtual

[0081] Simply put, the feedback force received by the control hand is proportional to the distance the virtual endoscope deviates from the motion boundary.

[0082] The feedback force at the slave boundary of the master controller is related to how the master controller is used as a control input. The form of the force on the master controller will be different in different control modes.

[0083] The above describes one way to implement force feedback. Depending on the actual situation, there may be other force feedback methods, such as using a constant force for feedback; or, as mentioned above, locking the degrees of freedom, that is, when moving to a critical position, under the force and torque that the human arm can output, the relevant joints of the controlling hand will no longer displace.

[0084] In one embodiment, the preset modes include a two-hand drag control mode, a one-hand matching control mode, and a virtual rigid body control mode. That is, the control flow of this embodiment can also follow... Figure 5The process involves several steps, including: selecting the control mode; receiving endoscopic images via an image trolley; the physician controlling the primary endoscope's movements based on their desired field of view; and adjusting the secondary endoscope using input signals from the primary endoscope's movement control module. The physician then confirms whether the current field of view matches their expectations based on the endoscopic feedback; if not, further adjustments are made. Simultaneously, the secondary endoscope also provides feedback via the primary endoscope's movement control module, alerting the physician to any critical points in the movement's boundaries.

[0085] It is understood that in other embodiments, both of the above modes may be included, or one of the above modes may be included in combination with other control modes, such as a control mode that directly controls joint displacement.

[0086] Figure 5 In the middle, the main hand motion control module 6 is as follows: Figure 6 The system operates as shown. First, it acquires the master hand motion data 64. Then, based on the boundary protection module 63 and the constraint feedback module 62, it calculates the corresponding constraint conditions, specifically including the boundary protection constraint force Fb, the boundary protection constraint torque Mb, the constraint feedback force Fc, and the constraint feedback torque Mc. This data is then transmitted to the relevant master hand joint 65 via the force guidance module 61 to achieve feedback or constraint. The master hand joint 65 is also known as the first joint. After the master hand motion data 64 is converted to the slave coordinate system, it undergoes low-pass filtering to remove high-frequency components and eliminate glitches, making the motion commands issued to the slave end smoother and avoiding sudden jumps. Then, it checks whether the desired motion command exceeds the endoscope's motion boundary. If it does, the endoscope will remain on the boundary and will not continue to move outwards.

[0087] Furthermore, when switching to the two-hand drag control mode, the virtual control object is the midpoint between the ends of the two main control hands. Please refer to [link / reference]. Figure 7 , Figure 7 The image shows two master control arms 7 and a virtual control object 71. The virtual control object 71 corresponds to the lens of the endoscope.

[0088] When switching to the two-hand drag control mode, the steps for calculating the target pose of the virtual endoscope based on the expected pose of the virtual control object are as follows: the displacement of the virtual endoscope in the three translational directions is consistent with or proportional to the displacement of the virtual control object in the three translational directions.

[0089] In other words, in the two-hand drag control mode, the operator's experience is like directly pulling the camera movement.

[0090] When the endoscope is a snake-bone endoscope, the endoscope has more than three degrees of freedom of motion. When switching to the two-hand drag control mode, the steps for calculating and controlling the displacement of the second joint based on the target pose of the endoscope include:

[0091] Based on the optimization algorithm, the displacement of the endoscope in each degree of freedom of motion is determined. The objective function is the sum of the terminal evaluation index and the dynamic evaluation index. The terminal evaluation index is an indicator of the degree of overlap between the target pose and the actual pose of the endoscope. The higher the degree of overlap, the lower the index value. The dynamic evaluation index is the total momentum during the motion process. The optimization direction of the optimization algorithm is to make the objective function as small as possible.

[0092] In one specific embodiment, based on feedback from the current endoscopic field of view, the surgeon manipulates the primary hand to move closer to the desired surgical field. The motion data parameters of the primary hand are processed to generate an optimal problem containing an objective function and constraints. By solving the optimal problem, the output joints are determined, the corresponding motor data is calculated, and motion commands are issued to the slave end. The endoscope adjusts its pose and performs boundary detection according to the commands.

[0093] When the center point of the line connecting the two hands of the console is used as the control input for adjusting the endoscope's field of view, it is a three-degree-of-freedom control of a five-degree-of-freedom control. The control input is greater than the control output, so different control outputs can be selected and coupled to achieve the desired control effect.

[0094] Therefore, a controller is designed based on the actual motion structure of the controlled object, namely the snake-shaped endoscope. First, an objective function is defined to quantitatively describe the controller's performance:

[0095]

[0096] The first item is the terminal evaluation index, which ensures that the system's terminal state is as close as possible to the given terminal state, reflecting the smoothness and accuracy of the adjustment process. The second item is the dynamic evaluation index, which ensures the overall performance of the system and reflects the magnitude of momentum during the process. q is the joint angle of the endoscope, u is the expected angular velocity of the endoscope joint, Φ and L are the relevant calculation functions, the specific forms of which are determined by the specific structure of the endoscope. Φ describes the system's terminal state, L describes the instantaneous value of momentum, and tf represents the motion termination time.

[0097] The performance is optimal when the objective function reaches its minimum. Controller design can be transformed into an optimization problem where the terminal is always free but its state is fixed.

[0098] By solving the optimal problem, the controller is designed and functions as a joint optimization selector. In this mode, we couple different control outputs (endoscope joint angles) to adjust the endoscope field of view to the desired position and optimize the performance indicators. We minimize the change in the endoscope joint angle so that the lens reaches the doctor's desired position.

[0099] The objective function calculation process of the above optimization algorithm can be referred to Figure 8 To understand.

[0100] When switching to the two-hand drag control mode, the endoscopic field of view adjustment method further includes the following steps: After a first preset time point, at least a portion of the degrees of freedom of motion in the first joint is locked, such that the change in the relative distance between the ends of the two control hands is within a first preset range. For example, please refer to [reference needed]. Figure 7 When the surgeon switches from instrument control to endoscopic control, the initial positional relationship of the two main hands in Cartesian space is recorded:

[0101] Δx0=x 0，right -x 0，left

[0102] In the subsequent control process, the right primary hand is used as the reference point. The desired position of the left primary hand is calculated and adjusted based on the initial positional relationship, ensuring that the positional relationship of the two primary hands always satisfies the initial positional relationship during control. That is, based on f(x) 0，left ,x 0，right ) = 0 determines the expression of the function f, for example: f(x) left x right )=|x right -x left -Δx0| consistently satisfies f(x) throughout subsequent operations. left ,x right The constraint condition is f(x) = 0. Understandably, in practical applications, achieving a strict equality of 0 is a difficult condition to fulfill; generally, f(x) is used instead. left ,x right The constraint condition is ≤ΔE1, where ΔE1 is the first preset range, and ΔE1 is determined by the actual situation. In this section, x is the displacement vector, representing displacement in three directions, and its specific data content is determined based on the choice of coordinate system and the logic of the program implementation.

[0103] The first preset time point is when switching to the two-hand drag control mode, meaning that the left and right primary hands are already relatively locked when switching to this mode. Alternatively, the first preset time point is when a control command indicating a locking intention is received. In one exemplary embodiment, a lock / unlock button is present on the user interface, allowing the user to switch the current constraint state. Another option is when a control command indicating the user has begun adjusting their field of view is received, such as when a field of view adjustment displacement is input. These different solutions can correspond to different practical situations. For safety reasons, the first solution can be chosen; for operational freedom, the latter two solutions can be selected.

[0104] Furthermore, the preset modes include a one-handed matching control mode; when switching to the one-handed matching control mode, the virtual control object overlaps with the end of one of the main control hands.

[0105] In other words, the virtual control object is controlled by controlling the end effector of one of the master control hands. The end effector of the master control hand has three translational degrees of freedom and three rotational degrees of freedom. Here, the end effector of the master hand should be understood as a 3D structure, such as a joystick. The master control hand can be either the left or right hand.

[0106] The endoscopic field of view adjustment method also includes: when switching to the single-handed matching control mode, adjusting the posture of the end effector of the master control hand so that the posture of the end effector of the master control hand is consistent with the current posture of the endoscope. This configuration is more conducive to the operator's intuitive understanding and prediction of the results of the next operation. Here, the master control hand refers to the master control hand used to control the virtual control object. In different embodiments, the above step may not be present. The coordinate system describing the posture of the end effector of the master control hand and the coordinate system describing the posture of the endoscope can be the same, for example, both can be base coordinate systems, or they can be different, for example, the former's x-direction is the same as the operator's facing direction, while the latter's x-direction is from the patient carriage towards the patient.

[0107] The snake-bone endoscope does not include rotation among its five degrees of freedom. To allow the master hand to match the endoscope's pose, the seven joints of the master hand will be subject to corresponding motion constraints in this control mode. The positions of the seven joints of the master hand should be determined by the endoscope's degrees of freedom. Considering that the endoscope originates from its base coordinate system, it can only perform rotation... x (Θ x Rot y (θ y The pose of the end-view camera is obtained by rotating the image from the field of view coordinate system. Correspondingly, for the main hand, only a Rotation transformation is performed starting from the field of view coordinate system. x (Θ x Rot y (θy The rotational transformation should yield a seven-joint coordinate system aligned with the current orientation (z-axis direction), from which the positions of the seven joints of the master hand can be inversely determined. Where, Θ x θ represents the angle of rotation of the endoscope around the x-axis. y Rot represents the angle of rotation of the endoscope around the y-axis. x Rot represents the rotation matrix determined based on the rotation angle in the x-direction. y This represents a rotation matrix determined based on the rotation angle in the y-direction.

[0108] In one embodiment, when switching to the single-handed matching control mode, the steps for calculating the target pose of the virtual endoscope based on the expected pose of the virtual control object are as follows: the displacement of the virtual endoscope in the three translational directions is consistent with or proportional to the displacement of the virtual control object in the three translational directions; the virtual endoscope does not respond to the displacement of the virtual control object in the corresponding rotational direction in the non-degree-of-freedom rotational direction; the displacement of the virtual endoscope in the degree-of-freedom rotational direction is consistent with the displacement of the virtual control object in the corresponding rotational direction.

[0109] The non-freedom rotation direction and the freedom rotation direction of the virtual endoscope are determined by the mechanical structure of the endoscope. For example, the non-freedom rotation direction of the snake bone endoscope is the direction of rotation around its own axis, and the freedom rotation direction of the snake bone endoscope is the remaining two directions.

[0110] In one embodiment, after the master-slave posture matching is completed, the doctor can manipulate the master hand to move towards their desired field of vision. The control cart records the movement data of the master hand, maps it to the slave endoscope, and then calculates the joint motor torque output of the endoscope based on the desired pose of the endoscope, adjusting the endoscope accordingly.

[0111] Specifically, in the single-hand matching module, the pose change of the master hand in the monitor's field of view coordinate system is mapped to the pose change of the slave endoscope in the base coordinate system. The mapping ratio of master-slave pose changes is 1:1, and this ratio can be adjusted appropriately according to the surgeon's operating habits. The movement and rotation direction of the slave endoscope in space is consistent with the direction of the master hand. The mapping ratio coefficient between the two can be adjusted within a certain range according to the surgeon's habits.

[0112] When switching to the single-handed matching control mode, the steps of calculating and controlling the displacement of the second joint based on the target pose of the endoscope include: calculating the displacement of the second joint based on the target pose of the endoscope using the inverse kinematic matrix.

[0113] Similarly, when switching to the single-hand matching control mode, the endoscopic field of view adjustment method also includes the following steps: after the second preset time point, lock the first joints other than the first joint at the end of the control master hand, so that the relative position of the end of the control master hand and the console base is locked.

[0114] The single-hand matching module uses a seven-DOF master hand as control input and a five-DOF snake-bone endoscope as control output. To avoid sudden changes in endoscope position during intraoperative instrument switching and reduce surgical risks, when the master hand switches from instrument control to endoscope control, it first adjusts its posture. After adjustment, the first non-distal joint is locked, thereby reducing surgical risks. Figure 9 A schematic diagram showing a portion of the first joint being locked in one embodiment is shown.

[0115] The second preset time point is when the user switches to the single-handed matching control mode and receives a control command that represents the intention to lock, or when the user receives a control command that reflects that the user has begun to adjust the field of vision.

[0116] The second preset time point can be understood by referring to the first preset time point.

[0117] Furthermore, when switching to the virtual rigid body control mode, the virtual control object is a virtual rigid body, which has two rod-shaped bodies that overlap with the grip of the main control hand. In the initial state, the rod-shaped bodies and the grip of the main control hand always overlap, so the shape of the virtual rigid body itself is not fixed.

[0118] The virtual rigid body also has a field of view observation point overlap, the center of which coincides with the center of the ends of the two control hands. When switching to the virtual rigid body control mode, the steps for calculating the target pose of the virtual endoscope based on the expected pose of the virtual control object are as follows: calculate the target pose of the virtual endoscope so that the difference between the pose movement trajectory of the observed object of the virtual endoscope relative to the virtual endoscope and the pose movement trajectory of the field of view observation point overlap relative to the control console base is within a second preset range.

[0119] In other words, the operator's experience in virtual rigid body control mode is that he is manipulating the observed object.

[0120] Understandably, the aforementioned rod-shaped body 86 and the overlapping portion 85 of the viewpoint are merely concepts set up to describe the motion rules of the virtual rigid body; the virtual rigid body itself may not actually have a fixed shape. The rod-shaped body 86 and the overlapping portion 85 of the viewpoint can be referenced... Figure 10 To understand.

[0121] Since the observed object does not actually move unless the patient is lying on a freely movable operating table—a situation not discussed in this manual—the fundamental reason for achieving the aforementioned effect is actually the movement of the endoscope. The specific implementation method is as follows.

[0122] The step of calculating the target pose of the virtual endoscope, such that the difference between the pose movement trajectory of the observed object and the pose movement trajectory of the overlapping part of the field of view observation point is within a second preset range, includes:

[0123] When switching to virtual rigid body control mode: a virtual rigid body coordinate system is generated and fixed to the virtual rigid body; a monitor coordinate system is generated and fixed to the lens of the virtual endoscope; the first pose of the control hand grip in the virtual rigid body coordinate system is recorded; the second pose of the monitor coordinate system in the virtual rigid body coordinate system is recorded.

[0124] After switching to virtual rigid body control mode: the motion trajectory of the main hand grip relative to the console base is calculated based on the displacement of the first joint of the main hand; the pose of the main hand grip in the monitor coordinate system is calculated based on the motion trajectory; alternative virtual rigid body coordinate systems are generated in the monitor coordinate system, and the alternative virtual rigid body coordinate systems are optimized to maximize the overlap between the pose of the main hand grip described in the alternative virtual rigid body coordinate system and the first pose; the optimized alternative virtual rigid body coordinate system is set as the virtual rigid body coordinate system, and the third pose of the monitor coordinate system in the virtual rigid body coordinate system is recorded; the target pose of the virtual endoscope is calculated based on the difference between the third pose and the second pose.

[0125] In other words, the monitor coordinate system is first used as a reference, and the virtual rigid body coordinate system moves with the movement of the control hand, maintaining a relative positional relationship. Then, the virtual rigid body coordinate system is used as a reference to calculate the movement path of the monitor coordinate system, providing a basis for the final calculation of the target pose of the virtual endoscope.

[0126] The above content can also be used as a reference. Figure 11 To understand, in Figure 11 The diagram shows the monitor coordinate system 81, the virtual rigid body coordinate system 82, the control left hand 83, the control right hand 84, and the overlapping part of the field of view observation point 85.

[0127] Similarly, when switching to virtual rigid body control mode, the steps of calculating and controlling the displacement of the second joint based on the target pose of the endoscope include: calculating the displacement of the second joint based on the target pose of the endoscope using the inverse kinematic matrix.

[0128] When switching to virtual rigid body control mode, the endoscopic field of view adjustment method also includes the following steps: after the third preset time point, lock at least a portion of the motion degrees of freedom in the first joint so that the deviation between the grips of the two control hands and the rod-like body is within the third preset range.

[0129] The third preset time point is when switching to the virtual rigid body control mode, when a control command representing the locking intention is received, or when a control command reflecting that the user has begun to adjust the field of view is received.

[0130] The third preset time point can also be understood in accordance with the first preset time point.

[0131] This embodiment also provides a control system, including two control hands, a processor and an endoscope. The processor is used to execute the above-described endoscope field of view adjustment method based on input instructions obtained by the control hands to control the endoscope. The degrees of freedom of movement of the endoscope are adaptively set.

[0132] Specifically, the processor includes a main hand motion control module, a joint optimization selection module, a posture matching module, and a virtual viewpoint positioning module. The main hand motion control module outputs control signals to the first joint based on two strategies: boundary protection and constraint feedback, to provide force feedback or restrict at least a portion of the degrees of freedom in the first joint. The joint optimization selection module is used to select and control the displacement of the second joint in the two-hand drag control mode; the specific implementation can be understood by referring to the previous description. The posture matching module is used to select and control the displacement of the second joint in the one-hand matching control mode; the specific implementation can be understood by referring to the previous description. The virtual viewpoint positioning module is used to select and control the displacement of the second joint in the virtual rigid body control mode; the specific implementation can be understood by referring to the previous description.

[0133] This embodiment also provides a readable storage medium on which a program is stored. When the program is run, the above-described endoscopic field of view adjustment method is executed.

[0134] The aforementioned control system and readable storage medium, because they are capable of executing the endoscopic field of view adjustment method or can provide a program for executing the endoscopic field of view adjustment method, can also solve the problems existing in the prior art.

[0135] In summary, this embodiment provides an endoscopic field of view adjustment method, a control system, and a readable storage medium. The endoscopic field of view adjustment method includes the following steps: acquiring a mode selection command, which corresponds to at least two preset modes; switching to the corresponding preset mode; and setting and kinematically calculating a virtual control object based on the switched preset mode, converting the displacement input of the control hand into different endoscopic movements. This configuration increases the number of endoscopic field of view adjustment methods, adapting to different surgeons' operating habits; simultaneously, based on the virtual control object, it makes the control of the endoscopic field of view more intuitive and easier to learn; thus solving the problems of limited and unintuitive endoscopic field of view adjustment methods and high learning curves in existing technologies.

[0136] The above description is only a description of preferred embodiments of the present invention and is not intended to limit the scope of the present invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the present invention.

[0137] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0138] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0139] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for adjusting the field of view of an endoscope, characterized in that, The method for adjusting the endoscopic field of view includes the following steps: Obtain a mode selection instruction, wherein the mode selection instruction corresponds to at least two preset modes; Switch to the corresponding preset mode; Obtain the displacement of the first joint, which is subordinate to the main control hand; The expected pose of the virtual control object is calculated based on the displacement of the first joint; The target pose of the virtual endoscope is calculated based on the expected pose of the virtual control object. The target pose of the endoscope is determined based on the target pose of the virtual endoscope. as well as, The displacement of the second joint is calculated and controlled based on the target pose of the endoscope, and the second joint is subordinate to the endoscope; The virtual control object, the algorithm for calculating the expected pose of the virtual control object, and the algorithm for calculating the target pose of the virtual endoscope are related to the selected preset mode. The preset mode includes a two-hand drag control mode, in which the virtual control object is the midpoint between the ends of the two main control hands when switched to the two-hand drag control mode; or, the preset mode includes a virtual rigid body control mode, in which the virtual control object is a virtual rigid body when switched to the virtual rigid body control mode. The endoscope has more than three degrees of freedom of motion. When switching to the two-hand drag control mode, the steps of calculating and controlling the displacement of the second joint based on the target pose of the endoscope include: Based on the optimization algorithm, the displacement of the endoscope in each degree of freedom of motion is determined; The optimization algorithm aims to minimize the objective function, which is used to quantitatively describe the controller's performance. When the objective function reaches its minimum value, the performance is optimal.

2. The endoscopic field of view adjustment method according to claim 1, characterized in that, The step of determining the target pose of the endoscope based on the target pose of the virtual endoscope specifically includes: Construct the kinematic geometry of the endoscope, and determine the motion boundaries of the endoscope; and, If the target pose of the virtual endoscope is within the motion boundary, the target pose of the virtual endoscope is set as the target pose of the endoscope; otherwise, the pose within the motion boundary that is closest to the target pose of the virtual endoscope is set as the target pose of the endoscope.

3. The endoscopic field of view adjustment method according to claim 2, characterized in that, The endoscopic field of view adjustment method also includes: If the target pose of the virtual endoscope is not within the motion boundary, force feedback is performed based on at least a portion of the first joint or the motion degrees of freedom of at least a portion of the first joint are restricted.

4. The endoscopic field of view adjustment method according to any one of claims 1 to 3, characterized in that, When switching to the two-hand drag control mode, the displacement of the virtual endoscope in the three translational directions is consistent with or proportional to the displacement of the virtual control object in the three translational directions.

5. The endoscopic field of view adjustment method according to claim 4, characterized in that, The objective function is the sum of the terminal evaluation index and the dynamic evaluation index; the terminal evaluation index is an indicator of the degree of overlap between the target pose of the endoscope and the actual pose of the endoscope, and the higher the degree of overlap, the lower the index value; the dynamic evaluation index is the total momentum during the motion process.

6. The endoscopic field of view adjustment method according to claim 4, characterized in that, When switching to the two-hand drag control mode, the endoscopic field of view adjustment method further includes the following steps: After a first preset time point, at least a portion of the motion degrees of freedom in the first joint are locked, such that the change in the relative distance between the ends of the two control hands is within a first preset range. The first preset time point is when the user switches to the two-hand drag control mode, receives a control command that represents the intention to lock, or receives a control command that reflects that the user has begun to adjust the field of view.

7. The endoscopic field of view adjustment method according to any one of claims 1 to 3, characterized in that, The preset mode also includes a one-handed matching control mode; When switching to the single-hand matching control mode, the virtual control object coincides with the end of one of the main control hands; When switching to the single-hand matching control mode, the posture of the end of the control hand is adjusted so that the posture of the end of the control hand is consistent with the current posture of the endoscope.

8. The endoscopic field of view adjustment method according to claim 7, characterized in that, When switching to the single-handed matching control mode, the step of calculating the target pose of the virtual endoscope based on the expected pose of the virtual control object is as follows: the displacement of the virtual endoscope in the three translational directions is consistent with or proportional to the displacement of the virtual control object in the three translational directions; the virtual endoscope does not respond to the displacement of the virtual control object in the corresponding rotational direction in the non-degree-of-freedom rotational direction; the displacement of the virtual endoscope in the degree-of-freedom rotational direction is consistent with the displacement of the virtual control object in the corresponding rotational direction.

9. The endoscopic field of view adjustment method according to claim 7, characterized in that, When switching to the single-handed matching control mode, the steps of calculating and controlling the displacement of the second joint based on the target pose of the endoscope include: The displacement of the second joint is calculated based on the kinematic inverse matrix and the target pose of the endoscope.

10. The endoscopic field of view adjustment method according to claim 7, characterized in that, When switching to the single-hand matching control mode, the endoscopic field of view adjustment method further includes the following steps: After the second preset time point, lock the other first joints of the end of the control hand except for the first joint, so that the relative position of the end of the control hand and the console base is locked. The second preset time point is when the user switches to the single-handed matching control mode and receives a control command that represents the intention to lock, or when the user receives a control command that reflects that the user has begun to adjust the field of vision.

11. The method for adjusting the endoscopic field of view according to any one of claims 1 to 3, characterized in that, The virtual rigid body has two rod-shaped bodies, each of which overlaps with the grip of the main control hand; The virtual rigid body also has a field of view observation point overlap portion, the center of which coincides with the center of the ends of the two control hands. When switching to the virtual rigid body control mode, the step of calculating the target pose of the virtual endoscope based on the expected pose of the virtual control object is as follows: calculate the target pose of the virtual endoscope, such that the difference between the pose movement trajectory of the observed object of the virtual endoscope relative to the virtual endoscope and the pose movement trajectory of the field of view observation point overlap portion relative to the console base is within a second preset range.

12. The endoscopic field of view adjustment method according to claim 11, characterized in that, The step of calculating the target pose of the virtual endoscope, such that the difference between the pose movement trajectory of the observed object and the pose movement trajectory of the overlapping part of the field of view observation point is within a second preset range, includes: When switching to the virtual rigid body control mode: a virtual rigid body coordinate system is generated, which is fixed to the virtual rigid body; a monitor coordinate system is generated, which is fixed to the lens of the virtual endoscope; the first pose of the grip of the control hand in the virtual rigid body coordinate system is recorded; the second pose of the monitor coordinate system in the virtual rigid body coordinate system is recorded. After switching to the virtual rigid body control mode: the motion trajectory of the grip of the main control hand relative to the console base is calculated based on the displacement of the first joint of the main control hand; the pose of the grip of the main control hand in the monitor coordinate system is calculated based on the motion trajectory; alternative poses of the virtual rigid body coordinate system in the monitor coordinate system are generated, and the alternative virtual rigid body coordinate system is optimized so that the grip pose of the main control hand described in the alternative virtual rigid body coordinate system has the highest degree of overlap with the first pose; the optimized alternative virtual rigid body coordinate system is set as the virtual rigid body coordinate system, and the third pose of the monitor coordinate system in the virtual rigid body coordinate system is recorded; the target pose of the virtual endoscope is calculated based on the difference between the third pose and the second pose.

13. The endoscopic field of view adjustment method according to claim 11, characterized in that, When switching to the virtual rigid body control mode, the steps of calculating and controlling the displacement of the second joint based on the target pose of the endoscope include: The displacement of the second joint is calculated based on the kinematic inverse matrix and the target pose of the endoscope.

14. The endoscopic field of view adjustment method according to claim 11, characterized in that, When switching to the virtual rigid body control mode, the endoscopic field of view adjustment method further includes the following steps: After the third preset time point, at least a portion of the motion degrees of freedom in the first joint are locked, so that the deviation between the grips of the two control hands and the rod-shaped body is within the third preset range. The third preset time point is when the virtual rigid body control mode is switched to, when a control command representing the locking intention is received, or when a control command reflecting that the user has begun to adjust the field of view is received.

15. A control system, characterized in that, It includes two control hands, a processor and an endoscope, wherein the processor is used to execute the endoscope field of view adjustment method as described in any one of claims 1 to 14 to control the endoscope based on input instructions acquired by the control hands, and the degrees of freedom of the endoscope are adaptively set.

16. A readable storage medium, characterized in that, The readable storage medium stores a program that, when executed, performs the endoscopic field of view adjustment method as described in any one of claims 1 to 14.

Images