A control method, device, equipment and medium for a flexible endoscope control system

Abstract

The present application provides a control method, device, equipment and medium for a flexible endoscope control system, wherein the flexible endoscope control system includes a first mechanical arm, a second mechanical arm and a binocular camera, and the method includes: obtaining image information collected by the binocular camera; determining multiple plane deviation distances and multiple position relationship information between multiple feature points in the flexible endoscope and a target plane according to the image information; determining the torsion state of the flexible endoscope according to the multiple plane deviation distances and multiple position relationship information; determining the movement control mode of the flexible endoscope control system for the flexible endoscope according to the torsion state of the flexible endoscope. In this method, during the movement of the flexible endoscope, it is possible to accurately determine whether the flexible endoscope is twisted and handle it in time to avoid damage to the flexible endoscope and ensure the safety of the movement process of the flexible endoscope.

Description

Technical Field

[0001] The present application relates to the field of medical device technology, and in particular to a control method, device, electronic equipment and computer storage medium for a flexible endoscope control system. Background Art

[0002] With the development of robot-assisted technology, flexible endoscope control systems (also called flexible endoscope robots) have emerged. Medical staff can use flexible endoscope control systems to control the flexible endoscope to move to the required position.

[0003] Normally, the flexible endoscope control system can detect the moving length of the flexible endoscope through the built-in encoder. However, the above method can only detect the moving length and it is difficult to obtain the moving state of the flexible endoscope. If the flexible endoscope is bent, piled up, or piled up during movement, it is difficult to detect it in time. Summary of the invention

[0004] The present application provides a control method for a flexible endoscope control system, which can accurately determine whether the flexible endoscope is twisted and handle it in time, avoid damage to the flexible endoscope, and ensure the safety of the flexible endoscope movement process. The present application also provides a device, electronic equipment and computer storage medium corresponding to the above method.

[0005] In a first aspect, the present application provides a control method for a flexible endoscope manipulation system, wherein the flexible endoscope manipulation system comprises a first robotic arm, a second robotic arm, and a binocular camera, and the method comprises:

[0006] Acquire image information collected by the binocular camera; wherein the image information includes an image of the flexible endoscope between the conveying device deployed at the end of the first robotic arm and the operating device deployed at the end of the second robotic arm;

[0007] According to the image information, a plurality of plane deviation distances and a plurality of position relationship information between a plurality of feature points in the flexible endoscope and a target plane are determined; wherein the target plane is a plane passing through the end point of the first robotic arm and the end point of the second robotic arm and perpendicular to a horizontal plane;

[0008] determining a torsion state of the flexible endoscope according to the plurality of plane deviation distances and the plurality of position relationship information;

[0009] According to the twisting state of the flexible endoscope, the movement control mode of the flexible endoscope control system for the flexible endoscope is determined.

[0010] In some possible implementations, determining, based on the image information, a plurality of plane deviation distances and a plurality of position relationship information between a plurality of feature points in the flexible endoscope and a target plane includes:

[0011] Establishing a target plane according to the coordinate values ​​of the end point of the first robotic arm and the coordinate values ​​of the end point of the second robotic arm;

[0012] Determining coordinate values ​​of a plurality of feature points in the flexible endoscope according to the image information;

[0013] According to the coordinate values ​​of the plurality of feature points in the flexible endoscope, a plurality of plane deviation distances and a plurality of position relationship information between the plurality of feature points and the target plane are determined.

[0014] In some possible implementations, determining the torsion state of the flexible endoscope according to the multiple plane deviation distances and the multiple position relationship information includes:

[0015] For a first feature point among the plurality of feature points whose plane deviation distance is greater than a set distance threshold, determining that a position corresponding to the first feature point in the flexible endoscope is twisted;

[0016] Determine, from the first feature points, a second feature point closest to a conveying device deployed at the end of the first robotic arm;

[0017] When the positional relationship information between the second feature point and the target plane indicates that the second feature point is located in the first area, determining that the flexible endoscope is twisted by the torque in the moving direction of the flexible endoscope;

[0018] When the positional relationship information between the second feature point and the target plane represents that the second feature point is located in the second area, it is determined that the flexible endoscope is twisted by a torque in the opposite direction of the moving direction of the flexible endoscope; wherein the first area is an area including the first robotic arm or the second robotic arm after being divided by the target plane, and the second area is an area excluding the first robotic arm and the second robotic arm after being divided by the target plane.

[0019] In some possible implementations, determining the movement control mode of the flexible endoscope by the flexible endoscope manipulation system according to the torsion state of the flexible endoscope includes:

[0020] When the flexible endoscope is twisted by the torque in the moving direction of the flexible endoscope, the operating device deployed at the end of the second robotic arm is controlled to rotate in the opposite direction of the moving direction of the flexible endoscope;

[0021] When the flexible endoscope is twisted by a torque in a direction opposite to the moving direction of the flexible endoscope, the operating device deployed at the end of the second robotic arm is controlled to rotate in the moving direction of the flexible endoscope.

[0022] In some possible implementations, the method further includes:

[0023] Determining a slope change of the flexible endoscope according to the image information;

[0024] determining a deformation state of the flexible endoscope according to a change in the slope of the flexible endoscope;

[0025] According to the deformation state of the flexible endoscope, the movement control mode of the flexible endoscope control system for the flexible endoscope is determined.

[0026] In some possible implementations, determining the slope change of the flexible endoscope according to the image information includes:

[0027] Determining coordinate values ​​of a plurality of feature points in the flexible endoscope according to the image information;

[0028] Determining the slope between adjacent feature points in the plurality of feature points according to the coordinate values ​​of the plurality of feature points in the flexible endoscope;

[0029] The slope change of the flexible endoscope is determined according to the slopes between adjacent feature points in the plurality of feature points.

[0030] In some possible implementations, determining the deformation state of the flexible endoscope according to the change in the slope of the flexible endoscope includes:

[0031] When the slopes between the adjacent characteristic points in the flexible endoscope are all greater than 0, and the change rates of the slopes between the adjacent characteristic points are all less than a first set change threshold, it is determined that the flexible endoscope is pulled;

[0032] When the slopes between adjacent feature points in the flexible endoscope are both greater than 0 and less than 0, and the rate of change of the slopes between adjacent feature points is greater than a second set change threshold, it is determined that the flexible endoscope is squeezed.

[0033] In some possible implementations, determining the movement control mode of the flexible endoscope by the flexible endoscope manipulation system according to the deformation state of the flexible endoscope includes:

[0034] When the flexible endoscope is pulled, controlling the operating device deployed at the end of the second robotic arm to move along the planned motion trajectory in the moving direction of the flexible endoscope;

[0035] When the flexible endoscope is squeezed, the operating device deployed at the end of the second robotic arm is controlled to move in the opposite direction of the moving direction of the flexible endoscope along the planned motion trajectory.

[0036] In a second aspect, the present application provides a control device for a flexible endoscope control system, wherein the flexible endoscope control system comprises a first robotic arm, a second robotic arm and a binocular camera, and the device comprises:

[0037] An acquisition module, used to acquire image information collected by the binocular camera; wherein the image information includes an image of the flexible endoscope between the conveying device deployed at the end of the first robotic arm and the operating device deployed at the end of the second robotic arm;

[0038] A first determination module is used to determine, based on the image information, a plurality of plane deviation distances and a plurality of position relationship information between a plurality of feature points in the flexible endoscope and a target plane; wherein the target plane is a plane passing through an end point of the first robotic arm and an end point of the second robotic arm and perpendicular to a horizontal plane;

[0039] A second determination module, configured to determine a torsion state of the flexible endoscope according to the plurality of plane deviation distances and the plurality of position relationship information;

[0040] The third determination module is used to determine the movement control mode of the flexible endoscope by the flexible endoscope manipulation system according to the torsion state of the flexible endoscope.

[0041] In some possible implementations, the first determining module is specifically configured to:

[0042] Establishing a target plane according to the coordinate values ​​of the end point of the first robotic arm and the coordinate values ​​of the end point of the second robotic arm;

[0043] Determining coordinate values ​​of a plurality of feature points in the flexible endoscope according to the image information;

[0044] According to the coordinate values ​​of the plurality of feature points in the flexible endoscope, a plurality of plane deviation distances and a plurality of position relationship information between the plurality of feature points and the target plane are determined.

[0045] In some possible implementations, the second determining module is specifically configured to:

[0046] For a first feature point among the plurality of feature points whose plane deviation distance is greater than a set distance threshold, determining that a position corresponding to the first feature point in the flexible endoscope is twisted;

[0047] Determine, from the first feature points, a second feature point closest to a conveying device deployed at the end of the first robotic arm;

[0048] When the positional relationship information between the second feature point and the target plane indicates that the second feature point is located in the first area, determining that the flexible endoscope is twisted by the torque in the moving direction of the flexible endoscope;

[0049] When the positional relationship information between the second feature point and the target plane represents that the second feature point is located in the second area, it is determined that the flexible endoscope is twisted by a torque in the opposite direction of the moving direction of the flexible endoscope; wherein the first area is an area including the first robotic arm or the second robotic arm after being divided by the target plane, and the second area is an area excluding the first robotic arm and the second robotic arm after being divided by the target plane.

[0050] In some possible implementations, the third determining module is specifically configured to:

[0051] When the flexible endoscope is twisted by the torque in the moving direction of the flexible endoscope, the operating device deployed at the end of the second robotic arm is controlled to rotate in the opposite direction of the moving direction of the flexible endoscope;

[0052] When the flexible endoscope is twisted by a torque in a direction opposite to the moving direction of the flexible endoscope, the operating device deployed at the end of the second robotic arm is controlled to rotate in the moving direction of the flexible endoscope.

[0053] In some possible implementations, the apparatus further includes a fourth determining module, where the fourth determining module is configured to:

[0054] Determining a slope change of the flexible endoscope according to the image information;

[0055] determining a deformation state of the flexible endoscope according to a change in the slope of the flexible endoscope;

[0056] According to the deformation state of the flexible endoscope, the movement control mode of the flexible endoscope control system for the flexible endoscope is determined.

[0057] In some possible implementations, the fourth determining module is specifically configured to:

[0058] Determining coordinate values ​​of a plurality of feature points in the flexible endoscope according to the image information;

[0059] Determining the slope between adjacent feature points in the plurality of feature points according to the coordinate values ​​of the plurality of feature points in the flexible endoscope;

[0060] The slope change of the flexible endoscope is determined according to the slopes between adjacent feature points in the plurality of feature points.

[0061] In some possible implementations, the fourth determining module is specifically configured to:

[0062] When the slopes between the adjacent characteristic points in the flexible endoscope are all greater than 0, and the change rates of the slopes between the adjacent characteristic points are all less than a first set change threshold, it is determined that the flexible endoscope is pulled;

[0063] When the slopes between adjacent feature points in the flexible endoscope are both greater than 0 and less than 0, and the rate of change of the slopes between adjacent feature points is greater than a second set change threshold, it is determined that the flexible endoscope is squeezed.

[0064] In some possible implementations, the fourth determining module is specifically configured to:

[0065] When the flexible endoscope is pulled, controlling the operating device deployed at the end of the second robotic arm to move along the planned motion trajectory in the moving direction of the flexible endoscope;

[0066] When the flexible endoscope is squeezed, the operating device deployed at the end of the second robotic arm is controlled to move in the opposite direction of the moving direction of the flexible endoscope along the planned motion trajectory.

[0067] In a third aspect, the present application provides an electronic device, wherein the electronic device comprises a processor and a memory, wherein the memory stores instructions, and the processor executes the instructions so that the electronic device executes the method as described in the first aspect of the present application or any implementation of the first aspect.

[0068] In a fourth aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium includes instructions, and the instructions instruct an electronic device to execute the method as described in the first aspect of the present application or any implementation of the first aspect.

[0069] Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations.

[0070] Based on the above description, it can be seen that the technical solution of this application has the following beneficial effects:

[0071] Specifically, the flexible endoscope control system includes a first robotic arm, a second robotic arm and a binocular camera. First, image information collected by the binocular camera is obtained, wherein the image information includes an image of the flexible endoscope between a conveying device deployed at the end of the first robotic arm and an operating device deployed at the end of the second robotic arm. Based on the image information, multiple plane deviation distances and multiple position relationship information between multiple feature points in the flexible endoscope and a target plane are determined; wherein the target plane is a plane passing through the end point of the first robotic arm and the end point of the second robotic arm and perpendicular to the horizontal plane. Next, based on the multiple plane deviation distances and the multiple position relationship information, the torsion state of the flexible endoscope is determined, and based on the torsion state of the flexible endoscope, the movement control method of the flexible endoscope control system for the flexible endoscope is determined.

[0072] In this method, for the scenario where the flexible endoscope control system controls the flexible endoscope to move, a binocular camera is deployed in the flexible endoscope control system, and the binocular camera is used to collect image information during the movement of the flexible endoscope. The deviation of the flexible endoscope from the target plane is determined based on the image information, and then the torsion state of the flexible endoscope is determined. The movement control method of the flexible endoscope is determined in combination with the torsion state. In this way, during the movement of the flexible endoscope, it is possible to accurately determine whether the flexible endoscope is twisted and handle it in time to avoid damage to the flexible endoscope and ensure the safety of the movement of the flexible endoscope. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] The above and other features, advantages and aspects of the embodiments of the present application will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0074] Figure 1 A schematic diagram of the structure of a flexible endoscope control system provided in an embodiment of the present application;

[0075] Figure 2 A schematic diagram of a binocular camera deployment method provided in an embodiment of the present application;

[0076] Figure 3 A schematic flow chart of a control method for a flexible endoscope control system provided in an embodiment of the present application;

[0077] Figure 4A and Figure 4B A schematic diagram of image information provided in an embodiment of the present application;

[0078] Figure 5 A schematic diagram of a flexible endoscope provided in an embodiment of the present application;

[0079] Figure 6A schematic diagram of a flexible endoscope torsion provided in an embodiment of the present application;

[0080] Fig. 7A and Figure 7B A schematic diagram of deformation of a flexible endoscope provided in an embodiment of the present application;

[0081] Figure 8 A schematic diagram of the structure of a control device of a flexible endoscope control system provided in an embodiment of the present application;

[0082] Fig. 9 A schematic diagram of the structure of an electronic device for controlling a flexible endoscope control system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0083] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0084] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0085] It should be noted that the concepts such as "first" and "second" mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0086] It should be noted that the modifications of "one" and "plurality" mentioned in the present application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0087] In order to facilitate understanding of the technical solution of the present application, the specific application scenarios in the present application are described below.

[0088] Natural cavities such as the digestive tract and respiratory tract are common sites for the occurrence of human diseases. Since the lesions are inside the natural cavities of the human body, they need to be examined or treated surgically using a flexible endoscope.

[0089] In the early stage of traditional flexible endoscopy examination or treatment, medical staff need to hold the endoscope and manually move the flexible endoscope to the desired position by operating the knob. With the development of robot-assisted technology, flexible endoscope control systems (also called flexible endoscope robots) have emerged. With the help of the flexible endoscope control system, medical staff can adjust the moving length and moving posture of the flexible endoscope by adjusting the handle switch and button, and control the flexible endoscope to move to the desired position.

[0090] Normally, the flexible endoscope control system can detect the moving length of the flexible endoscope through the built-in encoder. However, the above method can only detect the moving length and it is difficult to obtain the moving state of the flexible endoscope. If the flexible endoscope is bent, piled up, or piled up during movement, it is difficult to detect it in time.

[0091] Based on this, an embodiment of the present application provides a control method for a flexible endoscope control system. Specifically, the flexible endoscope control system includes a first robotic arm, a second robotic arm and a binocular camera. First, image information collected by the binocular camera is obtained, wherein the image information includes an image of the flexible endoscope between a conveying device deployed at the end of the first robotic arm and an operating device deployed at the end of the second robotic arm. According to the image information, multiple plane deviation distances and multiple position relationship information between multiple feature points in the flexible endoscope and a target plane are determined; wherein the target plane is a plane passing through the end point of the first robotic arm and the end point of the second robotic arm and perpendicular to the horizontal plane. Then, according to the multiple plane deviation distances and multiple position relationship information, the torsion state of the flexible endoscope is determined, and according to the torsion state of the flexible endoscope, the mobile control method of the flexible endoscope control system for the flexible endoscope is determined.

[0092] In this method, for the scenario where the flexible endoscope control system controls the flexible endoscope to move, a binocular camera is deployed in the flexible endoscope control system, and the binocular camera is used to collect image information during the movement of the flexible endoscope. The deviation of the flexible endoscope from the target plane is determined based on the image information, and then the torsion state of the flexible endoscope is determined. The movement control method of the flexible endoscope is determined in combination with the torsion state. In this way, during the movement of the flexible endoscope, it is possible to accurately determine whether the flexible endoscope is twisted and handle it in time to avoid damage to the flexible endoscope and ensure the safety of the movement of the flexible endoscope.

[0093] Next, the flexible endoscope control system in the embodiment of the present application will be described.

[0094] See also Figure 1 The schematic diagram of the structure of a flexible endoscope control system is shown, which includes a trolley 100, a control console 200, a workstation 300 and an operating table 400. The control console 200 is connected to the trolley 100 through the workstation 300, and the operating table 400 is close to the trolley 100.

[0095] The trolley 100 is equipped with a first robotic arm 101 and a second robotic arm 102, both of which are multi-degree-of-freedom robotic arms. A conveying device 103 is arranged at the end of the first robotic arm, and the conveying device 103 is used to convey the head end of the flexible endoscope. An operating device 104 is arranged at the end of the second robotic arm 102, and the operating device 104 is used to control the operating part of the flexible endoscope. The first robotic arm 101 and the second robotic arm 102 can perform the action of moving the flexible endoscope.

[0096] In the embodiment of the present application, the flexible endoscope control system is also equipped with a binocular camera. A binocular camera, also known as a stereo camera, is a device that simulates the depth perception ability of human eyes. It consists of two cameras arranged side by side and can capture two slightly different perspective images of the same scene. Figure 2 The schematic diagram of a binocular camera deployment method is shown, where the binocular camera 500 can be placed at a distance d in front of the trolley 100, so that the binocular camera 500 can capture images of the first robotic arm 101 and the second robotic arm 102. For example, the binocular camera 500 can be placed at a distance 1500 mm in front of the trolley 100, and the binocular camera 500 can observe the movement process of the flexible endoscope at this position.

[0097] The control method of the flexible endoscope control system provided in the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0098] Since a binocular camera is deployed in the flexible endoscope control system in the embodiment of the present application, it is necessary to perform hand-eye calibration on the binocular camera before actually controlling the flexible endoscope control system. Among them, hand-eye calibration can be understood as a process for determining the relationship between the binocular camera coordinate system and the world coordinate system of the flexible endoscope control system. In other words, through hand-eye calibration, the conversion relationship between the binocular camera coordinate system and the world coordinate system of the flexible endoscope control system can be obtained. In this way, for the image information collected during the actual operation of the binocular camera, the coordinate values ​​of the points in the image information in the world coordinate system of the flexible endoscope control system can be obtained.

[0099] Specifically, the binocular camera is usually equipped with a target ball, and the binocular camera can identify the position of the target ball. In other words, the binocular camera coordinate system and the target ball coordinate system can be converted. In an embodiment of the present application, the target ball is fixed at the end of the conveying device deployed at the end of the first robotic arm.

[0100] The world coordinate system of the flexible endoscope control system is denoted as O W , the binocular camera coordinate system is recorded as O C , the coordinate system of the first robot base is denoted as O B , the coordinate system of the end point of the first robotic arm is recorded as O T, the target sphere coordinate system is denoted as O Ta For the target point in a certain posture state of the first robot arm, the binocular camera knows that the target point is in the target sphere coordinate system O Ta The coordinate value under the target point is in the world coordinate system O W The coordinate value is recorded as P 1 , the target point is in the binocular camera coordinate system O C The coordinate value is recorded as P 2 , P 2 With P 1 The conversion relationship between them satisfies the following formula:

[0101]

[0102] in, O W With O C The transformation matrix between O W With O Ta The transformation matrix between O W With O B The transformation matrix between O B With O T The transformation matrix between O T With O Ta The transformation matrix between .

[0103] While ensuring that the target point remains unchanged (i.e. P 1 and P 2 Under the premise that the position and posture of the first robot arm remain unchanged, the position and posture of the first robot arm are relocated. B With O T The transformation matrix between O T With O Ta The transformation matrix between will also change. At this time, P 2 With P 1 The conversion relationship between them satisfies the following formula:

[0104]

[0105] in, is the position state after the change O W With O Ta The transformation matrix between is the position state after the change O B With O T The transformation matrix between is the position state after the change O T With OTa The transformation matrix between .

[0106] Combining the above two equations, we have .in, and It can be obtained by collecting the DH parameters of the first robot arm and using the principle of forward kinematics. and It can be obtained through the external parameters of the binocular camera calibration and the deployment position of the binocular camera. Therefore, after simplification, we have: ,Will Let it be A, Let it be B, , get O C With O W The conversion matrix between them is used to complete the hand-eye calibration of the binocular camera.

[0107] See also Figure 3 The flowchart of a control method of a flexible endoscope control system shown in FIG. 1 specifically includes the following steps:

[0108] S301: Acquire image information collected by a binocular camera.

[0109] The embodiments of the present application are aimed at scenarios in which a flexible endoscope manipulation system controls the movement of a flexible endoscope. For example, in the early stages of flexible endoscope inspection or treatment, the flexible endoscope needs to be fixed on a conveying device deployed at the end of a first robotic arm and an operating device deployed at the end of a second robotic arm. The conveying device can control the movement of the flexible endoscope, and the operating device can control the bending of the flexible endoscope, thereby controlling the flexible endoscope to move to an initial position for subsequent flexible endoscope inspection or treatment.

[0110] During the movement of the flexible endoscope, the deployed binocular camera is used to collect image information. The image information includes the image of the flexible endoscope between the conveying device deployed at the end of the first robotic arm and the operating device deployed at the end of the second robotic arm. In other words, during the process of the first robotic arm and the second robotic arm clamping the flexible endoscope and controlling the flexible endoscope to move, the binocular camera can collect the image of the flexible endoscope.

[0111] See also Figure 4A and Figure 4B A schematic diagram of image information is shown in FIG. Figure 4A is the left view collected by the binocular camera, Figure 4B is the right view captured by the binocular camera. Figure 4A The image 601 of the flexible endoscope between the conveying device deployed at the end of the first robotic arm and the operating device deployed at the end of the second robotic arm, Figure 4BThe image 602 includes a flexible endoscope between a delivery device deployed at the end of the first robotic arm and an operating device deployed at the end of the second robotic arm.

[0112] S302: Determine, based on the image information, a plurality of plane deviation distances and a plurality of position relationship information between a plurality of feature points in the flexible endoscope and a target plane.

[0113] After acquiring the image information, the image information can be corrected first to unify the height and depth of the two images captured by the binocular camera. Then, stereo matching calculation is performed on the image information to obtain a disparity map and a depth map. The embodiment of the present application does not limit the process of fitting to obtain the disparity map and the depth map, and any calculation method can be used to obtain the disparity map and the depth map.

[0114] In the embodiment of the present application, the target plane is a plane passing through the end point of the first robot arm and the end point of the second robot arm and perpendicular to the horizontal plane. In other words, the target plane is a vertical plane parallel to the Z axis in the world coordinate system of the flexible endoscope manipulation system.

[0115] Among them, the end point of the first robotic arm can be understood as the end point of the first robotic arm when the conveying device is not deployed, and the end point of the second robotic arm can be understood as the end point of the second robotic arm when the operating device is not deployed.

[0116] The feature points may be multiple points at different locations in the flexible endoscope. Figure 5 As shown, the soft endoscope is a black silicone rubber hose, and white stripes are arranged at regular intervals (eg, 50 mm). In this case, the feature points may be points corresponding to the white stripes.

[0117] In a specific implementation, a target plane is established according to the coordinate values ​​of the end point of the first robotic arm and the coordinate values ​​of the end point of the second robotic arm. Then, the coordinate values ​​of multiple feature points in the flexible endoscope are determined according to the image information. Based on the coordinate values ​​of the multiple feature points in the flexible endoscope, multiple plane deviation distances and multiple position relationship information between the multiple feature points and the target plane are determined.

[0118] The plane deviation distance can be understood as the distance between the feature point and the target plane, and the position relationship information can be understood as the information about the positions of the feature point and the target plane.

[0119] In other words, the target plane can be understood as a reference plane determined based on the end point of the first robotic arm and the end point of the second robotic arm, and whether the flexible endoscope deviates from the target plane is determined by the coordinate values ​​of multiple feature points in the flexible endoscope.

[0120] In conjunction with a specific example, the coordinate value of the end point of the first robotic arm in the world coordinate system of the flexible endoscope control system is recorded as P 0 (X0 ,Y 0 ,Z 0 ), the coordinate value of the end point of the second robotic arm in the world coordinate system of the flexible endoscope control system is recorded as P N (X N ,Y N ,Z N ).

[0121] The target plane is recorded as ax+by+cz+d=0. Since the target plane is parallel to the Z axis in the world coordinate system of the flexible endoscope control system, c=0. The expression of the target plane obtained by point-slope calculation is as follows:

[0122]

[0123] After the target plane is established, the coordinate values ​​of multiple feature points in the flexible endoscope are determined. For each feature point, the plane deviation distance is determined using the following formula:

[0124]

[0125] Where D is the plane deviation distance, a is , b is , and is the coordinate value of the feature point.

[0126] For each feature point, the x-axis coordinate value Substituting into the expression of the target plane, if , indicating that the feature point is located outside the target plane (i.e., away from the side of the flexible endoscope control system). , indicating that the feature point is located on the inner side of the target plane (i.e., the side close to the flexible endoscope control system). , indicating that the feature point is located on the target plane.

[0127] S303: Determine the torsion state of the flexible endoscope according to the multiple plane deviation distances and the multiple position relationship information.

[0128] Since the plane deviation distance and position relationship information can characterize the situation where the feature point deviates from the target plane, the torsion state of the flexible endoscope can be determined according to multiple plane deviation distances and multiple position relationship information corresponding to multiple feature points during the movement process of the flexible endoscope.

[0129] In a specific implementation, for a first feature point among multiple feature points whose plane deviation distance is greater than a set distance threshold, it is determined that there is torsion at the position corresponding to the first feature point in the flexible endoscope, and then, from the first feature point, a second feature point closest to a conveying device deployed at the end of the first robotic arm is determined, and when the positional relationship information between the second feature point and the target plane indicates that the second feature point is located in the first area, it is determined that the flexible endoscope is twisted by a torque in the moving direction of the flexible endoscope, and when the positional relationship information between the second feature point and the target plane indicates that the second feature point is located in the second area, it is determined that the flexible endoscope is twisted by a torque in the opposite direction of the moving direction of the flexible endoscope.

[0130] The first area is an area including the first robotic arm or the second robotic arm after being divided by the target plane, and the second area is an area excluding the first robotic arm and the second robotic arm after being divided by the target plane.

[0131] That is to say, when the plane deviation distance of the feature point is large, it indicates that there is a torsion phenomenon at the feature point of the flexible endoscope. Since the conveying device deployed at the end of the first robot arm is used to control the movement of the flexible endoscope, the second feature point closest to the conveying device is determined from the first feature point with the torsion phenomenon, and the torsion phenomenon specifically existing at the second feature point is determined.

[0132] If the second characteristic point is located on the side close to the flexible endoscope control system, it is determined that the flexible endoscope is torsion-caused by the torque in the moving direction. If the second characteristic point is located on the side of the flexible endoscope control system, it is determined that the flexible endoscope is torsion-caused by the torque in the opposite direction of the moving direction. In this way, during the movement of the flexible endoscope, the image information collected by the binocular camera is used to determine whether the flexible endoscope is torsion-caused.

[0133] For specific examples, see Figure 6 , the flexible endoscope includes feature point A and feature point B. During the movement of the flexible endoscope, the plane deviation distances of point A and point B are both greater than the set distance threshold, that is, point A and point B are both twisted, and point A and point B are both the first feature points. Assume Figure 6 The first area is above the target plane, the second area is below the target plane, point A is located in the first area, and point B is located in the second area. If point A is the second characteristic point, the flexible endoscope is twisted by the torque in the moving direction of the flexible endoscope, and if point B is the second characteristic point, the flexible endoscope is twisted by the torque in the opposite direction of the moving direction of the flexible endoscope.

[0134] S304: Determine a movement control method of the flexible endoscope by the flexible endoscope control system according to the torsion state of the flexible endoscope.

[0135] After determining the twisting state of the flexible endoscope, in order to ensure that the flexible endoscope moves smoothly and smoothly, the flexible endoscope control system can control the movement mode of the flexible endoscope to prevent the flexible endoscope from continuing to move in the twisting state.

[0136] In a specific implementation, when the flexible endoscope is twisted by the torque in the moving direction of the flexible endoscope, the operating device deployed at the end of the second robotic arm is controlled to rotate in the opposite direction of the moving direction of the flexible endoscope. When the flexible endoscope is twisted by the torque in the opposite direction of the moving direction of the flexible endoscope, the operating device deployed at the end of the second robotic arm is controlled to rotate in the moving direction of the flexible endoscope.

[0137] That is to say, when the flexible endoscope is twisted by torques in different directions, the operating device deployed at the end of the second robotic arm is controlled to rotate in different directions, thereby eliminating the twisting phenomenon and making the movement process of the flexible endoscope smooth.

[0138] In this method, for the scenario where the flexible endoscope control system controls the flexible endoscope to move, a binocular camera is deployed in the flexible endoscope control system, and the binocular camera is used to collect image information during the movement of the flexible endoscope. The deviation of the flexible endoscope from the target plane is determined based on the image information, and then the torsion state of the flexible endoscope is determined. The movement control method of the flexible endoscope is determined in combination with the torsion state. In this way, during the movement of the flexible endoscope, it is possible to accurately determine whether the flexible endoscope is twisted and handle it in time to avoid damage to the flexible endoscope and ensure the safety of the movement of the flexible endoscope.

[0139] In some embodiments, it is also possible to determine whether the flexible endoscope is deformed during movement based on the image information. Specifically, based on the image information, the change in the slope of the flexible endoscope is determined, and then based on the change in the slope of the flexible endoscope, the deformation state of the flexible endoscope is determined, and based on the deformation state of the flexible endoscope, the movement control method of the flexible endoscope control system for the flexible endoscope is determined.

[0140] The slope change can be understood as the change in the bending degree of multiple consecutive sections of the flexible endoscope, and the deformation state can be understood as the deformation of the flexible endoscope during movement.

[0141] In other words, the image information collected by the binocular camera is used to judge the bending of the flexible endoscope during movement, and then determine the deformation state of the flexible endoscope, so as to control the flexible endoscope and prevent it from continuing to move when deformation occurs.

[0142] In a specific implementation, the coordinate values ​​of multiple feature points in the flexible endoscope are determined based on the image information. Then, the slopes between adjacent feature points in the multiple feature points are determined based on the coordinate values ​​of the multiple feature points in the flexible endoscope. The slope change of the flexible endoscope is determined based on the slopes between adjacent feature points in the multiple feature points.

[0143] That is to say, by determining the coordinate values ​​of a plurality of feature points, the slopes of two adjacent feature points can be calculated respectively, and then the slope change can be determined according to the slopes of the plurality of adjacent feature points.

[0144] Combined with a specific example, the coordinate values ​​of any two adjacent feature points in the flexible endoscope in the world coordinate system of the flexible endoscope control system are recorded as P 1 (X 1 ,Y 1 ,Z 1 ) and P 2 (X 2 ,Y 2 ,Z 2 ), P 1 and P 2 Projected onto the target plane, the coordinate value of the projected feature point in the world coordinate system of the flexible endoscope control system is P' 1 (X 1 ,aX 1 +b,Z 1 ) and P' 2 (X 2 ,aX 2 +b,Z 2 ), therefore, P 1 and P 2 The slope between them is k=(Z 2 -Z 1 ) / (X 2 -X 1 ).

[0145] In this way, for multiple consecutive adjacent feature points in the flexible endoscope, the rate of change of the slope (i.e., the slope change) △=k is calculated n -k n-1 Among them, k n It refers to the slope between the nth adjacent feature points in the flexible endoscope, k n-1 It refers to the slope between the n-1th adjacent feature points in the flexible endoscope. The nth adjacent feature point and the n-1th adjacent feature point are adjacent.

[0146] Furthermore, when the slopes between each adjacent feature point in the flexible endoscope are greater than 0, and the rate of change of the slopes between each adjacent feature point is less than a first set change threshold, it is determined that the flexible endoscope is pulled; when the slopes between each adjacent feature point in the flexible endoscope are both greater than 0 and less than 0 at the same time, and the rate of change of the slopes between each adjacent feature point is greater than a second set change threshold, it is determined that the flexible endoscope is squeezed.

[0147] That is to say, if the change in slope indicates that the direction of deformation of the flexible endoscope is uniform and the flexible endoscope has not deformed to a large extent, it is judged that the flexible endoscope is pulled and presents a state close to a straight line. If the change in slope indicates that the direction of deformation of the flexible endoscope is not uniform and the flexible endoscope has deformed to a large extent, it is judged that the flexible endoscope is squeezed and presents a bent state.

[0148] like Fig. 7A and Figure 7B As shown, in Fig. 7A In , the slopes between adjacent feature points are greater than 0, and the rate of change of the slopes between adjacent feature points is small. Therefore, Fig. 7A The flexible endoscope in the Figure 7B In the case where the slopes between adjacent feature points are both greater than 0 and less than 0, and the rate of change of the slopes between adjacent feature points is large, therefore, Figure 7B The flexible endoscope in the patient is squeezed.

[0149] According to different deformation states of the flexible endoscope, the flexible endoscope control system can perform different movement control on the flexible endoscope. Specifically, when the flexible endoscope is pulled, the operating device deployed at the end of the second robotic arm is controlled to move in the direction of movement of the flexible endoscope on the planned movement trajectory; when the flexible endoscope is squeezed, the operating device deployed at the end of the second robotic arm is controlled to move in the opposite direction of the movement direction of the flexible endoscope on the planned movement trajectory.

[0150] That is to say, the flexible endoscope control system can plan the movement path in advance and determine the planned movement trajectory of the flexible endoscope. During the movement of the flexible endoscope, if the flexible endoscope is pulled, the control operation device moves along the movement direction on the planned movement trajectory, so as to eliminate the situation where the flexible endoscope is pulled. If the flexible endoscope is squeezed, the control operation device moves in the opposite direction of the movement direction on the planned movement trajectory, so as to eliminate the situation where the flexible endoscope is squeezed, so that the flexible endoscope remains in a flexible state during the movement process, and the flexible endoscope is prevented from being damaged.

[0151] In this method, for the scenario where the flexible endoscope control system controls the flexible endoscope to move, a binocular camera is deployed in the flexible endoscope control system, and the binocular camera is used to collect image information during the movement of the flexible endoscope. The slope change of the flexible endoscope is determined based on the image information, and then the deformation state of the flexible endoscope is determined. The movement control method of the flexible endoscope is determined in combination with the deformation state. In this way, during the movement of the flexible endoscope, it is possible to accurately determine whether the flexible endoscope is deformed and handle it in time to avoid damage to the flexible endoscope and ensure the safety of the movement process of the flexible endoscope.

[0152] Based on the above method provided in the embodiment of the present application, the embodiment of the present application also provides a control device for a flexible endoscope control system corresponding to the above method. The units / modules involved in the embodiments of the present application can be implemented by software or by hardware. In some cases, the name of the unit / module does not constitute a limitation on the unit / module itself.

[0153] See also Figure 8 The structure diagram of the control device of the flexible endoscope control system shown in FIG. 800 includes:

[0154] An acquisition module 801 is used to acquire image information collected by the binocular camera; wherein the image information includes an image of the flexible endoscope between the conveying device deployed at the end of the first robotic arm and the operating device deployed at the end of the second robotic arm;

[0155] A first determination module 802 is used to determine, based on the image information, a plurality of plane deviation distances and a plurality of position relationship information between a plurality of feature points in the flexible endoscope and a target plane; wherein the target plane is a plane passing through the end point of the first robotic arm and the end point of the second robotic arm and perpendicular to a horizontal plane;

[0156] A second determination module 803 is used to determine the torsion state of the flexible endoscope according to the multiple plane deviation distances and the multiple position relationship information;

[0157] The third determination module 804 is used to determine the movement control mode of the flexible endoscope control system for the flexible endoscope according to the twisting state of the flexible endoscope.

[0158] In some possible implementations, the first determining module is specifically configured to:

[0159] Establishing a target plane according to the coordinate values ​​of the end point of the first robotic arm and the coordinate values ​​of the end point of the second robotic arm;

[0160] Determining coordinate values ​​of a plurality of feature points in the flexible endoscope according to the image information;

[0161] According to the coordinate values ​​of the plurality of feature points in the flexible endoscope, a plurality of plane deviation distances and a plurality of position relationship information between the plurality of feature points and the target plane are determined.

[0162] In some possible implementations, the second determining module is specifically configured to:

[0163] For a first feature point among the plurality of feature points whose plane deviation distance is greater than a set distance threshold, determining that a position corresponding to the first feature point in the flexible endoscope is twisted;

[0164] Determine, from the first feature points, a second feature point closest to a conveying device deployed at the end of the first robotic arm;

[0165] When the positional relationship information between the second feature point and the target plane indicates that the second feature point is located in the first area, determining that the flexible endoscope is twisted by the torque in the moving direction of the flexible endoscope;

[0166] When the positional relationship information between the second feature point and the target plane represents that the second feature point is located in the second area, it is determined that the flexible endoscope is twisted by a torque in the opposite direction of the moving direction of the flexible endoscope; wherein the first area is an area including the first robotic arm or the second robotic arm after being divided by the target plane, and the second area is an area excluding the first robotic arm and the second robotic arm after being divided by the target plane.

[0167] In some possible implementations, the third determining module is specifically configured to:

[0168] When the flexible endoscope is twisted by the torque in the moving direction of the flexible endoscope, the operating device deployed at the end of the second robotic arm is controlled to rotate in the opposite direction of the moving direction of the flexible endoscope;

[0169] When the flexible endoscope is twisted by a torque in a direction opposite to the moving direction of the flexible endoscope, the operating device deployed at the end of the second robotic arm is controlled to rotate in the moving direction of the flexible endoscope.

[0170] In some possible implementations, the apparatus further includes a fourth determining module, where the fourth determining module is configured to:

[0171] Determining a slope change of the flexible endoscope according to the image information;

[0172] determining a deformation state of the flexible endoscope according to a change in the slope of the flexible endoscope;

[0173] According to the deformation state of the flexible endoscope, the movement control mode of the flexible endoscope control system for the flexible endoscope is determined.

[0174] In some possible implementations, the fourth determining module is specifically configured to:

[0175] Determining coordinate values ​​of a plurality of feature points in the flexible endoscope according to the image information;

[0176] Determining the slope between adjacent feature points in the plurality of feature points according to the coordinate values ​​of the plurality of feature points in the flexible endoscope;

[0177] The slope change of the flexible endoscope is determined according to the slopes between adjacent feature points in the plurality of feature points.

[0178] In some possible implementations, the fourth determining module is specifically configured to:

[0179] When the slopes between the adjacent characteristic points in the flexible endoscope are all greater than 0, and the change rates of the slopes between the adjacent characteristic points are all less than a first set change threshold, it is determined that the flexible endoscope is pulled;

[0180] When the slopes between adjacent feature points in the flexible endoscope are both greater than 0 and less than 0, and the rate of change of the slopes between adjacent feature points is greater than a second set change threshold, it is determined that the flexible endoscope is squeezed.

[0181] In some possible implementations, the fourth determining module is specifically configured to:

[0182] When the flexible endoscope is pulled, controlling the operating device deployed at the end of the second robotic arm to move along the planned motion trajectory in the moving direction of the flexible endoscope;

[0183] When the flexible endoscope is squeezed, the operating device deployed at the end of the second robotic arm is controlled to move in the opposite direction of the moving direction of the flexible endoscope along the planned motion trajectory.

[0184] The control device 800 of the flexible endoscope control system according to the embodiment of the present application may correspond to the method described in the embodiment of the present application, and the above and other operations and / or functions of each module / unit of the control device 800 of the flexible endoscope control system are respectively to achieve Figure 3 For the sake of brevity, the corresponding processes of each method in the illustrated embodiment are not described in detail here.

[0185] The functions described above in this document may be performed at least in part by one or more hardware logic components. Fig. 9The schematic diagram of the structure of the electronic device 900 for realizing the control of the flexible endoscope control system is shown. It should be noted that: Fig. 9 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0186] like Fig. 9 As shown, the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 are also stored. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0187] Typically, the following devices may be connected to the I / O interface 905: an input device 906 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 908 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although Fig. 9 The electronic device 900 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0188] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

[0189] Although several specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present application. Certain features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination.

[0190] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A control method for a flexible endoscope control system, characterized in that: The flexible endoscope manipulation system comprises a first mechanical arm, a second mechanical arm and a binocular camera, and the method comprises: Acquire image information collected by the binocular camera; wherein the image information includes an image of the flexible endoscope between the conveying device deployed at the end of the first robotic arm and the operating device deployed at the end of the second robotic arm; According to the image information, a plurality of plane deviation distances and a plurality of position relationship information between a plurality of feature points in the flexible endoscope and a target plane are determined; wherein the target plane is a plane passing through the end point of the first robotic arm and the end point of the second robotic arm and perpendicular to a horizontal plane; determining a torsion state of the flexible endoscope according to the plurality of plane deviation distances and the plurality of position relationship information; Determining a movement control mode of the flexible endoscope by the flexible endoscope control system according to the twisting state of the flexible endoscope; Wherein, determining the torsion state of the flexible endoscope according to the multiple plane deviation distances and the multiple position relationship information includes: For a first feature point among the plurality of feature points whose plane deviation distance is greater than a set distance threshold, determining that a position corresponding to the first feature point in the flexible endoscope is twisted; Determine, from the first feature points, a second feature point closest to a conveying device deployed at the end of the first robotic arm; When the positional relationship information between the second feature point and the target plane indicates that the second feature point is located in the first area, determining that the flexible endoscope is twisted by the torque in the moving direction of the flexible endoscope; When the positional relationship information between the second feature point and the target plane represents that the second feature point is located in the second area, it is determined that the flexible endoscope is twisted by a torque in the opposite direction of the moving direction of the flexible endoscope; wherein the first area is an area including the first robotic arm or the second robotic arm after being divided by the target plane, and the second area is an area excluding the first robotic arm and the second robotic arm after being divided by the target plane.

2. The method according to claim 1, characterized in that Determining, based on the image information, a plurality of plane deviation distances and a plurality of position relationship information between a plurality of feature points in the flexible endoscope and a target plane comprises: Establishing a target plane according to the coordinate values ​​of the end point of the first robotic arm and the coordinate values ​​of the end point of the second robotic arm; Determining coordinate values ​​of a plurality of feature points in the flexible endoscope according to the image information; According to the coordinate values ​​of the plurality of feature points in the flexible endoscope, a plurality of plane deviation distances and a plurality of position relationship information between the plurality of feature points and the target plane are determined.

3. The method according to claim 1, characterized in that Determining the movement control mode of the flexible endoscope by the flexible endoscope control system according to the twisting state of the flexible endoscope includes: When the flexible endoscope is twisted by the torque in the moving direction of the flexible endoscope, the operating device deployed at the end of the second robotic arm is controlled to rotate in the opposite direction of the moving direction of the flexible endoscope; When the flexible endoscope is twisted by a torque in a direction opposite to the moving direction of the flexible endoscope, the operating device deployed at the end of the second robotic arm is controlled to rotate in the moving direction of the flexible endoscope.

4. The method according to claim 1, characterized in that: The method further comprises: Determining a slope change of the flexible endoscope according to the image information; determining a deformation state of the flexible endoscope according to a change in the slope of the flexible endoscope; According to the deformation state of the flexible endoscope, the movement control mode of the flexible endoscope control system for the flexible endoscope is determined.

5. The method according to claim 4, characterized in that Determining the slope change of the flexible endoscope according to the image information includes: Determining coordinate values ​​of a plurality of feature points in the flexible endoscope according to the image information; Determining the slope between adjacent feature points in the plurality of feature points according to the coordinate values ​​of the plurality of feature points in the flexible endoscope; The slope change of the flexible endoscope is determined according to the slopes between adjacent feature points in the plurality of feature points.

6. The method according to claim 4, characterized in that Determining the deformation state of the flexible endoscope according to the change in the slope of the flexible endoscope includes: When the slopes between the adjacent characteristic points in the flexible endoscope are all greater than 0, and the change rates of the slopes between the adjacent characteristic points are all less than a first set change threshold, it is determined that the flexible endoscope is pulled; When the slopes between adjacent feature points in the flexible endoscope are both greater than 0 and less than 0, and the rate of change of the slopes between adjacent feature points is greater than a second set change threshold, it is determined that the flexible endoscope is squeezed.

7. The method according to claim 4, characterized in that Determining the movement control mode of the flexible endoscope by the flexible endoscope control system according to the deformation state of the flexible endoscope includes: When the flexible endoscope is pulled, controlling the operating device deployed at the end of the second robotic arm to move along the planned motion trajectory in the moving direction of the flexible endoscope; When the flexible endoscope is squeezed, the operating device deployed at the end of the second robotic arm is controlled to move in the opposite direction of the moving direction of the flexible endoscope along the planned motion trajectory.

8. A control device for a flexible endoscope control system, characterized in that: The flexible endoscope manipulation system comprises a first mechanical arm, a second mechanical arm and a binocular camera, and the device comprises: An acquisition module, used to acquire image information collected by the binocular camera; wherein the image information includes an image of the flexible endoscope between the conveying device deployed at the end of the first robotic arm and the operating device deployed at the end of the second robotic arm; A first determination module is used to determine, based on the image information, a plurality of plane deviation distances and a plurality of position relationship information between a plurality of feature points in the flexible endoscope and a target plane; wherein the target plane is a plane passing through an end point of the first robotic arm and an end point of the second robotic arm and perpendicular to a horizontal plane; A second determination module, configured to determine a torsion state of the flexible endoscope according to the plurality of plane deviation distances and the plurality of position relationship information; a third determination module, configured to determine a movement control mode of the flexible endoscope control system for the flexible endoscope according to the twisting state of the flexible endoscope; Wherein, the second determining module is specifically used for: For a first feature point among the plurality of feature points whose plane deviation distance is greater than a set distance threshold, determining that a position corresponding to the first feature point in the flexible endoscope is twisted; Determine, from the first feature points, a second feature point closest to a conveying device deployed at the end of the first robotic arm; When the positional relationship information between the second feature point and the target plane indicates that the second feature point is located in the first area, determining that the flexible endoscope is twisted by the torque in the moving direction of the flexible endoscope; When the positional relationship information between the second feature point and the target plane represents that the second feature point is located in the second area, it is determined that the flexible endoscope is twisted by a torque in the opposite direction of the moving direction of the flexible endoscope; wherein the first area is an area including the first robotic arm or the second robotic arm after being divided by the target plane, and the second area is an area excluding the first robotic arm and the second robotic arm after being divided by the target plane.

9. An electronic device, characterized in that: The electronic device comprises a processor and a memory; the processor is configured to execute instructions stored in the memory, so that the electronic device executes the method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The method comprises instructions, wherein the instructions instruct an electronic device to execute the method as claimed in any one of claims 1 to 7.

Images