Surgical robot doctor console visual field adjusting method, device, equipment, medium and product

Abstract

The embodiment of the invention discloses a visual field adjusting method, device and equipment for a doctor console of a surgical robot and a medium, and the method comprises the steps: obtaining a horizontal movement track point and a vertical movement track point of a main manipulator reference point corresponding to a main manipulator in a visual field; performing straight line fitting on the horizontal movement track point and the vertical movement track point to obtain a first straight line and a second straight line; determining a third straight line and a fourth straight line according to projection coordinates of at least two points on the first straight line and the second straight line in a corresponding coordinate plane of a main end base coordinate system corresponding to the main manipulator; according to the spatial position relation of the third straight line and the fourth straight line in the main end base coordinate system, a main end base coordinate system space correction plane is determined, so that a coordinate correction matrix is determined, and view correction is completed based on the coordinate correction matrix in the target operation. According to the technical scheme, the console view can be corrected based on user operation habits.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of artificial intelligence technology, and in particular, to a method, device, equipment, medium and product for adjusting the field of view of a surgical robot doctor console. Background Art

[0002] When the surgeon operates the open doctor console of the endoscopic surgical robot, the 3D display screen of the doctor console will provide feedback to the doctor, and the surgeon's own hand movements when operating the doctor console will also provide feedback of the hand movements in the doctor's field of view.

[0003] When the doctor performs horizontal, vertical or depth of field movements, the displayed movement direction may not be the real movement direction due to factors such as the doctor's body shape, sitting posture, and the position of the elbow in the doctor's armrest frame, causing the operating doctor to think that the surgical robot does not follow the movement. In addition, different operating doctors have different feelings about the console field of view of the same surgical robot. If the doctor's operation is not synchronized with the console field of view, it will be dangerous and the console field of view needs to be adjusted. Summary of the invention

[0004] The embodiments of the present invention provide a method, device, equipment, medium and product for adjusting the field of view of a surgical robot doctor console, which can adjust the field of view based on the operating habits of the surgical robot operator himself, so that the movement displayed in the field of view follows the actual operation of the operator.

[0005] In a first aspect, an embodiment of the present invention provides a method for adjusting the field of view of a surgical robot doctor console, the method comprising:

[0006] Before starting the target surgery, obtain the horizontal motion trajectory points and the vertical motion trajectory points of the main operator reference point corresponding to the main operator of the surgical robot within the field of view; wherein the horizontal motion trajectory points and the vertical motion trajectory points are trajectory points generated when the main operator, under the control of the target object, causes the surgical instrument reference point associated with the main operator reference point to move horizontally and vertically within the field of view;

[0007] Performing straight line fitting on the horizontal motion trajectory points to obtain a first straight line, and performing straight line fitting on the vertical motion trajectory points to obtain a second straight line;

[0008] Determine a third straight line in the horizontal plane according to the projection coordinates of at least two points on the first straight line in the horizontal coordinate plane of the main end base coordinate system corresponding to the main operator, and determine a fourth straight line in the vertical plane according to the projection coordinates of at least two points on the second straight line in the vertical coordinate plane of the main end base coordinate system;

[0009] Determine the spatial correction plane of the main end base coordinate system according to the spatial position relationship between the third straight line and the fourth straight line in the main end base coordinate system;

[0010] The coordinate correction matrix of the coordinate object in the main end base coordinate system is determined according to the spatial correction plane of the main end base coordinate system, so as to complete the field of view correction corresponding to the main end base coordinate system based on the coordinate correction matrix in the target surgery.

[0011] In a second aspect, an embodiment of the present invention provides a device for adjusting the field of view of a surgical robot doctor console, the device comprising:

[0012] The main operator motion trajectory point acquisition module is used to acquire the horizontal motion trajectory points and vertical motion trajectory points of the main operator reference point corresponding to the main operator of the surgical robot within the field of view before starting the target surgery; wherein the horizontal motion trajectory points and the vertical motion trajectory points are trajectory points generated when the main operator, under the control of the target object, causes the surgical instrument reference point associated with the main operator reference point to move horizontally and vertically within the field of view;

[0013] The main operator motion trajectory point analysis module is used to perform straight line fitting on the horizontal motion trajectory points to obtain a first straight line, and perform straight line fitting on the vertical motion trajectory points to obtain a second straight line;

[0014] The master operator motion trajectory point mapping module is used to determine the third straight line in the horizontal plane according to the projection coordinates of at least two points on the first straight line in the horizontal coordinate plane of the master end base coordinate system corresponding to the master operator, and determine the fourth straight line in the vertical plane according to the projection coordinates of at least two points on the second straight line in the vertical coordinate plane of the master end base coordinate system;

[0015] The main operator motion trajectory point correction analysis module is used to determine the main end base coordinate system spatial correction plane according to the spatial position relationship between the third straight line and the fourth straight line in the main end base coordinate system;

[0016] The coordinate correction matrix determination module is used to determine the coordinate correction matrix of the coordinate object in the main end base coordinate system according to the spatial correction plane of the main end base coordinate system, so as to complete the field of view correction corresponding to the main end base coordinate system based on the coordinate correction matrix in the target surgery.

[0017] In a third aspect, an embodiment of the present invention further provides a computer device, the computer device comprising:

[0018] one or more processors;

[0019] A memory for storing one or more programs;

[0020] When one or more programs are executed by one or more processors, the one or more processors implement the method for adjusting the field of view of the surgical robot doctor console provided in any embodiment of the present invention.

[0021] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for adjusting the field of view of a surgical robot doctor console as provided in any embodiment of the present invention.

[0022] In a fifth aspect, the embodiments of the present disclosure further provide a computer program product, including a computer program, which, when executed by a processor, implements the method for adjusting the field of view of a surgical robot doctor console as provided in any embodiment of the present invention.

[0023] The embodiments of the above invention have the following advantages or beneficial effects:

[0024] In an embodiment of the present invention, before starting a target surgery, horizontal motion trajectory points and vertical motion trajectory points of a master operator reference point corresponding to a master operator of a surgical robot are obtained within a field of view; wherein the horizontal motion trajectory points and the vertical motion trajectory points are trajectory points generated when the master operator, under the control of a target object, causes a surgical instrument reference point associated with the master operator reference point to move horizontally and vertically within the field of view; a first straight line is obtained by performing straight line fitting on the horizontal motion trajectory points, and a second straight line is obtained by performing straight line fitting on the vertical motion trajectory points; and a straight line is obtained by performing straight line fitting on the vertical motion trajectory points according to at least two points on the first straight line in the master operator corresponding to the master operator. The third straight line in the horizontal plane is determined based on the projection coordinates of the end base coordinate system in the horizontal coordinate plane, and the fourth straight line in the vertical plane is determined based on the projection coordinates of at least two points on the second straight line in the vertical coordinate plane of the main end base coordinate system; the main end base coordinate system space correction plane is determined based on the spatial position relationship between the third straight line and the fourth straight line in the main end base coordinate system; the coordinate correction matrix of the coordinate object in the main end base coordinate system is determined based on the main end base coordinate system space correction plane, so as to complete the field of view correction corresponding to the main end base coordinate system based on the coordinate correction matrix in the target surgery. The disclosed embodiment solves the problem of feedback of hand movement and actual operation following in the field of view of the surgical robot, realizes the adjustment of the field of view based on the operation habits of the user operating the surgical robot, so that the movement displayed in the field of view follows the actual operation of the user, and improves the application safety of the surgical robot in surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flow chart of a method for adjusting the field of view of a surgical robot doctor console provided by an embodiment of the present invention;

[0026] Figure 2It is a schematic diagram of the spatial position relationship of a straight line in a main end base coordinate system provided by an embodiment of the present invention;

[0027] Figure 3 This is a flow chart of a method for adjusting the field of view of a surgical robot doctor console provided by an embodiment of the present invention;

[0028] Figure 4 This is a flow chart of a method for adjusting the field of view of a surgical robot doctor console provided by an embodiment of the present invention;

[0029] Figure 5 It is a structural schematic diagram of a visual field adjustment device of a surgical robot doctor console provided by an embodiment of the present invention;

[0030] Figure 6 It is a structural schematic diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0032] Figure 1 This is a flowchart of a method for adjusting the field of view of a surgical robot doctor console provided in an embodiment of the present invention. This embodiment can be applied to the scenario of correcting the field of view of a surgical robot console, especially an open doctor console. The method can be performed by a surgical robot doctor console field of view adjustment device, which can be implemented by software and / or hardware and integrated into a computer device with application development function.

[0033] like Figure 1 As shown, the method for adjusting the field of view of the surgical robot doctor console of this embodiment includes the following steps:

[0034] S110, before starting the target surgery, obtaining the horizontal motion trajectory points and the vertical motion trajectory points of the main operator reference point corresponding to the main operator of the surgical robot within the field of view.

[0035] The operator of the surgical robot is the target object. Before operating the surgical robot, the operator needs to log in to the surgical robot system to confirm the identity of the target object. In this embodiment, considering the different operating habits of each target object, such as the body shape, sitting posture and elbow position of the console armrest frame of different target objects, the operation feedback of the target object in the surgical field of view will be affected. Therefore, the field of view of the console will be adjusted before the operation begins. Among them, the target operation refers to the operation to be performed after the target object logs in to the surgical robot system.

[0036] The horizontal motion trajectory points and the vertical motion trajectory points of the master operator's reference point within the field of view are trajectory points generated when the master operator, under the control of the target object, causes the surgical instrument reference point associated with the master operator's reference point to move horizontally and vertically within the field of view.

[0037] When different target objects operate the main operator to make the surgical instrument reference point associated with the main operator reference point move horizontally and vertically within the field of view, the trajectory points generated are different, which reflects the individual differences in the operating habits of different target objects.

[0038] S120, performing straight line fitting on the horizontal motion trajectory points to obtain a first straight line, and performing straight line fitting on the vertical motion trajectory points to obtain a second straight line.

[0039] The number of horizontal motion trajectory points and vertical motion trajectory points may be the same or different. In the process of performing straight line fitting, any straight line fitting method may be used, such as the least square method.

[0040] S130. Determine a third straight line in the horizontal plane according to the projection coordinates of at least two points on the first straight line in the horizontal coordinate plane of the main end base coordinate system corresponding to the main operator, and determine a fourth straight line in the vertical plane according to the projection coordinates of at least two points on the second straight line in the vertical coordinate plane of the main end base coordinate system.

[0041] In the process of obtaining the projection coordinates, at least two points may be randomly selected on the first straight line and the second straight line for projection.

[0042] For example, Figure 2 As shown, the first straight line is L1 obtained by fitting based on m horizontal motion trajectory points, and the second straight line is L2 obtained by fitting based on n vertical motion trajectory points. Two points are randomly selected in L1 and projected in the yz plane to obtain two projection point coordinates P1 and P2. Two points are randomly selected in L2 and projected in the xz plane to obtain two projection point coordinates P3 and P4. Furthermore, the third straight line L3 can be determined based on P1 and P2, and the fourth straight line L4 can be determined based on P3 and P4.

[0043] S140. Determine a spatial correction plane of the main end base coordinate system according to a spatial positional relationship between the third straight line and the fourth straight line in the main end base coordinate system.

[0044] The third straight line and the fourth straight line are two skew lines. A parallel line parallel to the third straight line or the fourth straight line can be used to construct a plane with the fourth straight line or the third straight line, which serves as a spatial plane corresponding to the horizontal motion trajectory and the vertical motion trajectory within the actual field of view of the target object, that is, the spatial correction plane of the main end base coordinate system.

[0045] S150, determining a coordinate correction matrix of the coordinate object in the main end base coordinate system according to the spatial correction plane of the main end base coordinate system, so as to complete field of view correction of the coordinate object in the main end base coordinate system in the target surgery based on the coordinate correction matrix.

[0046] Two lines in the main end base coordinate system space correction plane (such as a parallel line to the third straight line or the fourth straight line, and a line parallel to the fourth straight line or the third straight line) and the normal line of the main end base coordinate system space correction plane can form a coordinate system, and the coordinate axis direction vector of the coordinate system is the corresponding coordinate correction matrix, so as to complete the field of view correction of the coordinate object in the main end base coordinate system in the target surgery based on the coordinate correction matrix.

[0047] After determining the coordinate correction matrix, in the target surgery, the coordinate point of the operating master hand reference point in the main end base coordinate system can be used as a coordinate object; the coordinate object is transformed based on the coordinate correction matrix to obtain the target coordinate object, and the position and posture of the above coordinate point in the main end base coordinate system are corrected to complete the field of view correction corresponding to the main end base coordinate system.

[0048] The technical solution of this embodiment is to obtain the horizontal motion trajectory points and the vertical motion trajectory points of the main operator reference point corresponding to the main operator of the surgical robot within the field of view before starting the target surgery; wherein the horizontal motion trajectory points and the vertical motion trajectory points are trajectory points generated when the main operator, under the control of the target object, makes the surgical instrument reference point associated with the main operator reference point move horizontally and vertically within the field of view; perform straight line fitting on the horizontal motion trajectory points to obtain a first straight line, and perform straight line fitting on the vertical motion trajectory points to obtain a second straight line; according to at least two points on the first straight line corresponding to the main operator, The third straight line in the horizontal plane is determined based on the projection coordinates of the main end base coordinate system in the horizontal coordinate plane, and the fourth straight line in the vertical plane is determined based on the projection coordinates of at least two points on the second straight line in the vertical coordinate plane of the main end base coordinate system; the main end base coordinate system space correction plane is determined based on the spatial position relationship between the third straight line and the fourth straight line in the main end base coordinate system; the coordinate correction matrix of the coordinate object in the main end base coordinate system is determined based on the main end base coordinate system space correction plane, so as to complete the field of view correction corresponding to the main end base coordinate system based on the coordinate correction matrix in the target surgery. The disclosed embodiment solves the problem of feedback of hand movement and actual operation following in the field of view of the surgical robot, realizes the adjustment of the field of view based on the operation habits of the user operating the surgical robot, so that the movement displayed in the field of view follows the actual operation of the user, and improves the application safety of the surgical robot in surgery.

[0049] Figure 3This is a flowchart of a method for adjusting the field of view of a surgical robot doctor console provided in an embodiment of the present invention. This embodiment and the method for adjusting the field of view of a surgical robot doctor console in the above embodiment belong to the same inventive concept, and further describes the process of determining the coordinate correction matrix. The method can be executed by a surgical robot doctor console field of view adjustment device, which can be implemented by software and / or hardware and integrated into a computer device with application development function.

[0050] like Figure 3 As shown, the method for adjusting the field of view of the surgical robot doctor console of this embodiment includes the following steps:

[0051] S210: Before starting the target surgery, obtain the horizontal motion trajectory points and the vertical motion trajectory points of the main operator reference point corresponding to the main operator of the surgical robot within the field of view.

[0052] Among them, the horizontal motion trajectory points and the vertical motion trajectory points are trajectory points generated when the main operator, under the control of the target object, causes the surgical instrument reference point associated with the main operator reference point to move horizontally and vertically within the field of view.

[0053] S220 , performing straight line fitting on the horizontal motion trajectory points to obtain a first straight line, and performing straight line fitting on the vertical motion trajectory points to obtain a second straight line.

[0054] S230. Determine the third straight line in the horizontal plane according to the projection coordinates of at least two points on the first straight line in the horizontal coordinate plane of the main end base coordinate system corresponding to the main operator, and determine the fourth straight line in the vertical plane according to the projection coordinates of at least two points on the second straight line in the vertical coordinate plane of the main end base coordinate system.

[0055] S240, calculating the midpoint coordinates of the common perpendicular line between the third straight line and the fourth straight line, and obtaining the midpoint coordinates of the target common perpendicular line on the extension line of the third straight line.

[0056] The common perpendicular line refers to a straight line that intersects two skew lines perpendicularly. In the main end base coordinate system of the surgical robot, it can be used as a reference line to describe the relationship between the third straight line and the fourth straight line.

[0057] The method for determining the common perpendicular line can be to determine the position and direction of the common perpendicular line between the two planes of the main end base coordinate system by mathematical methods. Specifically, the direction vector of the common perpendicular line can be solved by using operations such as the cross product and dot product of vectors, and then the position of the common perpendicular line can be determined based on the known points of the third and fourth straight lines, and then the coordinates of the midpoint of the common perpendicular line can be obtained.

[0058] The coordinates of the midpoint of the common perpendicular line can be as follows Figure 2 The coordinates of point P5 in the schematic diagram shown.

[0059] S250. Determine a target parallel line that passes through the midpoint coordinates of the target common perpendicular line and is parallel to the direction vector of the fourth straight line according to the direction vector of the fourth straight line and the midpoint coordinates of the target common perpendicular line.

[0060] The target parallel line can be Figure 2 The corresponding straight line between points P5 and P6, where P6 can be any coordinate point on the target parallel line except P5.

[0061] S260, determining a spatial correction plane of the main end base coordinate system according to the target parallel line and the third straight line.

[0062] The main end base coordinate system space correction plane is a plane that includes both the target parallel line and the third straight line.

[0063] S270. Determine the normal vector of the main end base coordinate system space correction plane according to the cross product of the first direction vector of the target parallel line in the main end base coordinate system space correction plane and the second direction vector of the third straight line.

[0064] The first direction vector may correspond to Figure 2 The vector formed by P5P6 in the figure, the second direction vector can correspond to Figure 2 The vector formed by P5P2 in the figure. P5P6 is cross-multiplied by P5P2 to obtain the normal vector perpendicular to the plane where P6, P5 and P2 are located, that is, the normal vector of the space correction plane of the main end base coordinate system.

[0065] S280, respectively calculating the unit vectors of the first direction vector, the second direction vector and the normal vector, and forming a coordinate correction matrix based on the unit vectors, so as to complete the field of view correction corresponding to the main end base coordinate system based on the coordinate correction matrix in the target surgery.

[0066] The unit vectors of the first direction vector, the second direction vector and the normal vector are calculated respectively, that is, the first direction vector, the second direction vector and the normal vector are normalized, and a coordinate correction matrix is ​​formed based on the unit vectors. Thus, the position and posture of the object coordinates in the main end base coordinate system can be corrected. For example, the coordinates of the main operating hand reference point of the surgical robot in the main end base coordinate system are multiplied by the coordinate correction matrix to complete the position and posture correction, that is, the field of view correction corresponding to the main end base coordinate system is completed based on the coordinate correction matrix.

[0067] The technical solution of this embodiment is as follows: before starting the target surgery, the horizontal motion trajectory points and the vertical motion trajectory points of the main operator reference point corresponding to the main operator of the surgical robot are obtained within the field of view; the horizontal motion trajectory points are fitted with a straight line to obtain a first straight line, and the vertical motion trajectory points are fitted with a straight line to obtain a second straight line; according to the projection coordinates of at least two points on the first straight line in the horizontal coordinate plane of the main end base coordinate system corresponding to the main operator, a third straight line in the horizontal plane is determined, and according to the projection coordinates of at least two points on the second straight line in the vertical coordinate plane of the main end base coordinate system, a fourth straight line in the vertical plane is determined; the midpoint coordinates of the common perpendicular line of the third straight line and the fourth straight line are calculated, and the coordinates of the midpoint of the common perpendicular line of the third straight line are obtained. The midpoint coordinates of the target common perpendicular line on the extension line; according to the direction vector of the fourth straight line and the midpoint coordinates of the target common perpendicular line, determine a target parallel line passing through the midpoint coordinates of the target common perpendicular line and parallel to the direction vector of the fourth straight line; according to the target parallel line and the third straight line, determine the main end base coordinate system space correction plane; according to the cross product of the first direction vector of the target parallel line in the main end base coordinate system space correction plane and the second direction vector of the third straight line, determine the normal vector of the main end base coordinate system space correction plane; calculate the unit vectors of the first direction vector, the second direction vector and the normal vector respectively, and form a coordinate correction matrix based on the unit vectors, so as to complete the field of view correction corresponding to the main end base coordinate system based on the coordinate correction matrix in the target surgery. The disclosed embodiment solves the problem of feedback of the hand movement part and the actual operation following in the field of view of the surgical robot, realizes the adjustment of the field of view based on the operation habits of the user operating the surgical robot, so that the display movement in the field of view follows the actual operation of the user, and improves the application safety of the surgical robot in surgery.

[0068] Figure 4 This is a flowchart of a method for adjusting the field of view of a surgical robot doctor console provided in an embodiment of the present invention. This embodiment and the method for adjusting the field of view of a surgical robot doctor console in the above embodiment belong to the same inventive concept, and further illustrates the process of saving the target object whitelist. The method can be executed by a surgical robot doctor console field of view adjustment device, which can be implemented by software and / or hardware and integrated into a computer device with application development function.

[0069] like Figure 4 As shown, the method for adjusting the field of view of the surgical robot doctor console of this embodiment includes the following steps:

[0070] S310. After the target object logs into the surgical robot system, a coordinate correction matrix having a mapping relationship with the target object is searched according to the object identification of the target object.

[0071] Different target objects have different operation habits and may correspond to different coordinate correction matrices. After a target object logs into the surgical robot system, it is possible to query whether there is a coordinate correction matrix matching the target object based on the mapping relationship between the target object and the coordinate correction matrix.

[0072] S320: When a coordinate correction matrix having a mapping relationship with the target object is found, a field of view correction corresponding to the main end base coordinate system is completed in the target surgery based on the found coordinate correction matrix.

[0073] S330. When a coordinate correction matrix having a mapping relationship with the target object is not found, before starting the target surgery, the horizontal motion trajectory points and the vertical motion trajectory points of the main operator reference point corresponding to the main operator of the surgical robot are obtained within the field of view.

[0074] Among them, the horizontal motion trajectory points and the vertical motion trajectory points are trajectory points generated when the main operator, under the control of the target object, causes the surgical instrument reference point associated with the main operator reference point to move horizontally and vertically within the field of view.

[0075] In the case that no coordinate correction matrix with a mapping relationship with the target object is found, it is necessary to further determine the coordinate correction matrix corresponding to the target object, so as to meet the individual operation habits of the target object and realize the correction of the track points in the field of view.

[0076] S340, performing straight line fitting on the horizontal motion trajectory points to obtain a first straight line, and performing straight line fitting on the vertical motion trajectory points to obtain a second straight line.

[0077] S350. Determine the third straight line in the horizontal plane according to the projection coordinates of at least two points on the first straight line in the horizontal coordinate plane of the main end base coordinate system corresponding to the main operator, and determine the fourth straight line in the vertical plane according to the projection coordinates of at least two points on the second straight line in the vertical coordinate plane of the main end base coordinate system.

[0078] S360. Determine a spatial correction plane of the main end base coordinate system according to the spatial position relationship between the third straight line and the fourth straight line in the main end base coordinate system.

[0079] S370: Establish a mapping relationship between the coordinate correction matrix and the target object, and save the mapping relationship.

[0080] After determining the coordinate correction matrix associated with the target object, a mapping relationship between the coordinate correction matrix and the target object can be established and saved to form a list of surgical robot operating users. The target objects in the list can realize adaptive adjustment of the console field of view based on the stored coordinate correction matrix.

[0081] The technical solution of this embodiment is to query a coordinate correction matrix having a mapping relationship with the target object according to the object identifier of the target object after the target object logs in to the surgical robot system; when the coordinate correction matrix having a mapping relationship with the target object is queried, the field of view correction corresponding to the main end base coordinate system is completed based on the queried coordinate correction matrix in the target surgery; when the coordinate correction matrix having a mapping relationship with the target object is not queried, before starting the target surgery, the horizontal motion trajectory points and vertical motion trajectory points of the main operator reference point corresponding to the main operator of the surgical robot are obtained within the field of view; the horizontal motion trajectory points Perform straight line fitting to obtain the first straight line, and perform straight line fitting on the vertical motion trajectory points to obtain the second straight line; determine the third straight line in the horizontal plane according to the projection coordinates of at least two points on the first straight line in the horizontal coordinate plane of the main end base coordinate system corresponding to the main operator, and determine the fourth straight line in the vertical plane according to the projection coordinates of at least two points on the second straight line in the vertical coordinate plane of the main end base coordinate system; determine the main end base coordinate system space correction plane according to the spatial position relationship between the third straight line and the fourth straight line in the main end base coordinate system; establish a mapping relationship between the coordinate correction matrix and the target object, and save the mapping relationship. The disclosed embodiment solves the problem of feedback of the hand motion part and the actual operation following in the field of view of the surgical robot, and can save the coordinate correction matrix corresponding to different target objects to adapt to the different operation habits of different target objects, and adjust the field of view according to the operation habits of different surgical robot operators, so that the movement displayed in the field of view follows the actual operation of the operator, thereby improving the application safety of the surgical robot in surgery.

[0082] Figure 5 A structural schematic diagram of a field of view adjustment device for a surgical robot doctor console provided in an embodiment of the present invention. This embodiment can be applicable to the application process of a surgical robot, especially to the field of view correction of a surgical robot console. The field of view adjustment device for the surgical robot doctor console can be implemented by software and / or hardware and integrated into a computer terminal device with application development function.

[0083] like Figure 5 As shown, the field of view adjustment device of the surgical robot doctor console includes: a main operator motion trajectory point acquisition module 410, a main operator motion trajectory point analysis module 420, a main operator motion trajectory point mapping module 430, a main operator motion trajectory point correction analysis module 440 and a coordinate correction matrix determination module 450.

[0084] The main operator motion trajectory point acquisition module 410 is used to obtain the horizontal motion trajectory points and the vertical motion trajectory points of the main operator reference point corresponding to the main operator of the surgical robot in the field of view before starting the target surgery; the horizontal motion trajectory points and the vertical motion trajectory points are the trajectory points generated when the main operator, under the control of the target object, makes the surgical instrument reference point associated with the main operator reference point move horizontally and vertically in the field of view; the main operator motion trajectory point analysis module 420 is used to perform straight line fitting on the horizontal motion trajectory points to obtain a first straight line, and perform straight line fitting on the vertical motion trajectory points to obtain a second straight line; the main operator motion trajectory point mapping module 430 is used to obtain the first straight line according to at least two straight lines on the first straight line. The projection coordinates of the point in the horizontal coordinate plane of the main end base coordinate system corresponding to the main operator are used to determine the third straight line in the horizontal plane, and the fourth straight line in the vertical plane is determined according to the projection coordinates of at least two points on the second straight line in the vertical coordinate plane of the main end base coordinate system; the main operator motion trajectory point correction analysis module 440 is used to determine the main end base coordinate system space correction plane according to the spatial position relationship between the third straight line and the fourth straight line in the main end base coordinate system; the coordinate correction matrix determination module 450 is used to determine the coordinate correction matrix of the coordinate object in the main end base coordinate system according to the main end base coordinate system space correction plane, so as to complete the field of view correction corresponding to the main end base coordinate system based on the coordinate correction matrix in the target surgery.

[0085] The technical solution of this embodiment is to obtain the horizontal motion trajectory points and the vertical motion trajectory points of the main operator reference point corresponding to the main operator of the surgical robot within the field of view before starting the target surgery; wherein the horizontal motion trajectory points and the vertical motion trajectory points are trajectory points generated when the main operator, under the control of the target object, makes the surgical instrument reference point associated with the main operator reference point move horizontally and vertically within the field of view; perform straight line fitting on the horizontal motion trajectory points to obtain a first straight line, and perform straight line fitting on the vertical motion trajectory points to obtain a second straight line; according to at least two points on the first straight line corresponding to the main operator, The third straight line in the horizontal plane is determined based on the projection coordinates of the main end base coordinate system in the horizontal coordinate plane, and the fourth straight line in the vertical plane is determined based on the projection coordinates of at least two points on the second straight line in the vertical coordinate plane of the main end base coordinate system; the main end base coordinate system space correction plane is determined based on the spatial position relationship between the third straight line and the fourth straight line in the main end base coordinate system; the coordinate correction matrix of the coordinate object in the main end base coordinate system is determined based on the main end base coordinate system space correction plane, so as to complete the field of view correction corresponding to the main end base coordinate system based on the coordinate correction matrix in the target surgery. The disclosed embodiment solves the problem of feedback of hand movement and actual operation following in the field of view of the surgical robot, realizes the adjustment of the field of view based on the operation habits of the user operating the surgical robot, so that the movement displayed in the field of view follows the actual operation of the user, and improves the application safety of the surgical robot in surgery.

[0086] In an optional implementation, the main operator motion trajectory point correction analysis module 440 is specifically used to:

[0087] Calculate the midpoint coordinates of the common perpendicular line between the third straight line and the fourth straight line, and obtain the midpoint coordinates of the target common perpendicular line on the extension line of the third straight line;

[0088] According to the direction vector of the fourth straight line and the coordinates of the midpoint of the target common perpendicular line, a target parallel line passing through the coordinates of the midpoint of the target common perpendicular line and being parallel to the direction vector of the fourth straight line is determined;

[0089] The spatial correction plane of the main end base coordinate system is determined according to the target parallel line and the third straight line.

[0090] In an optional implementation, the coordinate correction matrix determination module 450 is specifically used for:

[0091] Determine the normal vector of the main end base coordinate system space correction plane according to the cross product of the first direction vector of the target parallel line in the main end base coordinate system space correction plane and the second direction vector of the third straight line;

[0092] The unit vectors of the first direction vector, the second direction vector and the normal vector are calculated respectively, and a coordinate correction matrix is ​​formed based on the unit vectors.

[0093] In an optional implementation, the coordinate correction matrix determination module 450 is specifically used for:

[0094] In the target surgery, the coordinates of the reference point of the operating hand in the coordinate system of the master end base are taken as coordinate objects;

[0095] The coordinate object is transformed based on the coordinate correction matrix to obtain the target coordinate object, thereby completing the field of view correction corresponding to the main end base coordinate system.

[0096] In an optional embodiment, the visual field adjustment device of the surgical robot doctor console further includes an information storage module for:

[0097] Establish a mapping relationship between the coordinate correction matrix and the target object, and save the mapping relationship.

[0098] In an optional implementation, the coordinate correction matrix determination module 450 may also be used to:

[0099] Before obtaining the horizontal motion trajectory points and the vertical motion trajectory points of the main operator reference point corresponding to the main operator of the surgical robot within the field of view, after the target object logs into the surgical robot system, querying the coordinate correction matrix having a mapping relationship with the target object according to the object identifier of the target object;

[0100] When a coordinate correction matrix having a mapping relationship with the target object is found, the visual field correction corresponding to the main end base coordinate system is completed in the target surgery based on the found coordinate correction matrix.

[0101] The surgical robot doctor console field of view adjustment device provided in the embodiment of the present invention can execute the surgical robot doctor console field of view adjustment method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0102] Figure 6 A schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Figure 6 A block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 6 The computer device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention. The computer device 12 can be any terminal device with computing capabilities, such as an intelligent controller and server, a mobile phone and other terminal devices.

[0103] like Figure 6As shown, the computer device 12 is in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0104] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0105] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0106] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 6 not shown, usually called a "hard drive"). Although Figure 6 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0107] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0108] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the computer device 12, and / or any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the computer device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with the other modules of the computer device 12 via the bus 18. It should be understood that although Figure 6 Not shown, other hardware and / or software modules may be used in conjunction with computer device 12, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0109] The processing unit 16 executes various functional applications and data processing by running the program stored in the system memory 28, such as implementing the field of view adjustment method of the surgical robot doctor console provided in the embodiment of the present invention, which includes:

[0110] Before starting the target surgery, obtain the horizontal motion trajectory points and the vertical motion trajectory points of the main operator reference point corresponding to the main operator of the surgical robot within the field of view; wherein the horizontal motion trajectory points and the vertical motion trajectory points are trajectory points generated when the main operator, under the control of the target object, causes the surgical instrument reference point associated with the main operator reference point to move horizontally and vertically within the field of view;

[0111] Performing straight line fitting on the horizontal motion trajectory points to obtain a first straight line, and performing straight line fitting on the vertical motion trajectory points to obtain a second straight line;

[0112] Determine a third straight line in the horizontal plane according to the projection coordinates of at least two points on the first straight line in the horizontal coordinate plane of the main end base coordinate system corresponding to the main operator, and determine a fourth straight line in the vertical plane according to the projection coordinates of at least two points on the second straight line in the vertical coordinate plane of the main end base coordinate system;

[0113] Determine the spatial correction plane of the main end base coordinate system according to the spatial position relationship between the third straight line and the fourth straight line in the main end base coordinate system;

[0114] The coordinate correction matrix of the coordinate object in the main end base coordinate system is determined according to the spatial correction plane of the main end base coordinate system, so as to complete the field of view correction corresponding to the main end base coordinate system based on the coordinate correction matrix in the target surgery.

[0115] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method for adjusting the field of view of a surgical robot doctor console provided in any embodiment of the present invention is implemented. The method includes:

[0116] Before starting the target surgery, obtain the horizontal motion trajectory points and the vertical motion trajectory points of the main operator reference point corresponding to the main operator of the surgical robot within the field of view; wherein the horizontal motion trajectory points and the vertical motion trajectory points are trajectory points generated when the main operator, under the control of the target object, causes the surgical instrument reference point associated with the main operator reference point to move horizontally and vertically within the field of view;

[0117] Performing straight line fitting on the horizontal motion trajectory points to obtain a first straight line, and performing straight line fitting on the vertical motion trajectory points to obtain a second straight line;

[0118] Determine a third straight line in the horizontal plane according to the projection coordinates of at least two points on the first straight line in the horizontal coordinate plane of the main end base coordinate system corresponding to the main operator, and determine a fourth straight line in the vertical plane according to the projection coordinates of at least two points on the second straight line in the vertical coordinate plane of the main end base coordinate system;

[0119] Determine the spatial correction plane of the main end base coordinate system according to the spatial position relationship between the third straight line and the fourth straight line in the main end base coordinate system;

[0120] The coordinate correction matrix of the coordinate object in the main end base coordinate system is determined according to the spatial correction plane of the main end base coordinate system, so as to complete the field of view correction corresponding to the main end base coordinate system based on the coordinate correction matrix in the target surgery.

[0121] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0122] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0123] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0124] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0125] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0126] The embodiments of the present disclosure also provide a computer program product, including a computer program, which, when executed by a processor, implements the method for adjusting the field of view of a surgical robot doctor console as provided in any embodiment of the present disclosure.

[0127] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code for the disclosed operation, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0128] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for adjusting the visual field of a surgical robot doctor console, characterized in that: include: Before starting the target surgery, horizontal motion trajectory points and vertical motion trajectory points of a main operator reference point corresponding to a main operator of the surgical robot within the field of view are obtained; wherein the horizontal motion trajectory points and the vertical motion trajectory points are trajectory points generated when the main operator, under the control of the target object, causes a surgical instrument reference point associated with the main operator reference point to move horizontally and vertically within the field of view; Performing straight line fitting on the horizontal motion trajectory points to obtain a first straight line, and performing straight line fitting on the vertical motion trajectory points to obtain a second straight line; Determine a third straight line in the horizontal plane according to the projection coordinates of at least two points on the first straight line in the horizontal coordinate plane of the main end base coordinate system corresponding to the main operator, and determine a fourth straight line in the vertical plane according to the projection coordinates of at least two points on the second straight line in the vertical coordinate plane of the main end base coordinate system; Determine a spatial correction plane of the main end base coordinate system according to a spatial positional relationship between the third straight line and the fourth straight line in the main end base coordinate system; The coordinate correction matrix of the coordinate object in the main end base coordinate system is determined according to the spatial correction plane of the main end base coordinate system, so as to complete the field of view correction corresponding to the main end base coordinate system based on the coordinate correction matrix in the target surgery.

2. The method according to claim 1, characterized in that Determining the main end base coordinate system spatial correction plane according to the spatial position relationship between the third straight line and the fourth straight line in the main end base coordinate system includes: Calculate the midpoint coordinates of the common perpendicular line between the third straight line and the fourth straight line to obtain the midpoint coordinates of the target common perpendicular line on the extension line of the third straight line; According to the direction vector of the fourth straight line and the coordinates of the midpoint of the target common perpendicular line, determine a target parallel line that passes through the coordinates of the midpoint of the target common perpendicular line and is parallel to the direction vector of the fourth straight line; The spatial correction plane of the main end base coordinate system is determined according to the target parallel line and the third straight line.

3. The method according to claim 2, characterized in that The step of determining a coordinate correction matrix of a coordinate object in the master end base coordinate system according to the master end base coordinate system spatial correction plane comprises: Determine the normal vector of the main end base coordinate system space correction plane according to the cross product of the first direction vector of the target parallel line and the second direction vector of the third straight line in the main end base coordinate system space correction plane; The unit vectors of the first direction vector, the second direction vector and the normal vector are calculated respectively, and the coordinate correction matrix is ​​composed based on the unit vectors.

4. The method according to claim 1, characterized in that: The completing the field of view correction corresponding to the main end base coordinate system based on the coordinate correction matrix in the target surgery includes: In the target surgery, the coordinates of the reference point of the operating master hand in the coordinate system of the master end base are used as coordinate objects; The coordinate object is transformed based on the coordinate correction matrix to obtain a target coordinate object, thereby completing the field of view correction corresponding to the main end base coordinate system.

5. The method according to any one of claims 1 to 4, characterized in that: Also includes: A mapping relationship between the coordinate correction matrix and the target object is established, and the mapping relationship is saved.

6. The method according to claim 5, characterized in that Before obtaining the horizontal motion trajectory points and the vertical motion trajectory points of the main operator reference point corresponding to the main operator of the surgical robot within the field of view, the method further includes: After the target object logs into the surgical robot system, querying a coordinate correction matrix having a mapping relationship with the target object according to the object identifier of the target object; In the case where a coordinate correction matrix having a mapping relationship with the target object is queried, the field of view correction corresponding to the main end base coordinate system is completed in the target surgery based on the queried coordinate correction matrix.

7. A device for adjusting the visual field of a surgical robot doctor's console, characterized in that: include: A master operator motion trajectory point acquisition module is used to acquire horizontal motion trajectory points and vertical motion trajectory points of a master operator reference point corresponding to a master operator of a surgical robot within a field of view before starting a target surgery; wherein the horizontal motion trajectory points and the vertical motion trajectory points are trajectory points generated when the master operator, under the control of a target object, causes a surgical instrument reference point associated with the master operator reference point to move horizontally and vertically within the field of view; A main operator motion trajectory point analysis module, used for performing straight line fitting on the horizontal motion trajectory points to obtain a first straight line, and performing straight line fitting on the vertical motion trajectory points to obtain a second straight line; A master operator motion trajectory point mapping module, used to determine a third straight line in the horizontal plane according to the projection coordinates of at least two points on the first straight line in the horizontal coordinate plane of the master end base coordinate system corresponding to the master operator, and to determine a fourth straight line in the vertical plane according to the projection coordinates of at least two points on the second straight line in the vertical coordinate plane of the master end base coordinate system; A main operator motion trajectory point correction analysis module, used to determine the main end base coordinate system spatial correction plane according to the spatial position relationship between the third straight line and the fourth straight line in the main end base coordinate system; A coordinate correction matrix determination module is used to determine the coordinate correction matrix of the coordinate object in the main end base coordinate system according to the spatial correction plane of the main end base coordinate system, so as to complete the field of view correction corresponding to the main end base coordinate system based on the coordinate correction matrix in the target surgery.

8. A computer device, characterized in that: The computer device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the field of view adjustment method of the surgical robot doctor console as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for adjusting the field of view of the surgical robot doctor console as described in any one of claims 1-6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the method for adjusting the field of view of a surgical robot doctor console as described in any one of claims 1 to 6.

Images