Endoscope surgery robot surgery instrument position judgment method and device and storage medium

Abstract

The application provides a laparoscopic surgery robot surgical instrument position determination method and device and a storage medium, and relates to the technical field of medical equipment. The laparoscopic surgery robot surgical instrument position determination method comprises the following steps: acquiring real-time operation parameters of an inner arm of a surgical arm; establishing a geometric parameter model of the inner arm of the surgical arm according to preset inner arm geometric parameters; obtaining the spatial coordinates of the end surgical instrument of the inner arm of the surgical arm in the geometric parameter model coordinate system according to the real-time operation parameters; inputting the spatial coordinates into a visual plane equation to obtain the visual coordinate range of the plane where the surgical instrument is located; and determining the position relationship between the surgical instrument and an endoscope image according to the coordinate range and the spatial coordinates. The application avoids the low recognition accuracy and inaccurate position determination caused by the dependence on image information in the traditional position acquisition method, and effectively improves the timeliness of the position acquisition of all surgical instruments during the operation of the operator.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical equipment, in particular to a laparoscopic surgery robot surgical instrument position determination method and device and storage medium. BACKGROUND

[0002] With the development of the medical robot industry, the application range of the laparoscopic surgery robot is gradually expanding due to its advantages such as small trauma, and suitability for micro-operation environment. The laparoscopic surgery robot should have high safety. However, during the operation, the doctor needs to know the real-time position of the tip of each surgical instrument in real time, so as to quickly find the surgical instrument to complete the switching operation, which is beneficial to improving the operation speed during the operation process, and also to avoid the surgical instrument from exceeding the visual range, thereby causing unnecessary damage to the human body by the surgical instrument outside the visual range. The existing main recognition method is to use an image processing method to identify whether the surgical instrument is located in the visual field. However, the working environment during the operation is complex, and the surgical instrument itself is often blocked by human tissues or other surgical instruments in the image, so that the obtained image cannot guarantee to meet the recognition requirement, and the operation amount is large and the real-time performance is poor. When the surgical instrument exceeds the screen range, there is also a problem that the image recognition method cannot determine the real-time position of the instrument outside the screen visual field, which may cause the human tissue to be damaged by moving the surgical instrument when the surgical instrument exceeds the visual range. SUMMARY

[0003] The problem solved by the present application is how to determine the relative position of the surgical instrument of the laparoscopic surgery robot in the endoscope image.

[0004] To solve the above problems, the present application provides a laparoscopic surgery robot surgical instrument position determination method, device and storage medium.

[0005] In a first aspect, the present application provides a laparoscopic surgery robot surgical instrument position determination method, comprising:

[0006] obtaining real-time running parameters of an inner arm of a surgical arm;

[0007] establishing a geometric parameter model of the inner arm of the surgical arm according to preset geometric parameters of the inner arm;

[0008] obtaining spatial coordinates of a surgical instrument at the tip of the inner arm of the surgical arm in a coordinate system of the geometric parameter model according to the real-time running parameters;

[0009] inputting the spatial coordinates into a visual plane equation to obtain a visual coordinate range of the plane where the surgical instrument is located;

[0010] determining the position relationship between the surgical instrument and the endoscope image according to the coordinate range and the spatial coordinates.

[0011] Optionally, the inner arm geometric parameters include the component dimensions of the surgical arm inner arm and the connection relationships of the surgical arm inner arm; the step of establishing a geometric parameter model of the surgical arm inner arm based on the preset inner arm geometric parameters includes:

[0012] The geometric relationship of the inner arm of the surgical arm is determined based on the component dimensions and the connection relationship;

[0013] The center point of the endoscopic image is determined as the base point, and the geometric parameter model is established based on the geometric relationship of the inner arm of the surgical arm. The geometric parameter model is used to obtain the pose of the inner arm based on the operating parameters.

[0014] Optionally, obtaining the spatial coordinates of the surgical instrument at the inner arm end of the surgical arm in the geometric parameter model coordinate system based on the real-time operating parameters includes:

[0015] The real-time operating parameters are input into the geometric parameter model to obtain the pose of the inner arm of the surgical arm under the current parameters;

[0016] The spatial coordinates of the surgical instrument at the end of the inner arm of the surgical arm in the endoscopic coordinate system are determined based on the pose.

[0017] Optionally, inputting the spatial coordinates into the visual plane equation yields the visual coordinate range of the plane where the surgical instrument is located, including:

[0018] Input the Z-axis coordinate of the spatial coordinates into the visual plane equation to obtain the X-axis range and Y-axis range of the XY plane at the Z-axis coordinate.

[0019] The visual coordinate range of the plane where the surgical instrument is located is determined based on the X-axis range and the Y-axis range.

[0020] Optionally, the visual plane equation includes:

[0021]

[0022] Where x is the X-axis range, y is the Y-axis range, z is the Z-axis coordinate of the spatial coordinates, k is the preset screen ratio, and θ is the angle of the endoscope's visual range.

[0023] Optionally, determining the positional relationship between the surgical instrument and the endoscopic image based on the coordinate range and the spatial coordinates includes:

[0024] Obtain the absolute difference between the coordinate range and the spatial coordinates along the coordinate axis;

[0025] When the absolute value difference is greater than a first preset threshold, it is determined that the surgical instrument is located inside the endoscopic image;

[0026] When the absolute value difference is less than a first preset threshold and greater than a second preset threshold, the surgical instrument is determined to be located at the edge of the endoscopic image.

[0027] When the absolute value difference is less than a second preset threshold, it is determined that the surgical instrument is outside the range of the endoscopic image.

[0028] Optionally, the method for determining the position of surgical instruments in laparoscopic surgical robots also includes:

[0029] When the surgical instrument is located at the edge of the endoscopic image, a first prompting command is generated to control the corresponding frame of the endoscopic image to flash as a prompt.

[0030] When the surgical instrument goes beyond the range of the endoscope image, a second prompt instruction is generated to control the corresponding frame of the endoscope image to remain constantly lit.

[0031] Optionally, the method for determining the position of surgical instruments in laparoscopic surgical robots also includes:

[0032] Each of the surgical instruments is assigned a tag.

[0033] When the first prompting instruction and / or the second prompting instruction of the surgical instrument are detected, the label is displayed at the corresponding frame of the endoscope image.

[0034] The beneficial effects of this invention are as follows: by acquiring parameter information of the surgical arm's inner arm during operation, and obtaining the pose of the surgical arm's inner arm under the current parameters based on a geometric parameter model established by preset geometric parameters, the coordinate position of the corresponding pose in the endoscopic image coordinate system is determined, and the visual plane boundary range where the coordinate point is located is obtained through the visual plane equation. The position of the surgical instrument is determined by comparing the relationship between the horizontal and vertical coordinates of the spatial coordinates and the boundary range. The spatial coordinates of the surgical instrument under the current pose are determined only by the inner arm parameters, and the constraint range of the corresponding cross section is determined based on the relationship of the visual plane equation. Thus, the positional relationship of the surgical instrument in the endoscopic image is obtained by comparing the spatial coordinates and the constraint range. This method avoids the problems of low recognition accuracy and inaccurate position judgment caused by relying on image information in traditional position acquisition methods. Furthermore, it determines the illumination space corresponding to the endoscopic image through the visual plane equation, and judges the positional relationship between the surgical instruments and the illumination space under the current parameters. This avoids the judgment error caused by the clarity of the endoscopic image. At the same time, it also solves the problem that the position of the surgical instruments cannot be clearly determined by the image occlusion caused by multiple surgical instruments themselves and / or each other in the endoscopic image. This effectively improves the timeliness of the operator's acquisition of the position of all surgical instruments during the operation and reduces the possibility of accidental injury caused by surgical instruments going out of the frame.

[0035] In a second aspect, the present invention provides a device for determining the position of surgical instruments in a laparoscopic surgical robot, comprising:

[0036] A detection unit, which is used to acquire real-time operating parameters of the surgical arm's inner arm;

[0037] The model simulation unit is used to establish a geometric parameter model of the inner arm of the surgical arm based on preset inner arm geometric parameters.

[0038] A coordinate acquisition unit is used to obtain the spatial coordinates of the surgical instrument at the end of the inner arm of the surgical arm in the geometric parameter model coordinate system according to the real-time operating parameters.

[0039] A plane determination unit is used to input the spatial coordinates into the visual plane equation to obtain the visual coordinate range of the plane where the surgical instrument is located;

[0040] A comparison unit is used to determine the positional relationship between the surgical instrument and the endoscopic image based on the coordinate range and the spatial coordinates.

[0041] The laparoscopic surgical robot instrument position determination device and the laparoscopic surgical robot instrument position determination method have the same advantages over the prior art, and will not be repeated here.

[0042] Thirdly, the present invention provides a computer-readable storage medium storing a computer program, which is read and executed by a processor to implement the method for determining the position of surgical instruments in a laparoscopic surgical robot as described in any one of the first aspects.

[0043] The computer-readable storage medium described in this invention has the same advantages over the prior art as the method for determining the position of surgical instruments in a laparoscopic surgical robot, and will not be repeated here. Attached Figure Description

[0044] Figure 1 This is a flowchart illustrating the method for determining the position of surgical instruments in a laparoscopic surgical robot according to an embodiment of the present invention.

[0045] Figure 2 This is a schematic diagram of the endoscopic illumination area of ​​the laparoscopic surgical robot according to an embodiment of the present invention;

[0046] Figure 3 This is a schematic diagram of the display screen of the laparoscopic surgical robot according to an embodiment of the present invention;

[0047] Figure 4 This is a schematic diagram of the surgical instrument position determination device of the laparoscopic surgical robot according to an embodiment of the present invention. Detailed Implementation

[0048] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Although some embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the accompanying drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0049] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments"; the term "optionally" means "optional embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc., mentioned in this invention are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0050] It should be noted that the terms "a" and "a plurality of" used in this invention are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0051] The names of the messages or information exchanged between the multiple devices in the embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0052] like Figure 1 As shown, an embodiment of the present invention provides a method for determining the position of surgical instruments in a laparoscopic surgical robot, comprising:

[0053] Step S1: Obtain the real-time operating parameters of the inner arm of the surgical arm;

[0054] Specifically, in this embodiment, the operating parameters of each joint point of the surgical arm are obtained through sensor devices at each joint of the surgical arm, such as the torsion angle and pitch angle at the joint.

[0055] Step S2: Establish the geometric parameter model of the inner arm of the surgical arm based on the preset inner arm geometric parameters;

[0056] Specifically, in this embodiment, a geometric parameter model of the inner arm of the surgical arm is established based on preset parameters such as the arm length and arm width of the inner arm, as well as the connection relationship between the various components of the inner arm of the surgical arm, wherein the surgical instrument is located at the end of the inner arm of the surgical arm.

[0057] Step S3: Obtain the spatial coordinates of the surgical instrument at the end of the inner arm of the surgical arm in the geometric parameter model coordinate system based on the real-time operating parameters;

[0058] Specifically, the operating parameters of each joint of the surgical arm obtained by detection are input into the ensemble parameter model to simulate the position and posture of the inner arm of the surgical arm under the current parameters, and the inner arm of the surgical arm in this position is placed into the endoscopic image coordinate system to obtain the spatial coordinate points of the surgical instruments.

[0059] Step S4: Input the spatial coordinates into the visual plane equation to obtain the visual coordinate range of the plane where the surgical instrument is located;

[0060] Specifically, the spatial coordinates obtained in the previous step are input into the visual plane equation to obtain the visible range of the plane parallel to the lens at the corresponding spatial coordinates. The coordinate threshold of the corresponding visible range is determined, and the obtained X-axis range and Y-axis range are determined respectively.

[0061] Step S5: Determine the positional relationship between the surgical instruments and the endoscopic image based on the coordinate range and the spatial coordinates.

[0062] Specifically, in this embodiment, the relationship between the X coordinate value and the X-axis coordinate range, and the relationship between the Y-axis coordinate value and the Y-axis coordinate range, of the obtained spatial coordinates (X, Y, Z) are compared to obtain the minimum absolute value of the difference between the spatial coordinates and the coordinate range. The scaling factor of the plane where the spatial coordinate point of the surgical instrument is located is determined by the actual display screen parameters and the visual coordinate range. The above-mentioned absolute value of the difference is magnified or reduced according to the scaling factor to obtain the mapping difference. Then, it is compared with the first preset threshold and the second preset threshold to determine the current position of the surgical instrument.

[0063] This embodiment only demonstrates the positional process of a single surgical instrument on the inner arm of the operating arm within the endoscopic image. It should be understood that, taking the determination of the position of a single surgical instrument at the end of the inner arm as an example, the operating parameters of the inner arm containing the surgical instrument are substituted into the corresponding geometric parameter model of the inner arm to determine its spatial coordinates. It is then determined whether these spatial coordinates exceed the visible range of the plane in which they are located, thereby determining the positional relationship of the surgical instrument in the endoscopic image. For operating arms carrying multiple surgical instruments, the method for determining the position of each instrument in the endoscopic image is not elaborated upon; the process of acquisition, simulation, plane range determination, and comparison described above can be followed.

[0064] In this embodiment, by acquiring the parameter information of the surgical arm's inner arm during operation and obtaining the pose of the surgical arm's inner arm under the current parameters based on the geometric parameter model established by the preset geometric parameters, the coordinate position of the corresponding pose in the endoscopic image coordinate system is determined. The visual plane boundary range where the coordinate point is located is obtained through the visual plane equation. The position of the surgical instrument is determined by comparing the relationship between the horizontal and vertical coordinates of the spatial coordinates and the boundary range. The spatial coordinates of the surgical instrument under the current pose are determined only by the inner arm parameters, and the constraint range of the corresponding cross section is determined based on the relationship of the visual plane equation. Thus, the positional relationship of the surgical instrument in the endoscopic image is obtained by comparing the spatial coordinates and the constraint range. This method avoids the problems of low recognition accuracy and inaccurate position judgment caused by relying on image information in traditional position acquisition methods. Furthermore, it determines the illumination space corresponding to the endoscopic image through the visual plane equation, and judges the positional relationship between the surgical instruments and the illumination space under the current parameters. This avoids the judgment error caused by the clarity of the endoscopic image. At the same time, it also solves the problem that the position of the surgical instruments cannot be clearly determined by the image occlusion caused by multiple surgical instruments themselves and / or each other in the endoscopic image. This effectively improves the timeliness of the operator's acquisition of the position of all surgical instruments during the operation and reduces the possibility of accidental injury caused by surgical instruments going out of the frame.

[0065] In an optional embodiment, the inner arm geometric parameters include the component dimensions of the surgical arm inner arm and the connection relationships of the surgical arm inner arm; the step of establishing a geometric parameter model of the surgical arm inner arm based on the preset inner arm geometric parameters includes:

[0066] The geometric relationship of the inner arm of the surgical arm is determined based on the component dimensions and the connection relationship;

[0067] The center point of the endoscopic image is determined as the base point, and the geometric parameter model is established based on the geometric relationship of the inner arm of the surgical arm. The geometric parameter model is used to obtain the pose of the inner arm based on the operating parameters.

[0068] Specifically, based on the actual geometric dimensions of the inner arm structure (such as length, width, height, and shape) and the connection relationships of the surgical inner arm (such as contact, sliding connection, and rotational connection between inner arm components), the geometric and physical relationships of the surgical inner arm are determined. The center point of the endoscopic image is used as the base point, the image normal passing through the center point is the Z-axis, the right direction of the image is the positive X-axis, and the top direction of the image is the positive Y-axis to establish a spatial coordinate system for the endoscopic image. Based on the geometric and physical relationships of the surgical inner arm, a geometric parameter model is established at the origin of the coordinate system, and the base point of the geometric parameter model coincides with the origin of the coordinate system.

[0069] In this embodiment, a geometric parameter model of the inner arm of the surgical arm is simulated by preset geometric information and actual real-time parameter information, and set in the endoscope coordinate system to obtain the spatial coordinates in the endoscopic image range. The actual terminal position coordinates are quickly and accurately determined. The use of parameter model avoids the dependence of traditional visual recognition methods on images and solves the problem of not being able to accurately obtain the position of surgical instruments in real time when the image is blurry or the target is occluded.

[0070] In an optional embodiment, obtaining the spatial coordinates of the surgical instrument at the inner arm end of the surgical arm in the geometric parameter model coordinate system based on the real-time operating parameters includes:

[0071] The real-time operating parameters are input into the geometric parameter model to obtain the pose of the inner arm of the surgical arm under the current parameters;

[0072] The spatial coordinates of the surgical instrument at the end of the inner arm of the surgical arm in the endoscopic coordinate system are determined based on the pose.

[0073] Specifically, the acquired real-time operating parameters are input into the established geometric parameter model, such as the torsion angle of each joint, to obtain the pose simulated by the geometric parameter model. This pose state is then represented by the spatial coordinate system of the parameter model base point to obtain the coordinate position of the actual surgical instruments in the endoscopic image.

[0074] In this embodiment, by combining the geometric parameter model with the spatial coordinate system, the actual pose of the geometric parameter model is simulated based on the acquired operating parameters and its spatial coordinates in the visual range are determined. This enables the rapid determination of the actual position of the surgical instrument, reduces the amount of computation in the recognition and judgment process, improves the sensitivity to changes in the position of the surgical instrument, and reduces errors in the judgment process.

[0075] In an optional embodiment, inputting the spatial coordinates into the visual plane equation to obtain the visual coordinate range of the plane where the surgical instrument is located includes:

[0076] Input the Z-axis coordinate of the spatial coordinates into the visual plane equation to obtain the X-axis range and Y-axis range of the XY plane at the Z-axis coordinate.

[0077] The visual coordinate range of the plane where the surgical instrument is located is determined based on the X-axis range and the Y-axis range.

[0078] Furthermore, the visual plane equation includes:

[0079]

[0080] Where x is the X-axis range, y is the Y-axis range, z is the Z-axis coordinate of the spatial coordinates, k is the preset screen ratio, and θ is the angle of the endoscope's visual range.

[0081] Specifically, such as Figure 2 As shown, the spatial coordinates (x1, y1, z1) of the surgical instrument are obtained through the physical geometric model. The ordinate z1 is input into the above-mentioned f(z) function to obtain the constraint range x2 of the X-axis and the constraint range y2 of the Y-axis. Thus, the range of the plane where the surgical instrument is located is determined to be x∈(-x2, x2) and y∈(-y2, y2). Then, the relationship between x1 and y1 in the spatial coordinates and the constraint range is compared.

[0082] The visual plane equation is used to calculate the geometric information of the cross-section of the endoscope view at the current height based on the preset screen ratio k and the measured endoscope visual angle θ, and by substituting the obtained height coordinate z. This yields the four endpoints A, B, C, and D of the cross-section rectangle. A(-x, y) is the endpoint of the second quadrant in the XY plane; B(-x, y) is the endpoint of the first quadrant in the XY plane; C(-x, -y) is the endpoint of the third quadrant in the XY plane; and D(x, -y) is the endpoint of the fourth quadrant in the XY plane. Z(H) is the distance of the cross-section from the endoscope coordinate system, L is the length of the visual range, W is the width of the visual range, and R is the visual radius of the corresponding cross-section.

[0083] In this embodiment, by setting a visual plane equation, the visual plane range of the current height of the surgical instrument is obtained, and then the relationship between the two is determined. This is beneficial for quickly determining the positional relationship of the surgical instrument, with fast judgment speed and high reliability, which helps users to discover and adjust the position of the surgical instrument in a timely manner.

[0084] In an optional embodiment, determining the positional relationship between the surgical instruments and the endoscopic image based on the coordinate range and the spatial coordinates includes:

[0085] Obtain the absolute difference between the coordinate range and the spatial coordinates along the coordinate axis;

[0086] When the absolute value difference is greater than a first preset threshold, it is determined that the surgical instrument is located inside the endoscopic image;

[0087] When the absolute value difference is less than a first preset threshold and greater than a second preset threshold, the surgical instrument is determined to be located at the edge of the endoscopic image.

[0088] When the absolute value difference is less than a second preset threshold, it is determined that the surgical instrument is outside the range of the endoscopic image.

[0089] Furthermore, the method for determining the position of surgical instruments in laparoscopic surgical robots also includes:

[0090] When the surgical instrument is located at the edge of the endoscopic image, a first prompting command is generated to control the corresponding frame of the endoscopic image to flash as a prompt.

[0091] When the surgical instrument goes beyond the range of the endoscope image, a second prompt instruction is generated to control the corresponding frame of the endoscope image to remain constantly lit.

[0092] Furthermore, the method for determining the position of surgical instruments in laparoscopic surgical robots also includes:

[0093] Each of the surgical instruments is assigned a tag.

[0094] When the first prompting instruction and / or the second prompting instruction of the surgical instrument are detected, the label is displayed at the corresponding frame of the endoscope image.

[0095] Specifically, such as Figure 3As shown, for example, if the spatial coordinate point P(3, 2, 6) is obtained, the visible coordinate range of the XY plane where the height 6 is located is then determined. A(-x, y), B(-x, y), C(-x, -y), and D(x, -y) are the four endpoints of the cross-sectional rectangle, where x∈(-5, 5) and y∈(-4, 4). That is, we get A(-5, 4), B(-5, 4), C(-5, -4), and D(5, -4). Then, we determine that the absolute value difference of point P on the X-axis is 2 and the absolute value difference on the Y-axis is 3. Based on the actual screen values, we determine the scaling factor to be 0.5. After scaling, we obtain the final minimum difference value of 1. At this time, we compare it with the preset first threshold of 0.5 and the second threshold of 0 to determine that point P is within the screen range and the surgical instrument is within the visible range.

[0096] In another scenario, there are points Q (3, 3.6, 6) with a final minimum difference of 0.2 and points W (-5.2, 3, 6) with a minimum difference of -0.1. By comparing the preset threshold ranges, we can determine that point Q is located at the edge of the screen. In this case, the surgical instrument label corresponding to point Q is generated and placed at the upper edge of the endoscope image, and the upper edge of the image is controlled to flash. Point W is outside the endoscope image, and the surgical instrument is not within the visible range. The surgical instrument corresponding to point W is located outside the left edge of the image. In this case, the surgical instrument label corresponding to point W is generated and placed at the left edge of the endoscope image, and the left edge of the image is controlled to remain lit as a prompt.

[0097] In this embodiment, by comparing the visible coordinate range of the spatial cross-section where the surgical instrument is located with a preset threshold, the position, orientation, and position status of the surgical instrument in the endoscopic image are determined. When the surgical instrument approaches the edge of the screen, a label is generated and a prompt is given. When the surgical instrument goes beyond the screen range, label information is generated at the corresponding boundary and a warning is given. This helps the operator to confirm the position of each surgical instrument in a timely manner and helps to find surgical instruments that are beyond the screen position in a timely manner. This not only makes it convenient for the operator to find the location of the surgical instrument in a timely manner, but also helps to prompt the location of unknown instruments. This helps to avoid the situation where unknown surgical instruments that go beyond the screen cause accidental injury to the human body when the surgical device moves forward.

[0098] For example, alternative solutions of the present invention further include: analyzing the endoscopic image using a preset image segmentation model to obtain the positional relationship of the surgical instruments in the image; when the image segmentation model cannot obtain the target information, obtaining the positional relationship of the surgical instruments using the technical solution in any of the above embodiments, and virtually displaying the blurred and / or occluded target in the endoscopic image according to the obtained positional relationship result.

[0099] This solution uses basic image recognition methods to determine the position of surgical instruments. When the surgical instruments cannot be accurately identified as the target object, the above-mentioned surgical instrument position determination method is used to obtain the positional relationship of the surgical instruments in the image. Then, the obscured or unclear surgical instruments are virtually marked, which helps users intuitively understand the position of the surgical instruments in the image.

[0100] Secondly, combining Figure 4 As shown, the present invention provides a device for determining the position of surgical instruments in a laparoscopic surgical robot, comprising:

[0101] A detection unit, which is used to acquire real-time operating parameters of the surgical arm's inner arm;

[0102] The model simulation unit is used to establish a geometric parameter model of the inner arm of the surgical arm based on preset inner arm geometric parameters.

[0103] A coordinate acquisition unit is used to obtain the spatial coordinates of the surgical instrument at the end of the inner arm of the surgical arm in the geometric parameter model coordinate system according to the real-time operating parameters.

[0104] A plane determination unit is used to input the spatial coordinates into the visual plane equation to obtain the visual coordinate range of the plane where the surgical instrument is located;

[0105] A comparison unit is used to determine the position of the surgical instrument and the endoscopic image based on the coordinate range and the spatial coordinates.

[0106] The laparoscopic surgical robot instrument position determination device and the laparoscopic surgical robot instrument position determination method have the same advantages over the prior art, and will not be repeated here.

[0107] Thirdly, the present invention provides a computer-readable storage medium storing a computer program, which is read and executed by a processor to implement the method for determining the position of surgical instruments in a laparoscopic surgical robot as described in any one of the first aspects.

[0108] The computer-readable storage medium described in this invention has the same advantages over the prior art as the method for determining the position of surgical instruments in a laparoscopic surgical robot, and will not be repeated here.

[0109] The present invention will now describe electronic devices that can serve as servers or clients of the present invention, which are examples of hardware devices that can be applied to various aspects of the present invention. Electronic devices are intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0110] Electronic devices include a computing unit that can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) or loaded from a storage unit into random access memory (RAM). The RAM can also store various programs and data required for device operation. The computing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0111] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc. In this application, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention according to actual needs. Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units can be implemented in hardware or as software functional units.

[0112] While the present invention has been disclosed above, its scope of protection is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and all such changes and modifications will fall within the scope of protection of the present invention.

Claims

1. A laparoscopic surgery robot surgery instrument position determination device, characterized by, The method comprises the following steps: a detection unit is used to obtain real-time running parameters of an inner arm of a surgical arm; a model simulation unit is used to establish a geometric parameter model of the inner arm of the surgical arm according to preset inner arm geometric parameters, wherein the inner arm geometric parameters comprise component sizes of the inner arm of the surgical arm and connection relationships of the inner arm of the surgical arm; the establishment of the geometric parameter model of the inner arm of the surgical arm according to the preset inner arm geometric parameters comprises: determining a geometric relationship of the inner arm of the surgical arm according to the component sizes and the connection relationships; determining a center point of an endoscope image as a base point, and establishing the geometric parameter model according to the geometric relationship of the inner arm of the surgical arm, wherein the geometric parameter model is used to obtain a pose of the inner arm according to the running parameters; a coordinate acquisition unit is used to obtain a spatial coordinate of a surgical instrument at a terminal end of the inner arm of the surgical arm in a coordinate system of the geometric parameter model according to the real-time running parameters; a plane determination unit is used to input the spatial coordinate into a visual plane equation to obtain a visual coordinate range of a plane where the surgical instrument is located; a comparison unit is used to determine a positional relationship between the surgical instrument and the endoscope image according to the coordinate range and the spatial coordinate.

2. The laparoscopic surgical robotic surgical instrument position determination device of claim 1, wherein, the obtaining of the spatial coordinate of the surgical instrument at the terminal end of the inner arm of the surgical arm in the coordinate system of the geometric parameter model according to the real-time running parameters comprises: inputting the real-time running parameters into the geometric parameter model to obtain a pose of the inner arm of the surgical arm under current parameters; determining the spatial coordinate of the surgical instrument at the terminal end of the inner arm of the surgical arm in an endoscope coordinate system according to the pose.

3. The laparoscopic surgical robotic surgical instrument position determination device of claim 1, wherein, the inputting of the spatial coordinate into the visual plane equation to obtain the visual coordinate range of the plane where the surgical instrument is located comprises: inputting a Z-axis coordinate of the spatial coordinate into the visual plane equation to obtain an X-axis range and a Y-axis range of an X-Y plane at the Z-axis coordinate; determining the visual coordinate range of the plane where the surgical instrument is located according to the X-axis range and the Y-axis range.

4. The laparoscopic surgical robotic surgical instrument position determination device of claim 3, wherein, the visual plane equation comprises: ； wherein x is the X-axis range, y is the Y-axis range, z is the Z-axis coordinate of the spatial coordinate, k is a preset screen ratio, and θ is an included angle of an endoscope visual range.

5. The apparatus according to claim 1, wherein the determination of the positional relationship between the surgical instrument and the endoscope image according to the coordinate range and the spatial coordinate comprises: obtaining an absolute value difference of the coordinate range and the spatial coordinate along a coordinate axis direction; when the absolute value difference is greater than a first preset threshold, it is determined that the surgical instrument is located inside the endoscope image; when the absolute value difference is less than the first preset threshold and greater than a second preset threshold, it is determined that the surgical instrument is located at an edge of the endoscope image; when the absolute value difference is less than the second preset threshold, it is determined that the surgical instrument is out of the range of the endoscope image.

6. The laparoscopic surgical robotic surgical instrument position determination device of claim 5, wherein, further comprising: when the surgical instrument is located at the edge of the endoscope image, a first prompt instruction is generated to control a corresponding picture frame in the endoscope image to flash and prompt. When the surgical instrument is out of the endoscope image range, a second prompt instruction is generated for controlling the corresponding picture frame in the endoscope image to be constantly prompted.

7. The laparoscopic surgical robotic surgical instrument position determination device of claim 6, wherein, Further comprising: Label information is respectively given to a plurality of surgical instruments; When the first prompt instruction and / or the second prompt instruction of the surgical instrument is detected, the label information is displayed at the corresponding picture frame in the endoscope image.

Images