A laparoscopic surgery robot display system, method, device and storage medium

Abstract

The application provides a kind of endoscopic surgery robot display system, method, device and storage medium, it is related to medical equipment technical field, wherein endoscopic surgery robot display system includes: according to the pose information of surgical instrument determined according to the geometric parameter of surgical arm device, whether surgical instrument is located in the visible range of display screen according to pose information determination;When surgical instrument is located in the visible range of display screen, the image information of endoscope device is acquired;And according to pre-trained image recognition model, image information is analyzed, to determine whether there is shielding condition;When image information exists shielding, according to the matching characteristics of pose information and image information, surgical instrument contour image is generated, and surgical instrument contour image is displayed on display screen, to virtually display the surgical instrument that is shielded in image information. Improve the timeliness of the position acquisition of operating personnel to surgical instrument, improve the control of operating personnel for the current working condition of surgical instrument.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical equipment, in particular to a laparoscopic surgery robot display system, method, device and storage medium. BACKGROUND

[0002] With the development of the medical robot industry, the position of the laparoscopic surgery robot rises to the top of the surgical robot. The laparoscopic surgery robot should have high safety, and the doctor needs to know the real-time pose of the end of the surgical instrument in the visual field in real time during the operation. However, in the actual application process, the image of the endoscope is not stable, and the image is often not clear due to the blocking of the light source, the image is not visible due to the blocking of the surgical instrument by human tissue or the end mechanical device during the movement, and the doctor cannot observe the instrument pose through the endoscope when the smoke, blood flow and other obstructions appear during the operation, which has a great safety hazard. SUMMARY

[0003] The problem solved by the present application is how to display the surgical instrument image in the display screen in real time.

[0004] To solve the above problems, the present application provides a laparoscopic surgery robot surgical instrument display system, which comprises a surgical arm device, an endoscope device, a central processing device and a display screen, the central processing device is respectively connected with the surgical arm device, the endoscope device and the display screen in communication; the endoscope device and the display screen are connected in communication; the end of the surgical arm device is provided with a surgical instrument, and the central processing device is used for:

[0005] determining the pose information of the surgical instrument relative to the endoscope device according to the geometric parameters of the surgical arm device, and judging whether the surgical instrument is located in the visible range of the display screen according to the pose information;

[0006] when the surgical instrument is located in the visible range of the display screen, acquiring the image information of the endoscope device; and analyzing the image information according to a pre-trained image recognition model to judge whether there is a surgical instrument blocking condition in the image information;

[0007] when the image information is blocked, generating a surgical instrument contour image according to the pose information and the image information, and displaying the surgical instrument contour image on the display screen, so as to virtually display the surgical instrument blocked in the image information.

[0008] Optionally, the determination of the pose information of the surgical instrument relative to the endoscope device according to the geometric parameters of the surgical arm device, and the judgment of whether the surgical instrument is located in the visible range of the display screen according to the pose information, comprises:

[0009] substituting the geometric parameters into a kinematics model of the surgical arm device, determining vector information of an end of the surgical arm device relative to the endoscope device, the vector information being used to determine pose information and position information of the surgical instrument;

[0010] determining whether the end of the surgical arm device is beyond a screen range according to a relationship between a range corresponding to the vector information and a display range of the endoscope.

[0011] Optionally, the geometric parameters include: link length, link offset, link torsion angle and link rotation angle; and the kinematics equation includes:

[0012] T = f(ai, di, ai, qi);

[0013] wherein ai is the link length, di is the link offset, ai is the link torsion angle, qi is the link rotation angle, and T is a pose matrix of the surgical instrument.

[0014] Optionally, the analyzing the image information according to the pre-trained image recognition model to determine whether the image information has a surgical instrument occlusion situation includes:

[0015] preprocessing the image information to obtain an input image;

[0016] inputting the input image into a convolutional neural network to obtain image features;

[0017] identifying the image features according to the image recognition model to obtain feature point information of an end of the surgical arm device;

[0018] determining a feature point reference sequence of the surgical instrument according to a type of the surgical instrument currently equipped by the surgical arm device;

[0019] comparing the feature point information and the feature point reference sequence to determine whether the surgical instrument is occluded.

[0020] Optionally, a database is established in a training process of the image recognition model; and the determining the feature point reference sequence of the surgical instrument according to the type of the surgical instrument currently used by the surgical instrument includes:

[0021] respectively classifying and training images of the surgical arm device carrying different surgical instruments to obtain feature point sequences corresponding to the surgical instruments;

[0022] merging a plurality of the feature point sequences to obtain an image feature database;

[0023] According to the current surgical instrument of the surgical arm device, the feature point reference sequence in the image feature database is selected.

[0024] Optionally, when the image information exists occlusion, generating a surgical instrument contour image according to the pose information and the image information comprises:

[0025] According to the pose information and the type of the surgical instrument, a feature point reference sequence corresponding to the pose information is determined.

[0026] A matching feature point set of the feature point information and the feature point reference sequence corresponding to the pose information is determined, an occlusion feature point set is determined according to the feature point set and the feature point reference sequence, and the contour image is generated according to the occlusion feature point set.

[0027] Optionally, the contour image is generated according to the occlusion feature point set comprises:

[0028] The current pose of the surgical instrument is determined according to the feature point reference sequence and the feature point information, the position information of the occluded region of the surgical instrument is determined according to the current pose and the occlusion feature point set, and the contour image is generated on the display screen according to the position information.

[0029] In a second aspect, the present application further provides a laparoscopic surgical robot surgical instrument display method, applied to the laparoscopic surgical robot surgical instrument display system in any one of the first aspect, the method comprises:

[0030] According to the geometric parameters of the surgical arm device, the pose information of the surgical instrument relative to the endoscope device is determined, and it is judged whether the surgical instrument is located in the visual range of the display screen according to the pose information.

[0031] When the surgical instrument is located in the visual range of the display screen, the image information of the endoscope device is acquired, and the image information is analyzed according to a pre-trained image recognition model to judge whether there is an occlusion of the surgical instrument in the image information.

[0032] When the image information exists occlusion, a surgical instrument contour image is generated according to the matching features of the pose information and the image information, and the surgical instrument contour image is displayed on the display screen to virtually display the occluded surgical instrument in the image information.

[0033] In a third aspect, the present application further provides a laparoscopic surgical robot, comprising the laparoscopic surgical robot surgical instrument display system in any one of the first aspect.

[0034] In a fourth aspect, the present application also provides a computer readable storage medium storing a computer program, which, when read and run by a processor, implements the endoscopic surgery robot surgery instrument display method according to the second aspect.

[0035] The present application has the beneficial effects that: by acquiring the geometric parameters of the surgical arm device, the geometric parameters are brought into the kinematics formula of the surgical arm device to obtain the pose of the current surgical arm device, and the position of the surgical instrument in the current state is compared with the range of the endoscope picture to determine that the surgical instrument is in the screen range, then according to the preset image recognition model, it is judged whether the image of the surgical instrument exists in the shielding state, and when the image is shielded, the surgical instrument shielding contour is displayed on the display screen; the judgment error caused by the clarity of the endoscope image is avoided, and the problem that the image shielding caused by the surgical instrument itself and / or human tissue and other factors in the endoscope image cannot clearly determine the position of the surgical instrument is solved, the timeliness of the operator in the operation process to obtain the position of all surgical instruments is effectively improved, and the operator's control over the current working condition of the surgical instrument is improved. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 The endoscopic surgery robot surgery instrument display system structure diagram described in the embodiments of the present application;

[0037] Figure 2 The flow chart of the endoscopic surgery robot surgery instrument display system described in the embodiments of the present application;

[0038] Figure 3 The endoscopic surgery robot surgery instrument display method described in the embodiments of the present application. DETAILED DESCRIPTION

[0039] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings. Although some embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms, and should not be interpreted as being limited to the embodiments described herein, on the contrary, these embodiments are provided to make the present application more thorough and complete. It should be understood that the drawings and embodiments of the present application are only for illustrative purposes, and should not be used to limit the protection scope of the present application.

[0040] The term "include" and variations thereof, as used in this document, means "to include, without limitation." The term "based on" means "based at least in part on." The term "one embodiment" means "at least one embodiment." The term "another embodiment" means "at least one additional embodiment." The term "some embodiments" means "at least some embodiments." The term "optional" means "optional in at least some embodiments." Related terms shall be construed accordingly. It should be noted that "a" or "an" entity as used herein means "one or more" entities. The terms "first," "second," and the like as used in this document do not imply a quantity or order, but rather are used to distinguish different entities.

[0041] It should be noted that the terms "one" and "a" as used herein mean "one or more" unless explicitly indicated to the contrary or indicated otherwise by context. The term "plurality" as used herein means "a number greater than one."

[0042] The names of the messages or information exchanged between the plurality of devices in the embodiments of the present application are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0043] As shown in Figure 1 The embodiment of the present application provides a kind of endoscopic surgery robot surgery class instrument display system, including surgical arm device, endoscope device, central processing device and display screen, the central processing device is connected with the surgical arm device, the endoscope device and the display screen respectively;The endoscope device and the display screen are connected;Surgical instrument is equipped at the end of the surgical arm device, and the central processing device is used to:

[0044] According to the geometric parameters of the surgical arm device, the pose information of the surgical instrument relative to the endoscope device is determined, and whether the surgical instrument is located in the visual range of the display screen is judged according to the pose information;

[0045] Specifically, the angle of each joint of surgical arm device and the geometric length of each member of surgical arm device are acquired, the spatial coordinate system of surgical instrument is established, and the spatial position of the corresponding end of surgical arm device under each arm length and angle of surgical arm device is determined based on the position of the origin of coordinate system, so as to determine the position and twist angle of the end position surgical instrument in the current spatial coordinate system and further determine the pose information, and further judge whether the surgical instrument falls into the field of view of endoscope under the current pose.

[0046] When the surgical instrument is located in the visual range of the display screen, the image information of the endoscope device is acquired;And the image information is analyzed according to the pre-trained image recognition model, to judge whether there is surgical instrument shielding condition in the image information;

[0047] Specifically, according to a pre-trained image recognition model, the image transmitted by the endoscope is recognized to determine the surgical instrument in the picture, and the image of the surgical instrument is further convolved to extract the feature of the surgical instrument image, and it is determined whether the feature is complete, and when the feature is partially incomplete, it is determined that the surgical instrument is currently in a shielding state.

[0048] When the image information is shielded, a surgical instrument contour image is generated according to the matching feature of the pose information and the image information, and the surgical instrument contour image is displayed on the display screen to virtually display the surgical instrument shielded in the image information.

[0049] Specifically, if it is detected that the instrument is in a shielding state, the relative position of the instrument end is calculated according to the instrument end joint angle of the surgical arm device, the spatial pose of the surgical instrument is determined, and the contour of the surgical instrument is projected into the display screen based on the horizontal plane projection display effect, wherein the shielded contour is completed according to the matching image and projected; then the virtual model is enhanced according to the need to add lighting effects, shadows, reflections, etc. to enhance the realism of the virtual model, and the projected contour and the endoscope image are integrated in layers.

[0050] In the embodiment, only the judgment and display process of the position of the surgical instrument of the surgical robot are shown, and it should be understood that, taking the above-mentioned single surgical instrument judgment embodiment as an example, when it is determined that the surgical instrument is located within the range of the display screen, the endoscope image is recognized and segmented to determine the position of the surgical instrument and the shielding condition and display the contour. For the surgical arm carrying multiple surgical instruments, the display method of each surgical instrument in the endoscope image will not be described again, and the above-mentioned acquisition, simulation, plane range determination and comparison process can be executed.

[0051] In the embodiment, by acquiring the geometric parameters of the surgical arm device, the current pose of the surgical arm device is obtained by bringing the geometric parameters into the kinematics formula of the surgical arm device, and the position of the surgical instrument in the current state is compared with the range of the endoscope image to determine whether the surgical instrument is within the range of the screen. Then, according to the preset image recognition model, it is determined whether the image of the surgical instrument exists in a shielding state, and the shielded contour of the surgical instrument is displayed on the display screen when the image is shielded; avoiding the judgment error caused by the clarity of the endoscope image, and also solving the problem that the image shielding caused by the surgical instrument itself and / or human tissue in the endoscope image cannot clearly determine the position of the surgical instrument, effectively improving the timeliness of the operator in obtaining the position of all surgical instruments during operation, and improving the operator's control over the current working condition of the surgical instrument.

[0052] Exemplarily, as Figure 2As shown, the display system has: S1, based on kinematics, judges whether the surgical instrument is located within the screen range, when the surgical instrument is located outside the screen range, generates a warning information and ends; when the surgical instrument is located within the screen range, S2 further judges based on the endoscope image whether the surgical instrument position is within the visual range of the screen; when the kinematics and image of the surgical instrument are both within the screen range, the surgical instrument image is displayed and the process ends; S3, when the surgical instrument is within the screen range, and the surgical instrument is invisible in the image, it is determined that the surgical instrument is blocked; S4, based on the mixed reality implementation of the instrument end model, the pose and contour of the surgical instrument are obtained, and the projection of the surgical instrument relative to the display screen is displayed in the display screen, and the process ends.

[0053] In an optional embodiment, the determination of the pose information of the surgical instrument relative to the endoscope device according to the geometric parameters of the surgical arm device, and the judgment of whether the surgical instrument is located within the visual range of the display screen according to the pose information, comprises:

[0054] Substitute the geometric parameters into the kinematics model of the surgical arm device to determine the vector information of the end of the surgical arm device relative to the endoscope device, and the vector information is used to determine the pose information and position information of the surgical instrument;

[0055] According to the relationship between the range corresponding to the vector information and the display range of the endoscope, it is judged whether the end of the surgical arm device is beyond the screen range.

[0056] Further, the geometric parameters include: link length, link offset, link torsion angle and link rotation angle; the kinematics equation includes:

[0057] T=f(ai,di,αi,θi);

[0058] Wherein: ai is the link length, di is the link offset, αi is the link torsion angle, θi is the link rotation angle, and T is the pose matrix of the surgical instrument.

[0059] Specifically, the geometric parameters of each component of the surgical arm device are obtained, including link length, link offset, link torsion angle, link rotation angle, etc., and the corresponding parameters are brought into the kinematics equation to obtain a 4x4 pose matrix of the surgical instrument relative to the endoscope lens, including a 3x3 pose matrix and a 3x1 position vector, the position vector includes the position relationship of the surgical instrument relative to the endoscope device, and the pose matrix includes the moment and torque of the surgical instrument relative to the endoscope device in the spatial direction.

[0060] In the embodiment, by combining the geometric parameter model and the kinematic equation, a composite matrix with a position vector and a pose matrix is obtained, the actual position and the torsion of the surgical instrument can be quickly determined, the recognition speed of the surgical instrument in the identification and judgment process is reduced, and the reliability of the position judgment of the surgical instrument is enhanced.

[0061] In an optional embodiment, the analyzing the image information according to the pre-trained image recognition model to determine whether the image information exists the occlusion of the surgical instrument includes:

[0062] preprocessing the image information to obtain an input image;

[0063] inputting the input image into a convolutional neural network to obtain image features;

[0064] identifying the image features according to the image recognition model to obtain feature point information of the end of the surgical arm device;

[0065] determining a feature point reference sequence of the surgical instrument according to the type of the current surgical instrument;

[0066] comparing the feature point information and the feature point reference sequence to determine whether the surgical instrument is occluded.

[0067] Further, a database is established in the training process of the image recognition model; the determining the feature point reference sequence of the surgical instrument according to the type of the current surgical instrument includes:

[0068] respectively classifying and training the images of the surgical arm device carrying different surgical instruments to obtain feature point sequences corresponding to the surgical instruments;

[0069] merging a plurality of the feature point sequences to obtain an image feature database;

[0070] selecting the feature point reference sequence in the image feature database according to the type of the surgical instrument currently equipped by the surgical arm device.

[0071] Specifically, in the training process of the image recognition model, images of surgical instruments of different types and different positions are respectively taken as samples for training, and finally a trained image recognition model is obtained, which has a feature classification database and classifies image features of different surgical instruments. In the application process, the corresponding feature database is matched according to the type of the surgical instrument currently held by the surgical robot. In the training stage, different surgical instrument pictures are respectively convolved and feature points are extracted to obtain a feature sequence composed of different feature points of the surgical instrument in different poses. In the application, the feature points of the current surgical instrument are matched according to the image obtained by the endoscope. When the image and the information in the database are successfully matched, the coincidence degree of the feature points is checked. When the missing proportion of the feature points exceeds the preset threshold, it is considered that the surgical instrument is currently completely or partially occluded.

[0072] In the embodiment, the surgical instrument type is matched with the corresponding database sequence, effectively reducing the operation amount in the image recognition process, reducing the misjudgment possibility caused by fuzzy data, improving the operation speed in the surgical instrument image recognition, and determining the occluded part of the surgical instrument in the current pose by image matching and determining the feature point coincidence degree. The preparation workload of supplementing the virtual contour is effectively reduced, and the display effect of the surgical instrument in the surgical instrument display is improved.

[0073] In an optional embodiment, when the image information is occluded, the surgical instrument contour image is generated according to the pose information and the image information, including:

[0074] According to the pose information and the type of the surgical instrument, a feature point reference sequence corresponding to the pose information is determined;

[0075] The matching feature point set of the feature point information and the feature point reference sequence corresponding to the pose information is determined, the occluded feature point set is determined according to the feature point set and the feature point reference sequence, and the contour image is generated according to the occluded feature point set.

[0076] Further, the contour image is generated according to the occluded feature point set, including:

[0077] The current pose of the surgical instrument is determined according to the feature point reference sequence and the feature point information, the position information of the occluded area of the surgical instrument is determined according to the current pose and the occluded feature point set, and the contour image is generated on the display screen according to the position information.

[0078] Specifically, when the surgical instrument is determined to exist an occlusion condition in the current pose of the surgical instrument, the missing part of the feature points of the surgical instrument is determined according to the matching of the feature points, so as to determine the current missing image of the surgical instrument and virtually display the corresponding image part and the contour. It should be noted that the contour acquisition method also includes the method mentioned in the foregoing embodiments, that is, virtually displaying the actual pose of the surgical arm device and projecting in the direction of the endoscope to obtain the contour. After obtaining the virtual contour information, the pixel points are arranged according to the contour in the current pose of the surgical instrument according to the actual pose state, and the pixel points are connected to obtain the contour image in the display screen.

[0079] In the embodiment, the image of the surgical instrument that is occluded and missing is determined, the contour of the image is simulated and displayed in the display screen in the form of pixel points to obtain the contour image, so that the display capability of the occluded image is effectively improved, and the risk of injury caused by the misoperation or misjudgment of the surgical instrument due to insufficient vision is reduced.

[0080] In a second aspect, the present application further provides a display method of a surgical instrument of a laparoscopic surgery robot, which is applied to the display system of the surgical instrument of the laparoscopic surgery robot as described in any one of the first aspect.

[0081] According to the geometric parameters of the surgical arm device, the pose information of the surgical instrument relative to the endoscope device is determined, and whether the surgical instrument is located in the visible range of the display screen is judged according to the pose information.

[0082] When the surgical instrument is located in the visible range of the display screen, the image information of the endoscope device is obtained, and the image information is analyzed according to a pre-trained image recognition model to judge whether the image information exists an occlusion condition of the surgical instrument.

[0083] When the image information exists an occlusion, a contour image of the surgical instrument is generated according to the matching features of the pose information and the image information, and the contour image of the surgical instrument is displayed on the display screen to virtually display the occluded surgical instrument in the image information.

[0084] In a third aspect, as shown in the first aspect, Figure 3 The present application further provides a laparoscopic surgery robot comprising the display system of the surgical instrument of the laparoscopic surgery robot as described in any one of the first aspect.

[0085] In a fourth aspect, the present application further provides a computer readable storage medium, which stores a computer program, and when the computer program is read and run by a processor, the display method of the surgical instrument of the laparoscopic surgery robot as described in the second aspect is realized.

[0086] The computer readable storage medium of the present application has the same advantages as the endoscopic surgery robot surgery instrument position determination method relative to the prior art, and will not be described here.

[0087] An electronic device that can be a server or a client of the present application will now be described, which is an example of a hardware device that can be applied to aspects of the present application. The electronic device is intended to represent various forms of digital electronic computer devices such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computer devices. The electronic device can also represent various forms of mobile devices such as personal digital assistants, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components, their connections, and their functions, as described herein, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0088] The electronic device includes a computing unit that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) or a computer program loaded into a random access memory (RAM) from a storage unit. In the RAM, various programs and data required for device operation can also be stored. The computing unit, the ROM, and the RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0089] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing relevant hardware through a computer program, and the program can be stored in a computer readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiment methods. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM), or a random access memory (RAM), etc. In this application, the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment of the present application. In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0090] Although the present application has been disclosed with reference to the above embodiments, the application is not limited to the above embodiments. It will be apparent to those skilled in the art that various changes and modifications can be made thereto without departing from the spirit and scope of the application, and it is intended to include all such changes and modifications as fall within the scope of the application.

Claims

1. A laparoscopic surgical robotic display system, comprising: The application relates to a surgical arm device, an endoscope device, a central processing device and a display screen, wherein the central processing device is connected in communication with the surgical arm device, the endoscope device and the display screen respectively; the endoscope device is connected in communication with the display screen; the surgical arm device is provided with a surgical instrument at the end thereof; and the central processing device is used for: determining pose information of the surgical instrument relative to the endoscope device according to geometric parameters of the surgical arm device, judging whether the surgical instrument is located within the visual range of the display screen according to the pose information; when the surgical instrument is located within the visual range of the display screen, acquiring image information of the endoscope device; and analyzing the image information according to a pre-trained image recognition model to judge whether the image information has a surgical instrument occlusion condition, including: pre-processing the image information to obtain an input image; inputting the input image into a convolutional neural network to obtain image features; identifying the image features according to the image recognition model to obtain feature point information of the surgical instrument; determining a feature point reference sequence of the surgical instrument according to the type of the surgical instrument currently equipped by the surgical arm device; comparing the feature point information and the feature point reference sequence to determine whether the surgical instrument is occluded; wherein a database is established in the training process of the image recognition model; the feature point reference sequence of the surgical instrument is determined according to the type of the surgical instrument currently equipped by the surgical arm device, including: respectively classifying and training images of the surgical arm device carrying different surgical instruments to obtain feature point sequences corresponding to the surgical instruments; merging a plurality of the feature point sequences to obtain an image feature database; and selecting the feature point reference sequence in the image feature database according to the surgical instrument of the surgical arm device; when the image information has occlusion, generating a surgical instrument contour image according to the pose information and the image information, and displaying the surgical instrument contour image on the display screen to virtually display the occluded surgical instrument in the image information.

2. The display system for a surgical robotic system of claim 1, wherein, The pose information of the surgical instrument relative to the endoscope device is determined according to geometric parameters of the surgical arm device, and whether the surgical instrument is located within the visual range of the display screen is judged according to the pose information, including: the geometric parameters are substituted into a kinematics model of the surgical arm device to determine vector information of the end of the surgical arm device relative to the endoscope device, and the vector information is used to determine pose information and position information of the surgical instrument; whether the end of the surgical arm device is beyond the screen range is judged according to the relationship between the range corresponding to the vector information and the acquisition range of the endoscope.

3. The display system for a surgical robotic system of claim 2, wherein, The geometric parameters include: link length, link offset, link torsion angle and link rotation angle; and the equation of the kinematics model includes: T=f(ai,di,ai,thetai). Wherein: ai is the length of the connecting rod, di is the offset of the connecting rod, αi is the torsion angle of the connecting rod, θi is the rotation angle of the connecting rod, and T is the pose matrix of the surgical instrument.

4. The display system for a surgical robotic system of claim 1, wherein, According to the pose information and the image information, a surgical instrument contour image is generated when the image information exists occlusion. According to the pose information and the surgical instrument category, a feature point reference sequence corresponding to the pose information is determined. The matching feature point set of the feature point information and the feature point reference sequence corresponding to the pose information is determined, the occlusion feature point set is determined according to the feature point set and the feature point reference sequence, and the contour image is generated according to the occlusion feature point set.

5. The display system of claim 4, wherein, The contour image is generated according to the occlusion feature point set, which includes: According to the feature point reference sequence and the feature point information, the current pose of the surgical instrument is determined, the position information of the occluded area of the surgical instrument is determined according to the current pose and the occlusion feature point set, and the contour image is generated on the display screen according to the position information.

6. A laparoscopic surgery robot display method characterized by comprising: The method is applied to the laparoscopic surgery robot display system as claimed in any one of claims 1-5, which includes: According to the geometric parameters of the surgical arm device, the pose information of the surgical instrument relative to the endoscope device is determined, and it is judged whether the surgical instrument is located in the visual range of the display screen according to the pose information; When the surgical instrument is located in the visual range of the display screen, the image information of the endoscope device is acquired, and the image information is analyzed according to the pre-trained image recognition model to judge whether there is occlusion of the surgical instrument in the image information; When the image information exists occlusion, a surgical instrument contour image is generated according to the matching feature of the pose information and the image information, and the surgical instrument contour image is displayed on the display screen to virtually display the occluded surgical instrument in the image information.

7. A surgical robotic system, comprising: The laparoscopic surgery robot display system as claimed in any one of claims 1-6 is included.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, which is read and run by the processor to realize the laparoscopic surgery robot display method as claimed in claim 7.

Images