Endoscope visual image processing system and surgical robot

Abstract

The utility model relates to the field of medical instruments, and discloses an endoscope visual image processing system and a surgical robot. The endoscope visual image processing system comprises an endoscope and a parallel processing circuit. The endoscope comprises an image shooting device used for shooting multiple frames of continuous images; the parallel processing circuit comprises an image acquisition circuit, an image transmission circuit, an image decoding circuit and an image processing circuit; the image acquisition circuit acquires an image from an image shooting device, the image transmission circuit uploads the acquired image to the image decoding circuit, the image decoding circuit decodes the received image, and the image processing circuit processes the decoded image and sends the processed image to the display circuit. The image acquisition circuit, the image transmission circuit, the image decoding circuit and the image processing circuit carry out continuous parallel processing on multiple frames of continuous images. Through the design of a plurality of circuits capable of processing multiple frames of continuous images in parallel, the image processing time can be shortened, and the endoscope visual image processing system with low visual delay is realized.

Description

Technical Field

[0001] The present disclosure relates to the field of medical devices, and in particular to an endoscopic visual image processing system and a surgical robot. Background Art

[0002] The visual image processing system is an important component of the surgical robot. The existing visual image processing system has a long processing time and a large visual delay, which makes it difficult to meet the high-performance requirements of the surgical robot system for visual delay in clinical operations. Utility Model Content

[0003] In some embodiments, an endoscope visual image processing system includes:

[0004] An endoscope, comprising an image capturing device, the image capturing device being used to capture a plurality of frames of continuous images;

[0005] A parallel processing circuit, the parallel processing circuit includes: an image acquisition circuit, which is communicatively connected to an image shooting device, and the image acquisition circuit acquires an image from the image shooting device; an image transmission circuit, which is communicatively connected to the image acquisition circuit; an image decoding circuit, which is communicatively connected to the image transmission circuit, and the image transmission circuit uploads the acquired image to the image decoding circuit, and the image decoding circuit decodes the received image; and an image processing circuit, which is communicatively connected to the image decoding circuit, and the image processing circuit performs image processing on the decoded image. The image acquisition circuit, the image transmission circuit, the image decoding circuit and the image processing circuit perform continuous and parallel processing on multiple frames of continuous images.

[0006] In some embodiments, the parallel processing circuit also includes a display processing circuit, which is communicatively connected to the image processing circuit; the endoscope visual image processing system also includes a display device, which is communicatively connected to the display processing circuit; the display processing circuit displays the image processed on the display device for each frame of the image.

[0007] In some embodiments, the multiple frames of continuous images include N frames of continuous images, and the parallel processing circuit performs continuous and parallel processing on the N frames of continuous images, where N is a positive integer greater than or equal to 2.

[0008] In some embodiments, the parallel processing circuit further includes an adjustment circuit, which adjusts the number N of image frames processed in parallel based on the complexity of image processing performed on each frame of image.

[0009] In some embodiments, the image processing includes one or more of noise reduction processing, contrast processing, brightness processing, saturation processing, sharpening processing, and distortion correction processing.

[0010] In some embodiments, the processing time of the parallel processing circuit for each frame of image is less than 90 ms.

[0011] In some embodiments, the image capturing device includes a left-eye camera and a right-eye camera, the multiple frames of continuous images include multiple frames of left-eye continuous images and multiple frames of right-eye continuous images, and the parallel processing circuit performs independent and continuous parallel processing on the multiple frames of left-eye continuous images and the multiple frames of right-eye continuous images in parallel.

[0012] In some embodiments, the display processing circuit further synchronizes the processed left-eye continuous images and right-eye continuous images and displays them on the stereoscopic display.

[0013] In some embodiments, the display processing circuit further displays the left-eye continuous images or the right-eye continuous images after image processing on a two-dimensional display.

[0014] In some embodiments, a surgical robot includes:

[0015] at least one robotic arm; and

[0016] An endoscopic visual image processing system as described in any embodiment of the present disclosure, wherein the endoscope of the endoscopic visual image processing system is detachably disposed at the end of one of the at least one robotic arm.

[0017] Some embodiments of the present disclosure have one or more of the following beneficial effects: The endoscopic visual image processing system of the present disclosure is designed with multiple circuits including an image acquisition circuit, an image transmission circuit, an image decoding circuit, an image processing circuit, etc. that can perform parallel image processing. By processing multiple frames of continuous images in parallel through multiple circuits, the image processing time can be shortened, an endoscopic visual image processing system with low visual delay can be realized, and the flexibility of system design can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the description of the embodiments of the present disclosure. The drawings described below only illustrate some embodiments of the present disclosure. For ordinary technicians in this field, other embodiments can be obtained based on the contents of the embodiments of the present disclosure and these drawings without creative work.

[0019] Figure 1 A structural block diagram of an endoscope visual image processing system according to some embodiments of the present disclosure is shown;

[0020] Figure 2 A block diagram showing a process of continuous parallel processing of multiple frames of images performed by an endoscopic visual image processing system according to some embodiments of the present disclosure;

[0021] Figure 3A structural block diagram of a surgical robot according to some embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0022] In order to make the technical problems solved by the present disclosure, the technical solutions adopted and the technical effects achieved more clearly, the technical solutions of the embodiments of the present disclosure will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only exemplary embodiments of the present disclosure, rather than all embodiments.

[0023] In the description of the present disclosure, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present disclosure and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present disclosure. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In the description of the present disclosure, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", and "coupled" should be understood in a broad sense, for example, it can be a fixed connection or a detachable connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection of two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present disclosure can be understood according to the specific circumstances. In the present disclosure, the end close to the operator (e.g., doctor) is defined as the proximal end, near part, or rear end, and the end close to the surgical patient is defined as the distal end, far end, or front end, or front part. Those skilled in the art will appreciate that the embodiments of the present disclosure may be used for medical instruments or surgical robots, and may also be used for other non-medical devices.

[0024] Figure 1 FIG. 1 shows a structural block diagram of an endoscope visual image processing system 1000 according to some embodiments of the present disclosure. Figure 2 FIG. 1 is a block diagram showing a process of continuous parallel processing of multiple frames of images performed by an endoscope visual image processing system 1000 according to some embodiments of the present disclosure. Figure 1 As shown, the endoscope visual image processing system 1000 may include an endoscope 110 and a parallel processing circuit 120. The image acquisition circuit 120, the image transmission circuit 130, the image decoding circuit 140 and the image processing circuit 150.

[0025] The endoscope 110 may include an image capturing device 112, which is used to capture multiple frames of continuous images. For example, the image capturing device 112 may include a binocular camera. The parallel processing circuit 120 includes an image acquisition circuit 121, an image transmission circuit 122, an image decoding circuit 123, and an image processing circuit 124. The image acquisition circuit 121 is communicatively connected to the image capturing device 112, the image transmission circuit 122 is communicatively connected to the image acquisition circuit 121, the image decoding circuit 123 is communicatively connected to the image transmission circuit 122, and the image processing circuit 124 is communicatively connected to the image decoding circuit 123, for example, via a USB or bus connection. The image acquisition circuit 121, the image transmission circuit 122, the image decoding circuit 123, and the image processing circuit 124 perform continuous and parallel processing on multiple frames of continuous images. For each frame of image, the image acquisition circuit 121 acquires the image from the image shooting device 112, the image transmission circuit 122 uploads the image acquired by the image acquisition circuit 121 to the image decoding circuit 123, the image decoding circuit 123 decodes the received image, and the image processing circuit 124 performs image processing on the decoded image. To simplify the description, Figure 2 The process block diagram of continuously and parallelly processing 5 frames of images is shown in FIG. The above is only an example and is not limited thereto. Figure 2 As shown, the processing process for each frame of image may include multiple processing boxes, such as acquiring an image, uploading an image, decoding an image, and processing an image, wherein the image acquisition circuit 121 performs image acquisition, the image transmission circuit 122 performs image uploading, the image decoding circuit 123 performs image decoding, and the image processing circuit 124 performs image processing. It should be understood that at any processing box where the parallel processing circuit 120 processes the first frame of image (for example, at the image uploading processing box of the first frame of image), the parallel processing circuit 120 may start processing the second frame of image. Similarly, at any processing box where the parallel processing circuit 120 processes the second frame of image (for example, at the image uploading processing box of the second frame of image), the parallel processing circuit 120 may start processing the third frame of image, and so on, to achieve continuous parallel processing of multiple frames of continuous images. By processing multiple frames of continuous images in parallel through the parallel processing circuit 120 (e.g., the image acquisition circuit 121, the image transmission circuit 122, the image decoding circuit 123, and the image processing circuit 124), the image processing time can be shortened, and an endoscopic visual image processing system with low visual delay can be realized. It should be understood that the image capture device 112 can compress the captured multiple frames of continuous images, which can facilitate the image acquisition circuit 121 to acquire images from the image capture device 112 and upload them to the image decoding circuit 123 through the image transmission circuit 122, so as to save image transmission time.

[0026] In some embodiments, Figure 1As shown, the parallel processing circuit 120 may further include a display processing circuit 125, and the endoscope visual image processing system 1000 may further include a display device. The display processing circuit 125 is communicatively connected to the image processing circuit 124, and the display device is communicatively connected to the display processing circuit 125. The display processing circuit 125 is used to display the image processed on the display device for each frame of image. For example, the display device may include a stereo display 130 or a two-dimensional (2D) display 140. The display processing circuit 125 may send display data to the display device to display the image processed on the display device, such as a video captured by an endoscope. By displaying the image on the display device, it is convenient for the operator to observe the shooting field of view in real time. As Figure 2 As shown, the processing of each frame of image may also include display processing, and the display processing circuit 125 performs the display processing.

[0027] In some embodiments, Figure 2 As shown, the multiple frames of continuous images include N frames of continuous images, and the parallel processing circuit 120 is used to perform continuous parallel processing on the N frames of continuous images, where N is a positive integer greater than or equal to 2. For example, when N is equal to 5, the parallel processing circuit 120 can simultaneously perform parallel processing on 5 frames of images. In this way, the throughput of the parallel processing circuit 120 can be increased and the delay in the image processing process can be reduced. It should be understood that the above is only an example, and N can also be 2, 3, 4 or other positive integers. In some embodiments, the parallel processing circuit 120 may also include an adjustment circuit 126, and the adjustment circuit 126 adjusts the number of image frames N processed in parallel based on the complexity of the image processing performed on each frame of the image. The above is only an example and is not limited to this.

[0028] Existing endoscope visual image processing systems are expensive for image processing, have large visual delays, and have poor design flexibility. The endoscope visual image processing system 1000 disclosed herein realizes parallel processing of images by designing multiple circuits capable of parallel processing (e.g., image acquisition circuit 121, image transmission circuit 122, image decoding circuit 123, image processing circuit 124, display processing circuit 125), thereby saving system costs, shortening the delay of visual image processing, and improving the flexibility of system design.

[0029] In some embodiments, image processing may include various image processing techniques, such as noise reduction processing, contrast processing, brightness processing, saturation processing, sharpening processing, and distortion correction processing. It should be understood that one or more of the image processing steps can be selected based on the actual image processing operation requirements and the algorithms or requirements involved in the image processing process. For example, for occasions with low image processing operation requirements, the image processing steps can be reduced, and for occasions with high image processing operation requirements, the image processing steps can be increased to increase the adaptability of the endoscope visual image processing system.

[0030] In some embodiments, the processing time of each frame of image by the parallel processing circuit 120 may be less than 90 ms. In this way, the performance requirements in actual use, such as the visual delay requirements of clinical operation of endoscopic surgery system, can be met.

[0031] In some embodiments, the image capture device may include a left-eye camera and a right-eye camera, and the multiple frames of continuous images may include multiple frames of left-eye continuous images and multiple frames of right-eye continuous images. The parallel processing circuit 120 may perform independent continuous parallel processing on the multiple frames of left-eye continuous images and the multiple frames of right-eye continuous images in parallel. Processing the multiple frames of left-eye continuous images and the multiple frames of right-eye continuous images in parallel can further improve the image processing capability of the parallel processing circuit 120 and reduce processing delays.

[0032] In some embodiments, the display processing circuit 125 also synchronizes the image-processed left-eye continuous images and right-eye continuous images, and displays them on the stereoscopic display 130. It should be understood that the image acquisition circuit 121, the image transmission circuit 122, the image decoding circuit 123 and the image processing circuit 124 can continuously and parallelly process multiple frames of left-eye continuous images, and at the same time, can continuously and parallelly process multiple frames of right-eye continuous images. After the multiple frames of left-eye continuous images and the multiple frames of right-eye continuous images are processed respectively, the display processing circuit 125 can synchronize the image-processed left-eye continuous images and right-eye continuous images using appropriate technology, such as "Triple Buffering" technology, and display them on the stereoscopic display 130. In some embodiments, the display processing circuit 125 also displays the image-processed left-eye continuous images or right-eye continuous images on the 2D display 140. For example, the display processing circuit 125 can select the image-processed left-eye continuous images to be displayed on the 2D display 140, or the display processing circuit 125 can also select the image-processed right-eye continuous images to be displayed on the 2D display 140. It should be understood that the display processing circuit 125 can display the image after image processing on the stereoscopic display 130 and the 2D display 140 at the same time.

[0033] Those skilled in the art can understand that various circuits involved in the present disclosure, such as image acquisition circuit, image transmission circuit, image decoding circuit, image processing circuit, display processing circuit, adjustment circuit, etc., can be implemented by various suitable hardware including, for example, dedicated integrated circuits, general integrated circuits, etc. The specific hardware structure used to implement the image acquisition circuit, image transmission circuit, image decoding circuit, image processing circuit, display processing circuit, and adjustment circuit can be implemented by various technologies, such as analog circuits, digital circuits, configurable circuits, or dedicated circuits, etc.

[0034] Figure 3 FIG. 1 is a block diagram of a surgical robot 100 according to some embodiments of the present disclosure. Figure 3 As shown, the present disclosure also provides a surgical robot 100, comprising at least one mechanical arm 10 and an endoscopic visual image processing system 1000 in any embodiment of the present disclosure. The endoscope 110 of the endoscopic visual image processing system 1000 is detachably disposed at the end of one of the at least one mechanical arm 10.

[0035] In some embodiments, at least one robot arm 10 may include multiple robot arms. Each robot arm may include a multi-section arm body and multiple joints connecting the multi-section arm body, and the multi-section arm body includes a terminal arm. The endoscope 110 is detachably arranged on the terminal arm of one of the robot arms, and one or more surgical instruments may be arranged on the terminal arms of the remaining one or more robot arms. In some embodiments, the surgical robot 100 may include a robot arm 10. The robot arm 10 may include a multi-section arm body and multiple joints connecting the multi-section arm body, the multi-section arm body includes a terminal arm, and the terminal arm includes multiple tool mounting positions, the endoscope 110 is detachably arranged on one of the tool mounting positions, and one or more surgical instruments may be arranged on the remaining one or more tool mounting positions.

[0036] The endoscope 110 may include an endoscope arm 111 and an image capturing device 112, such as a binocular camera, disposed at the distal end of the endoscope arm 111. During operation, the surgical robot 100 controls at least one mechanical arm 10 to move so that the end of at least one mechanical arm 10 moves to an appropriate posture (e.g., a target posture). The operator may control the movement of the endoscope arm 111 so that the image capturing device 112 of the endoscope 110 is located at the optimal shooting field of view, so that the image capturing device 112 can capture the scene in the surgical field of view in real time. The operator may perform surgical operations such as cutting, clamping, electrocoagulation, electrocuting, suturing, etc., based on the captured images, using surgical instruments. By processing multiple frames of continuous images (e.g., video streams) in parallel through a parallel processing circuit 120 (e.g., an image acquisition circuit 121, an image transmission circuit 122, an image decoding circuit 123, an image processing circuit 124, and a display processing circuit 125), the image processing time may be shortened, the visual delay during the image processing process may be reduced, and the surgical quality may be improved.

[0037] Note that the above are only exemplary embodiments of the present disclosure and the technical principles used. Those skilled in the art will understand that the present disclosure is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present disclosure. Therefore, although the present disclosure is described in more detail through the above embodiments, the present disclosure is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present disclosure, and the scope of the present disclosure is determined by the scope of the appended claims.

Claims

1. An endoscope visual image processing system, characterized in that: include: An endoscope, comprising an image capturing device, wherein the image capturing device captures a plurality of frames of continuous images; A parallel processing circuit, the parallel processing circuit comprising: An image acquisition circuit, connected to the image capture device for communication, and acquiring an image from the image capture device; An image transmission circuit, communicatively connected to the image acquisition circuit; an image decoding circuit, which is in communication with the image transmission circuit, the image transmission circuit uploads the acquired image to the image decoding circuit, and the image decoding circuit decodes the received image; and The image processing circuit is communicatively connected with the image decoding circuit, and the image processing circuit performs image processing on the decoded image. The image acquisition circuit, the image transmission circuit, the image decoding circuit and the image processing circuit perform continuous and parallel processing on the multiple frames of continuous images.

2. The endoscope visual image processing system according to claim 1, characterized in that: The parallel processing circuit further includes a display processing circuit, and the display processing circuit is communicatively connected with the image processing circuit; The endoscope visual image processing system further includes a display device, the display device being communicatively connected to the display processing circuit; The display processing circuit displays the processed image on the display device for each frame of image.

3. The endoscope visual image processing system according to claim 1, characterized in that: The multiple frames of continuous images include N frames of continuous images, and the parallel processing circuit performs continuous and parallel processing on the N frames of continuous images, wherein N is a positive integer greater than or equal to 2.

4. The endoscope visual image processing system according to claim 3, characterized in that: The parallel processing circuit further includes an adjustment circuit, which adjusts the number N of image frames processed in parallel based on the complexity of image processing performed on each frame of image.

5. The endoscope visual image processing system according to claim 1, characterized in that: The image processing includes one or more of noise reduction processing, contrast processing, brightness processing, saturation processing, sharpening processing, and distortion correction processing.

6. The endoscope visual image processing system according to claim 1, characterized in that: The parallel processing circuit takes less than 90 ms to process each frame of image.

7. The endoscope visual image processing system according to claim 2, characterized in that: The image capture device includes a left-eye camera and a right-eye camera, the multiple frames of continuous images include multiple frames of left-eye continuous images and multiple frames of right-eye continuous images, and the parallel processing circuit performs independent and continuous parallel processing on the multiple frames of left-eye continuous images and the multiple frames of right-eye continuous images in parallel.

8. The endoscope visual image processing system according to claim 7, characterized in that: The display processing circuit also synchronizes the processed left-eye continuous images and right-eye continuous images, and displays them on the stereoscopic display.

9. The endoscope visual image processing system according to claim 7, characterized in that: The display processing circuit further displays the left-eye continuous images or the right-eye continuous images after image processing on a two-dimensional display.

10. A surgical robot, characterized in that: include: at least one robotic arm; as well as An endoscopic visual image processing system as described in any one of claims 1-9, wherein the endoscope of the endoscopic visual image processing system is detachably disposed at the end of one of the at least one robotic arm.

Images