A method and system for real-time wireless transmission of endoscope images

The matrix block repair algorithm, which utilizes 5G wireless transmission and GPU parallel computing, solves the problem of bad pixels in wireless transmission, achieving efficient 4K image repair and long battery life. It supports image playback and improves the reliability of endoscopic surgery and its teaching applications.

CN118842891BActive Publication Date: 2024-11-26XIAN HONGHUI HOSPITAL
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411320034.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-23
Publication Date
2024-11-26
Estimated Expiration
2044-09-23

AI Technical Summary

Technical Problem

In spinal endoscopic surgery, wirelessly transmitted images with built-in rechargeable batteries are susceptible to environmental interference, which can lead to dead pixels, affecting image quality and potentially misleading doctors' judgments.

Method used

Employing 5G wireless transmission and GPU parallel computing, and using matrix segmentation and pixel value repair algorithms, it repairs bad pixels in real time and outputs high-definition 4K images.

Benefits of technology

It achieves stable output of 24 repaired 4K images within 40ms, ensuring image quality, supporting long battery life and image playback function, and enhancing the reliability of surgery and teaching value.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118842891B_ABST
    Figure CN118842891B_ABST
Patent Text Reader

Abstract

The present invention discloses a real-time wireless transmission method and system for endoscope images, and relates to the field of image communication technology, including a data acquisition terminal and a data display terminal, etc. The matrix partitioning and parallel operation method is adopted, so that the operation time of bad pixel repair is short, the time consumption is less than 40ms, and 24 repaired images can be stably output per second; the image resolution displayed by the display module is 3840×2160, that is, 4K high-definition image quality, which can assist doctors in making accurate judgments and operations; the two detachable rechargeable lithium batteries set in the mobile power module, only one of which is working at any time, meet the long battery life requirements, and increase safety. When the first rechargeable lithium battery is exhausted, the second rechargeable lithium battery is immediately enabled to avoid the risk of power failure; the display module supports the image data stored in the image storage module and the video storage module, and the setting of the Thunderbolt 4 interface can further export and retain these image data for teaching purposes.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image communication technology, and in particular to a real-time wireless transmission method and system for endoscopic images. Background Art

[0002] In the field of spinal endoscopic surgery, an endoscope with a built-in rechargeable battery and no external cables is used. It is simple, portable, and highly flexible, which will be more conducive to the surgeon's on-the-spot performance. In this scenario, the image data collected by the endoscope in real time can only be transmitted to the data receiving and display end through wireless transmission. During the wireless transmission process, the transmission signal is easily interfered by environmental factors, which can cause some data loss. This is reflected in the final displayed image as some abnormal pixels with obvious differences in color from the surrounding pixels. These abnormal pixels are usually called bad pixels.

[0003] Bad pixels can lead to a decline in image quality and even affect doctors' judgment in some special cases. Therefore, there is an urgent need for a real-time wireless image transmission method and a corresponding system that can eliminate bad pixels. Summary of the invention

[0004] The purpose of the present invention is to provide a wireless transmission method for real-time wireless transmission of high-definition 4K quality endoscopic images with a bad pixel repair function, and a system based on this method that can display endoscopic images in real time and can review and save image data.

[0005] In view of the above technical problems, the technical solution adopted by the present invention is: a method for real-time wireless transmission of endoscopic images, comprising the following steps:

[0006] Step 1: Obtain image data with a resolution of 3840×2160 in real time through 5G wireless transmission. The pixel channels of each image are composed of three channels: R, G, and B. The pixel value matrix of the R channel is R, the pixel value matrix of the G channel is G, and the pixel value matrix of the B channel is B;

[0007] Step 2: Divide the three pixel value matrices of each image into 16×9 blocks, and the dimension of each block matrix is ​​240×240;

[0008] Step 3: Use GPU to parallelize the change rate matrix K of the 16×9 block matrices in the matrix R. For any block matrix change rate matrix K, we have

[0009] ,

[0010] In the formula, is the first i Line j The element value of the column; is the first block matrix corresponding to the pixel value matrix R i Line j The element value of the column; For Adjacent element values; when When located at the four corners of the corresponding block matrix, n Take 3; when When located on the four sides of the corresponding block matrix, n Take 6; when When it is located at a position other than the corner of the corresponding block matrix, n Take 8;

[0011] Step 4: Definition The pixels corresponding to all the bad pixels are considered bad pixels. , and The values ​​of are replaced and repaired, and the replacement values ​​are , and ;

[0012] Step 5: Splice the repaired block matrix into the pixel value matrix before the block. The pixel value matrix of the R channel after the repair is , the pixel value matrix of the repaired G channel is , the pixel value matrix of the repaired B channel is ; Through the pixel value matrix , and Output the repaired image.

[0013] Furthermore, in step 4,

[0014] ,

[0015] In the formula, r is the red adjustment factor.

[0016] Furthermore, in step 4,

[0017] ,

[0018] In the formula, is the green adjustment factor; For Adjacent element values; is the first block matrix corresponding to the pixel value matrix G i Line j The element value of the column; when When located at the four corners of the corresponding block matrix, n Take 3; when When located on the four sides of the corresponding block matrix, n Take 6; when When it is located at a position other than the corner of the corresponding block matrix, n Take 8.

[0019] Furthermore, in step 4,

[0020] ,

[0021] In the formula, b is the blue adjustment factor; For Adjacent element values; is the first block matrix corresponding to the pixel value matrix B i Line j The element value of the column; when When located at the four corners of the corresponding block matrix, n Take 3; when When located on the four sides of the corresponding block matrix, n Take 6; when When it is located at a position other than the corner of the corresponding block matrix, n Take 8.

[0022] Furthermore, 24 restored images are output per second.

[0023] The present invention also proposes a system that applies the above-mentioned real-time wireless transmission method of endoscopic images, including a data acquisition end and a data display end; the data acquisition end is composed of a 5G wireless communication module, an image acquisition module, a lighting module and a mobile power module; the data display end is composed of a data receiving module, an image processing module, a display module, an image storage module and a video storage module; the mobile power module provides power for the 5G wireless communication module, the image acquisition module and the lighting module; the lighting module provides lighting for the image acquisition module, and the 5G wireless communication module is used to wirelessly transmit the image data collected by the image acquisition module to the data receiving module; the image processing module is used to process the image data received by the data receiving module and output the image data in JPEG or PNG format; the display module is used to display the image processed by the image processing module; the image storage module is used to store the image processed by the image processing module; the image processing module is also used to convert the image stored in the image storage module into a video in MP4 or AVI format and store it in the video storage module; the image storage module and the video storage module are both equipped with a solid-state hard disk of not less than 1TB; the display module can also call to view the image data stored in the image storage module and the video storage module.

[0024] Furthermore, a 4K camera is provided in the image acquisition module.

[0025] Furthermore, two detachable rechargeable lithium batteries are arranged in the mobile power module, the capacity of each rechargeable lithium battery is not less than 10000 mAh, and when any one rechargeable lithium battery is working, the other rechargeable lithium battery is not working.

[0026] Furthermore, the image processing module is provided with a CPU, a GPU and a memory of no less than 8GB.

[0027] Furthermore, the display module adopts an OLED capacitive touch screen and has a built-in embedded operating system; the display module also includes multiple Type-C interfaces.

[0028] Compared with the prior art, the present invention has the following advantages: (1) the use of matrix partitioning and parallel operation shortens the operation time of bad pixel repair to less than 40 ms, and can stably output 24 repaired images per second; (2) the image resolution displayed by the display module is 3840×2160, that is, 4K high-definition image quality, which can assist doctors in making accurate judgments and operations; (3) of the two detachable rechargeable lithium batteries set in the mobile power module, only one is working at any time, which not only meets the long battery life requirements but also greatly increases safety. When the first rechargeable lithium battery is exhausted, the second rechargeable lithium battery is immediately enabled to avoid the risk of power failure; (4) the display module supports the review of image data stored in the image storage module and the video storage module. The setting of the Type-C interface can further export and retain these image data for teaching purposes. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the structural principle of the real-time wireless transmission system of endoscopic images of the present invention.

[0030] In the figure: 1-5G wireless communication module; 2-image acquisition module; 3-lighting module; 4-mobile power module; 5-data receiving module; 6-image processing module; 7-display module; 8-image storage module; 9-video storage module. DETAILED DESCRIPTION

[0031] The technical solution of the present invention is further illustrated below in conjunction with the accompanying drawings and through specific implementation methods, wherein the accompanying drawings are only used for exemplary descriptions, representing only schematic diagrams rather than actual drawings, and cannot be understood as limitations on this patent; in order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the accompanying drawings may be omitted.

[0032] like Figure 1As shown, a real-time wireless transmission system for endoscopic images includes a data acquisition end and a data display end; the data acquisition end is composed of a 5G wireless communication module 1, an image acquisition module 2, a lighting module 3 and a mobile power module 4; the data display end is composed of a data receiving module 5, an image processing module 6, a display module 7, an image storage module 8 and a video storage module 9; two detachable rechargeable lithium batteries are arranged in the mobile power module 4, each of which has a capacity of 10000mAh, and when any rechargeable lithium battery is working, the other rechargeable lithium battery is not working; the mobile power module 4 provides power for the 5G wireless communication module 1, the image acquisition module 2 and the lighting module 3; the lighting module 3 provides lighting for the image acquisition module 2, and a 4K camera is arranged in the image acquisition module 2, and the 5G wireless communication module 1 is used to wirelessly transmit the image data collected by the image acquisition module 2 to the data receiving module 5; the image processing module 6 is used to process the data received by the data receiving module 5 The image processing module 6 is provided with a CPU, a GPU and a 16GB memory; the display module 7 is used to display the image processed by the image processing module 6; the display module 7 adopts an OLED capacitive touch screen and has a built-in embedded operating system; the image storage module 8 is used to store the image processed by the image processing module 6; the image processing module 6 is also used to convert the image stored in the image storage module 8 into a video in MP4 or AVI format and store it in the video storage module 9; the image storage module 8 and the video storage module 9 are both built-in with a solid-state hard drive with a capacity of 1TB; the display module 7 can also call to view the image data stored in the image storage module 8 and the video storage module 9; the display module 7 also includes two Type-C interfaces. In this embodiment, the Type-C interface adopts a Thunderbolt 4 interface; the image data stored in the image storage module 8 and the video storage module 9 can be exported through the Thunderbolt 4 interface.

[0033] The method for real-time wireless transmission of endoscopic images based on the above embodiment includes the following steps:

[0034] Step 1: The image data with a resolution of 3840×2160 collected by the image acquisition module 2 is wirelessly transmitted to the data receiving module 5 in real time through the 5G wireless communication module 1. The image acquisition module 2 collects 24 images per second. The pixel channels of each image are composed of three channels: R, G, and B. The pixel value matrix of the R channel is R, the pixel value matrix of the G channel is G, and the pixel value matrix of the B channel is B;

[0035] Step 2: The image processing module 6 reads the image data received by the data receiving module 5 in sequence, divides the three pixel value matrices of each image into blocks, and uniformly divides them into 16×9 blocks, and the dimension of each block matrix is ​​240×240;

[0036] Step 3: The image processing module 6 uses the built-in GPU to parallelly calculate the change rate matrix K of the 16×9 block matrices in the matrix R. For the change rate matrix K of any block matrix,

[0037] ,

[0038] In the formula, is the first i Line j The element value of the column; is the first block matrix corresponding to the pixel value matrix R i Line j The element value of the column; For Adjacent element values; when When located at the four corners of the corresponding block matrix, n Take 3; when When located on the four sides of the corresponding block matrix, n Take 6; when When it is located at a position other than the corner of the corresponding block matrix, n Take 8;

[0039] Step 4: Definition The pixels corresponding to all the bad pixels are considered bad pixels. , and The values ​​of are replaced and repaired, and the replacement values ​​are , and ;

[0040] Step 5: Splice the repaired block matrix into the pixel value matrix before the block. The pixel value matrix of the R channel after the repair is , the pixel value matrix of the repaired G channel is , the pixel value matrix of the repaired B channel is ; Image processing module 6 through the pixel value matrix , and The restored image is output to the display module 7.

[0041] The display module 7 displays the restored final images in sequence according to the order in which the data is received. It can display 24 restored images per second, thus achieving a dynamic video effect of 24 frames per second.

[0042] While the display module 7 displays the image, the image storage module 8 stores the images processed by the image processing module 6 in sequence; after the operation is completed, the image processing module 6 converts the images stored in the image storage module 8 into videos in MP4 or AVI format and stores them in the video storage module 9; the display module 7 calls the image data stored in the image storage module 8 and the video storage module 9 in real time through the image processing module 6, and when necessary, it can also export the image data stored in the image storage module 8 and the video storage module 9 through the Thunderbolt 4 interface on the display module 7.

[0043] In step 4 above,

[0044] ,

[0045] In the formula, r is the red adjustment factor, r The value range of The rate of change does not exceed 5%; since the element values ​​in the pixel value matrix R are large, 5% is an acceptable percentage threshold.

[0046] In step 4 above,

[0047] ,

[0048] In the formula, is the green adjustment factor, The value range of The rate of change does not exceed 15%; since the element values ​​in the pixel value matrix G are small, 15% is an acceptable percentage threshold; For Adjacent element values; is the first block matrix corresponding to the pixel value matrix G i Line j The element value of the column; when When located at the four corners of the corresponding block matrix, n Take 3; when When located on the four sides of the corresponding block matrix, n Take 6; when When it is located at a position other than the corner of the corresponding block matrix, n Take 8.

[0049] In step 4 above,

[0050] ,

[0051] In the formula, b is the blue adjustment factor, b The value range of The rate of change does not exceed 12%; since the element values ​​in the pixel value matrix B are small, 12% is an acceptable percentage threshold; For Adjacent element values; is the first block matrix corresponding to the pixel value matrix B i Line j The element value of the column; when When located at the four corners of the corresponding block matrix, n Take 3; when When located on the four sides of the corresponding block matrix, n Take 6; when When it is located at a position other than the corner of the corresponding block matrix, n Take 8.

[0052] The operator can manually adjust the display module 7 during use. r , g , b To obtain the best restoration quality, by default, r , g , b The value of is 1.

Claims

1. A method for real-time wireless transmission of endoscopic images, characterized in that: Step 1: Obtain image data with a resolution of 3840×2160 in real time through 5G wireless transmission. The pixel channels of each image are composed of three channels: R, G, and B. The pixel value matrix of the R channel is R, the pixel value matrix of the G channel is G, and the pixel value matrix of the B channel is B; Step 2: Divide the three pixel value matrices of each image into 16×9 blocks, and the dimension of each block matrix is ​​240×240; Step 3: Use GPU to parallelize the change rate matrix K of the 16×9 block matrices in the matrix R. For any block matrix change rate matrix K, we have , In the formula, is the first i Line j The element value of the column; is the first block matrix corresponding to the pixel value matrix R i Line j The element value of the column; For Adjacent element values; when When located at the four corners of the corresponding block matrix, n Take 3; when When located on the four sides of the corresponding block matrix, n Take 6; when When it is located at a position other than the corner of the corresponding block matrix, n Take 8; Step 4: Definition The pixels corresponding to all the bad pixels are considered bad pixels. , and Replace the value of is the first block matrix corresponding to the pixel value matrix G i Line j The element value of the column, is the first block matrix corresponding to the pixel value matrix B i Line j The element value of the column; the replacement values ​​are , and ; Step 5: Splice the repaired block matrix into the pixel value matrix before the block. The pixel value matrix of the R channel after the repair is , the pixel value matrix of the repaired G channel is , the pixel value matrix of the repaired B channel is ; Through the pixel value matrix , and Output the repaired image.

2. A method for real-time wireless transmission of endoscopic images as claimed in claim 1, characterized in that: In step four, , In the formula, r is the red adjustment factor.

3. A method for real-time wireless transmission of endoscopic images as claimed in claim 2, characterized in that: In step four, , In the formula, is the green adjustment factor; For Adjacent element values; when When located at the four corners of the corresponding block matrix, n Take 3; when When located on the four sides of the corresponding block matrix, n Take 6; when When it is located at a position other than the corner of the corresponding block matrix, n Take 8.

4. A method for real-time wireless transmission of endoscopic images as claimed in claim 3, characterized in that: In step four, , In the formula, b is the blue adjustment factor; For Adjacent element values; when When located at the four corners of the corresponding block matrix, n Take 3; when When located on the four sides of the corresponding block matrix, n Take 6; when When it is located at a position other than the corner of the corresponding block matrix, n Take 8.

5. A method for real-time wireless transmission of endoscopic images as claimed in claim 4, characterized in that: Outputs 24 restored images per second.

6. A real-time wireless transmission system for endoscopic images using the real-time wireless transmission method for endoscopic images according to claim 5, comprising a data acquisition terminal and a data display terminal, characterized in that: The data acquisition end is composed of a 5G wireless communication module (1), an image acquisition module (2), a lighting module (3) and a mobile power module (4); the data display end is composed of a data receiving module (5), an image processing module (6), a display module (7), an image storage module (8) and a video storage module (9); the mobile power module (4) provides power to the 5G wireless communication module (1), the image acquisition module (2) and the lighting module (3); the lighting module (3) provides lighting to the image acquisition module (2); the 5G wireless communication module (1) is used to wirelessly transmit the image data acquired by the image acquisition module (2) to the data receiving module (5); the image processing module (6) is used to The image data received by the data receiving module (5) is processed and the image data in JPEG or PNG format is output; the display module (7) is used to display the image processed by the image processing module (6); the image storage module (8) is used to store the image processed by the image processing module (6); the image processing module (6) is also used to convert the image stored in the image storage module (8) into a video in MP4 or AVI format and store it in the video storage module (9); the image storage module (8) and the video storage module (9) are both built with a solid state hard disk of not less than 1 TB; the display module (7) can also call and view the image data stored in the image storage module (8) and the video storage module (9).

7. A real-time wireless transmission system for endoscope images as claimed in claim 6, characterized in that: A 4K camera is arranged in the image acquisition module (2).

8. The real-time wireless transmission system for endoscope images according to claim 7, characterized in that: Two detachable rechargeable lithium batteries are arranged in the mobile power module (4), the capacity of each rechargeable lithium battery is not less than 10000 mAh, and when any one rechargeable lithium battery is working, the other rechargeable lithium battery is not working.

9. A real-time wireless transmission system for endoscope images as claimed in claim 8, characterized in that: The image processing module (6) is provided with a CPU, a GPU and a memory of no less than 8 GB.

10. The real-time wireless transmission system for endoscope images according to claim 9, characterized in that: The display module (7) adopts an OLED capacitive touch screen and has a built-in embedded operating system; the display module (7) also includes a plurality of Type-C interfaces.