4K laparoscope image device and image optimization method

Abstract

The invention relates to a 4K laparoscopic imaging device and an image optimization method, and belongs to the technical field of image processing. The device comprises a cold light source, a pneumoperitoneum machine and a photosensitive element, and further comprises an optical hard lens, a camera, a camera connecting line, a camera host, a signal connecting line and a rear end device. Low-delay and high-pixel transmission is met, image information is completely reserved, a host with high back-end performance can better complete image processing, and the final image quality is guaranteed; the problem of image information loss caused by image processing at the camera module end is reduced, and the problems of module temperature rise, component damage, influence on imaging quality and the like caused by image processing at the camera module end are solved. The method can be applied to the field of laparoscopic imaging.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and particularly to a 4K abdominal endoscope imaging device and an endoscope image optimization method. Background Art

[0002] An endoscope generally inserts a flexible insertion portion having an elongated shape into a subject to be examined, illuminates illumination light supplied from a light source device from the front end of the insertion portion, and captures an in-vivo image by receiving the reflected light of the illumination light with an imaging unit at the front end of the insertion portion.

[0003] The disclosed patent: Invention Name, Image Sensor Data Transmission Device of 3D Endoscope and 3D Endoscope, Publication Number: CN215383848U, Publication Date of Application: January 4, 2022. The device includes: a first image sensor module 102: converting an optical signal into a first electrical signal; a second image sensor module 103: converting an optical signal into a second electrical signal; an image sensor data transmission device 104 for receiving the first electrical signal and the second electrical signal, and outputting first image data corresponding to the first electrical signal to a data processing system, and outputting second image data corresponding to the second electrical signal to the data processing system; a data processing system 105 for receiving the first image data and the second image data, and after synthesizing the first image data and the second image data, outputting synthesized image data for 3D display; a data interface circuit 2015 is connected to a parallel-to-serial circuit to transmit data to the backend; The disadvantages of the above-disclosed patent technology are: Image processing is not performed simultaneously, and the first image and the second image are obtained sequentially; this system can only be used for the endoscope image system; Published Patent: Invention Title: A 3D Fluorescence Endoscope, Imaging Method, Device and Debugging Method, Publication Number: CN117398043A, Publication Date: January 16, 2024. Disclosed is a 3D fluorescence endoscope, imaging method, device and debugging method, belonging to the field of endoscopes. The endoscope includes an endoscope sleeve, an objective lens and a photosensitive element arranged in the endoscope sleeve. A beam splitting device is arranged between the objective lens and the photosensitive element. The beam splitting device divides the light beam passing through the objective lens into a left visible light beam, a left fluorescence beam, a right visible light beam and a right fluorescence beam that are incident on the photosensitive element. A left focusing lens that makes the optical paths of the left visible light beam and the left fluorescence beam consistent and a right focusing lens that makes the optical paths of the right visible light beam and the right fluorescence beam consistent are arranged between the beam splitting device and the photosensitive element. This endoscope can simultaneously image the four beams of light on the photosensitive element, with no time difference between the left and right images, and no time difference between the visible light image and the fluorescence image. The entire endoscope only uses one photosensitive element, and can achieve consistency in time and focal plane in a relatively small space, which is beneficial to the miniaturization and high-sensitivity display of the 3D fluorescence endoscope. The technical unknowns of the above-mentioned published patent are: There are a total of 4 photosensitive elements. The image obtained by the light reflected twice on the photosensitive element is the mirror image of the real object, and the direction is the same as the light directly passing through the dichroic mirror 21, which is convenient for image processing operations during imaging; this system can only be used for imaging endoscope systems; it does not involve image optimization and only processes image signals.

[0004] And the technical route of existing laparoscopes: One is the FPGA (Field Programmable Gate Array) solution, features: high parallelism, low latency, hardware reconfigurability; data is processed at the camera end, and an FPGA chip is used to implement ISP image processing. The disadvantage is that the algorithm for processing images needs to be burned on the FPGA main board, and if you want to change it after solidification, you need to rewrite the program on the chip. Application scenarios: high-realtime tasks, multi-modal imaging fusion; it is currently the main solution for 4K, 3D, and fluorescence.

[0005] The other is the SOC (System-on-Chip) solution, features: software-hardware collaboration, system integration; some do not have ISP functions, and some ISP functions are solidified on the chip. The disadvantage is that the underlying logic cannot be customized, resulting in limited flexibility, but the system integration degree is high.

[0006] When researching and developing a 4K laparoscope imaging device, the GPU solution is adopted. The data directly reaches the GPU, and the camera end does not process it. The GPU can obtain the most original image data; the GPU then completes ISP image processing. All image data can be implanted with AI algorithms, and the image algorithms can be updated in real time without having to rewrite. It is very necessary to optimize and improve the image quality and accuracy. Summary of the Invention

[0007] To overcome the deficiencies of existing medical endoscope imaging technologies, a 4K abdominal endoscope imaging device and an imaging optimization method are invented.

[0008] A 4K abdominal endoscope imaging device includes an endoscope cannula, a cold light source, a pneumoperitoneum machine, a photosensitive element, a front-end camera, a camera host, and a rear-end device; The cold light source includes: a cold light source touch screen, a cold light source module, a cold light source main board, a cold light source power supply, and a light guide beam; the cold light source main board is respectively connected to the cold light source touch screen, the cold light source module, and the cold light source power supply; the light guide beam is connected to the cold light source host; The pneumoperitoneum machine includes: a gas source management module, a flow control module, a pressure monitoring module, a pneumoperitoneum machine main board, a safety protection module, a power supply module, and an exhaust system; the pneumoperitoneum machine main board is respectively connected to the gas source management module, the flow control module, the pressure monitoring module, the safety protection module, the power supply module, and the exhaust system; The endoscope cannula integrates an optical lens imaging part and an optical fiber light transmission part; one end of the light guide beam is connected to the cold light source output port, and the other end of the light guide beam is arranged at the front end of the endoscope cannula; the inflation port of the pneumoperitoneum machine is arranged at the front end of the supporting instruments of the endoscope cannula; The front-end camera includes a camera, and the camera includes: an optical rigid endoscope, an optical bayonet, a filter, a photosensitive element, a gyroscope, a keypad, a camera main board, a serial board, and a camera cable; The gyroscope is connected to the keypad; the keypad is connected to the camera main board; The filter is arranged in front of the photosensitive element; the gyroscope is built into the camera keypad; The filter filters out interfering miscellaneous colors and transmits the filtered image to the photosensitive element; The photosensitive element transmits image data to the input interface of the camera host through the camera cable, and the gyroscope also transmits status information to the camera host. The camera host synthesizes the gyroscope information and processes the image; The camera host includes: a host main board, a video output board, a core board, a hard disk, a deserialization board, a host touch screen, a host power supply, a fan, a video expansion board, a network port, a USB port, an RS232 port, a signal output port, a display, a signal connection line, and a signal input interface; The camera connection cable refers to a GMSL3 data cable; The photosensitive element is used to collect the image obtained by the camera; The host main board described above includes: a core board; the core board includes a fusion algorithm module, and the fusion algorithm module includes an acquisition module, a DPC module, a BLC module, a WB module, a Demosaic module, a Gamma correct module, a CCM module, a Contrast / Brightness Enhancement module, an Edge Enhancement module, an output module, an AE Stat module, an AE Control module, a denoising algorithm module, and a total delay algorithm module; the main board is respectively connected to a touch screen, a host power supply, a fan, a video adapter board, a signal output interface or other host interfaces; The output module includes a video output board, a video adapter board, and a signal output terminal; the signal output terminal includes a signal output port, a network port, a USB port, and an RS232 port; The host main board is respectively connected to a host touch screen, a video adapter board, and a signal output terminal; the backend device includes: a display, or a remote receiving device; The hardware connection relationship of the 4K abdominal cavity endoscope imaging device is as follows: the lens of the optical rigid endoscope collects the image information of the target area, the optical rigid endoscope is adaptively connected to the optical bayonet, and the collected image information is optically transmitted to the filter, and the photosensitive element is connected to the camera main board; the gyroscope is connected to the keypad; the keypad is connected to the camera main board; the camera main board is connected to the serial board; the serial board is connected to the deserialization board through a camera cable, and the deserialization board is connected to the core board, and the core board is simultaneously connected to the hard disk and the video output board; the video output board is connected to the video expansion board, the video expansion board is connected to the signal output port, and the signal input interface is connected to the display; The imaging data transmission relationship of the described 4K laparoscope imaging device is as follows: The camera converts the optical signals collected by the photosensitive element into electrical signals and transmits them to the camera main board. The gyroscope collects the status information of the photosensitive element and outputs it to the camera main board. The serial board converts the data of the camera main board into serial data and transmits it at high speed to the deserialization board through the camera cable. The deserialization board converts the serial data into parallel data and inputs it into the imaging host; The core board of the imaging host reads the number of connected cameras and pixels through the fusion algorithm module; Match the built-in image processing algorithm module, and perform preprocessing through the exposure algorithm module, noise reduction algorithm module, and contrast enhancement algorithm module to obtain single or multiple images, which is State 1; Extract the image features of State 1, automatically select the single-image algorithm module or the multi-image fusion algorithm module, and obtain State 2 through the fusion algorithm module; Distribute the images in State 2, identify them as single images, and output the spectrally filtered images and normal images respectively after processing. The core board is connected to the image ISP processing module, the ISP processing module is connected to the host main board, the host main board outputs the image data to the signal output port, the signal output port outputs to the monitor, the monitor displays the optimized image, or transmits it to the remote device through the signal output port via wired or wireless network.

[0009] The described filter is used to filter stray light and only allows light with a wavelength of 400 - 650nm to enter the photosensitive element.

[0010] The photosensitive element is any one of CMOS, CCD, or infrared sensor chips.

[0011] The host is a touch screen or a key switch.

[0012] For the described total delay algorithm module, the delay time of the imaging data from the front-end device to the back-end device is within 50 - 100ms.

[0013] The described noise reduction algorithm module: X1. Pixel classification and detection This algorithm first classifies each pixel. If a pixel is judged as noise, noise reduction is performed on this pixel; otherwise, enhancement is performed. Noise point judgment conditions: The brightness value of the current pixel is less than or greater than the brightness values of all adjacent 8 points. The values of the 4 adjacent pixels above, below, left, and right are denoted as n1, n2, n3, n4. The pixel values of the 4 adjacent diagonal points are denoted as d1, d2, d3, d4. ; v ≤ (n1,n2,n3,n4,d1,d2,d3,d4) or v ≥ (n1,n2,n3,n3,d1,d2,d3,d4), the current point is a noise point. Otherwise, it is a non-noise point. If the current point is a noise point, go to step 5; if the current point is a non-noise point, go to step 2. X2. Calculate the high-frequency mask (mask) Calculate the sum of the differences between the current pixel point and its adjacent 8 points. dv = v * 8 – ((n1 + n2 + n3 + n4) + (d1 + d2 + d3 + d4)); X3. Calculate the high-frequency superposition component Calculate three enhanced superposition values: Z1. Direct superposition value: dv1 = dv * c1; Z2. Square root superposition value; dv2 = sqrt(|dv|) * c2; Z3. Cube root superposition value: dv3 = cbrt(dv) * c3; Z4. High-frequency component superposition v’ = v + dv1 + dv2 + dv3; Z5. Noise reduction processing The noise reduction value v’ of the pixel point = ((v * 8) + (n1 + n2 + n3 + n4) + (d1 + d2 + d3 + d4)) / 16.

[0014] The total delay algorithm module described above Y1. Total delay algorithm module of the 4K laparoscope imaging device The total delay of the 4K laparoscope imaging device is the superposition of the delays of each link: ;

[0015] Y2. Real-time scheduling algorithm EDF: Earliest Deadline First The task is completed within the deadline, and the scheduling condition is: ;

[0016] Among them, C i is the task calculation time, and T i is the task period; Y3. Parallel block processing Divide the image into K×K blocks for parallel processing. The total delay is: ;

[0017] Among them, among them, T merge is the result merging time; Y4. Display synchronization technology Adaptive synchronization (such as dynamic refresh rate) reduces display latency: ;

[0018] Among them, f content is the content frame rate, and T processing is the processing latency.

[0019] The total latency algorithm module described above, combined with GPU parallel computing and EDF real-time task processing, is jointly optimized to achieve the latency time of image data from the front-end device to the back-end device; the specific steps are as follows: (1) GPU acceleration deployment: Transplant the image processing algorithm to the GPU and optimize the memory access mode; Verify the performance improvement and ensure that the single-frame processing time ≤ 5ms; (2) EDF scheduling integration: Configure the EDF scheduler in the real-time operating system (Linux); Define the task parameters (C i , T i ) and test the scheduling feasibility; (3) Joint debugging: Monitor the latency from the front-end device to the back-end device and adjust the GPU resource reservation ratio; (4) Handle extreme situations (such as sudden high load) to ensure system robustness; (5) Clinical verification: Test the impact of latency on operations in a simulated environment.

[0020] The usage steps of the 4K laparoscope imaging device, i.e., the image optimization method, are as follows: (1) Check that the endoscope cannula, cold light source, insufflator, photosensitive element, camera front end, camera host, and back-end device are correctly connected; (2) Connect the camera host to a 4K monitor, and the 4K monitor is turned on; (3) The camera is connected to the camera host; (4) The camera is installed with a medical endoscope; (5) And confirm that the camera host is turned on.

[0021] The image processing method steps of the 4K laparoscope imaging device are as follows: Step 1: Set the filter in front of the photosensitive element to filter out stray light and avoid image interference; Step 2: To ensure the integrity and high pixel count of the image information, after the image sensor captures an image, the parallel data will be serially processed through a serial board to achieve high-speed transmission. Step 3: Adopt the GMSL3 transmission method to obtain a transmission rate of 6 Gbps. After sending the image to the deserialization board, convert the serial signal into a parallel signal and transmit the parallel signal to the core board. During this process, the gyroscope transmits the position information of the image sensor to the core board in real time to correct the image. Step 4: The core board's fusion algorithm module automatically identifies the number and pixel count of the connected cameras, automatically matches the built-in image processing algorithm module, and performs preprocessing such as exposure algorithm module, noise reduction algorithm module, total delay algorithm module, and contrast enhancement algorithm module, color restoration, image fusion, etc. to obtain single or multiple images as State 1. The method for automatically identifying the number and pixel count of cameras: Identify the connected sensors through the sensor driver, and obtain the corresponding resolution according to the identified sensors. Step 5: Extract the image features of State 1, automatically select a single-image algorithm module or a multi-image fusion algorithm module, and obtain State 2 through the fusion algorithm module. Step 6: Distribute the State 2 image, identify it as a single image or multiple images, and respectively output the spectrally filtered image and the normal image after processing. Step 7: The image ISP processing module continues to process the single image or multiple images. Step 8: The core board transmits the processed image signal to the signal output port, outputs the processed video signal to the display, and displays the camera image in real time. Step 9: According to the instructions issued by the serial port control, the video signal processed by the core board can be stored on the hard disk or transmitted through the network port.

[0022] The process for achieving low latency is as follows: GPU acceleration deployment: Transplant the image processing algorithm to the GPU and optimize the memory access mode.

[0023] Verify the performance improvement to ensure that the single-frame processing time ≤ 5 ms.

[0024] EDF scheduling integration: Configure the EDF scheduler in the real-time operating system (Linux).

[0025] Define the task parameters (C i , T i ) and test the scheduling feasibility.

[0026] Joint debugging: Monitor the latency from the front-end device to the back-end device and adjust the GPU resource reservation ratio.

[0027] Handle extreme situations (such as sudden high load) to ensure system robustness.

[0028] Clinical verification: Test the impact of latency on operations (such as the doctor's manipulation feedback time) in a simulated environment.

[0029] Goal: The latency from the front-end device to the back-end device ≤ 100 ms.

[0030] Definitions and explanations of terms in this patent: This patent is the result of continuous improvement and innovation based on the applicant's previously applied patent "A Medical Imaging Optimization Device" with the patent application number: 202510278642.6; it is a 4K laparoscope imaging device and an endoscope imaging optimization method.

[0031] The insufflator is a well-known technical special device for establishing and maintaining pneumoperitoneum in laparoscopic surgery. Its working principle is to perfuse medical CO2 gas into the patient's abdominal cavity through mechanical pressurized inflation, separating the abdominal wall from the viscera, thereby forming a surgical operation and visual field space and avoiding organ damage caused by puncture trocars. Gas source management module, gas source interface: Connect to a carbon dioxide (CO2) gas cylinder or a central gas supply system to ensure gas input. Flow control module, flow sensor: Real-time monitor the gas flow rate (unit: L / min), usually divided into high flow (initial inflation stage) and low flow (maintenance stage). Solenoid valve / proportional valve: Automatically adjust the gas flow according to the set value to ensure rapid establishment of pneumoperitoneum and maintain stable pressure. Pressure regulator: Reduce the pressure of the high-pressure gas source (such as a gas cylinder) to a safe range suitable for abdominal cavity use (usually 30 - 50 psi). Gas filtration device: Filter impurities or microorganisms in the gas to ensure gas sterility. Pressure monitoring module, pressure sensor: Real-time detect the pressure in the abdominal cavity (usually set at 12 - 15 mmHg) to avoid excessive pressure (causing complications) or too low pressure (affecting the surgical field). Pressure feedback system: Feed back the monitoring data to the control module to automatically adjust inflation or exhaust. Alarm system: Trigger an audible and visual alarm in case of abnormal pressure (such as leakage, overpressure) and suspend gas supply. Insufflator main board, microprocessor: The core control chip, processes sensor data and adjusts flow and pressure. User interface: Touch screen or button panel for medical staff to set parameters such as target pressure and flow rate. Safety protection module, overpressure protection valve: Mechanical or electronic redundant design, cut off the gas source or release excess gas in case of system failure. Gas leakage detection: Judge leakage through abnormal changes in flow and pressure and trigger an alarm. Temperature control: Some devices are equipped with a gas heating function to avoid discomfort caused by cold gas entering the abdominal cavity. Power supply module, main power adapter: Provide the power required for device operation. Exhaust system, exhaust valve: Quickly discharge the gas in the abdominal cavity at the end of the operation, usually controlled by a solenoid valve.

[0032] Optical rigid endoscope: Briefly called the lens, there is a type of lens used in endoscopes called the optical rigid endoscope. The optical rigid endoscope is a type of endoscope. Its characteristic is that the optical components are made of columnar glass, and the outer tube is of metal structure and cannot be bent. It mainly consists of three parts: a mechanical system, an optical system, and a light guide system.

[0033] Gyroscope: bmi088, to clarify the position information of the photosensitive element.

[0034] GMSL3: High-speed data transmission reaches 6 Gbps.

[0035] MIPI stands for Mobile Industry Processor Interface, developed by the MIPI Alliance. It is a high-performance, low-power, and low-cost serial communication interface. The purpose is to standardize the interfaces inside the device, such as the camera, display interface, radio frequency / baseband interface, etc., thereby reducing the complexity of device design and increasing design flexibility.

[0036] Image sensor: Used to collect images, with a pixel count reaching 4K.

[0037] Filter: Used to filter out stray light and ensure that natural light in the range of 450 - 650 nm enters the photosensitive element.

[0038] Chip used for the serial board: MAX96789.

[0039] Chip used for the deserialization board: MAX96752.

[0040] 4K medical endoscope: An imaging system in which the sensor collects image signals and outputs them in real time to a 4K monitor for medical staff to view during the operation.

[0041] Sensor input: The sensor collects image input into the isp pipeline.

[0042] DPC (Dead Pixel Correction): Image dead pixel detection: Detect the dead pixels existing in the sensor and correct these dead pixels. Dead pixels are the white dots in the output image in a completely black environment and the black dots in the output image in a highly bright environment.

[0043] BLC (Black Level Compensation): Black level compensation: From the characteristics of the sensor, the lowest output voltage of the sensor is the black level voltage. By calibrating the black level size, the influence of the black level on the image is eliminated.

[0044] WB (White Balance): White balance: Correct the color deviation that occurs in the sensor under different color temperatures. Calculate the RGB three-channel gain values through a white cardboard to complete the white balance correction.

[0045] Demosaic: Debayer the image captured by a single CMOS sensor which only contains one of the RGB colors, and restore the missing colors in the image through interpolation processing.

[0046] Gamma Correction: Also known as gamma correction or gamma non-linearity, it is a non-linear operation or inverse operation technology used to adjust image or video signals. It is used to encode and decode linear brightness or RGB values to match the non-linear characteristics of display devices. In addition, gamma correction can expand or compress the dynamic range of the image.

[0047] CCM (Color Correction Matrix): Due to the non-ideality of the sensor's spectral response and the different spectral distributions of ambient light sources, there will be significant color differences between the sensor image and the actual scene. By calculating the color correction matrix, the image color can be adjusted to make the image color closer to the actual color.

[0048] Contrast / Brightness Enhancement: By enhancing the contrast and brightness of the image, the brightness distribution of the image under non-uniform illumination becomes more uniform, and the image has a stronger sense of transparency.

[0049] Edge Enhancement: Use edge detection technology to extract the edges of the image, enhance the edges of the image, and improve the sharpness of the image.

[0050] Output: Convert the RGB image format to YUV420 for output.

[0051] AE Stat: Automatic exposure information statistics: Statistic information such as image brightness, variance, histogram, etc. to evaluate whether the current image is underexposed or overexposed.

[0052] AE Control: Evaluate the current exposure situation based on the exposure statistical information, calculate the exposure parameters for the next frame, and send the exposure parameters of the next frame back to the sensor to complete automatic exposure control.

[0053] Contrast Enhancement refers to adjusting the contrast of an image to make its details clearer and enhancing the visual effect of the image. Contrast refers to the degree of difference between the brightest and darkest parts of an image. By increasing this difference, the details of the image can be made more prominent.

[0054] Brightness Enhancement adjusts the brightness level of an image to make it brighter or darker. Brightness refers to the average light intensity of an image. By adjusting the brightness, the light and dark levels of the image can be changed, thus affecting the visual effect.

[0055] Automatic identification of the number of cameras and pixel implementation method: The connected sensors are identified through sensor drivers, and the corresponding resolution is obtained according to the identified sensors.

[0056] CCD: Charge Coupled Device, a semiconductor device used for image acquisition.

[0057] CMOS: Abbreviation for Complementary Metal Oxide Semiconductor; it is an integrated technology that fabricates components such as transistors, resistors, capacitors, and diodes on the same silicon wafer using standard processes. CMOS chips have advantages such as low power consumption, high integration, and strong anti-interference ability, and are widely used in microprocessors, digital signal processors, memories, and image sensors.

[0058] GMSL3: The third-generation Gigabit Multimedia Serial Link technology; High data transfer rate: GMSL3 supports a data transfer rate of up to 12 Gbps, which can meet the bandwidth requirements of high-definition cameras and other high-speed data transfer devices; Low latency: Achieves low-latency video data transfer to ensure the real-time performance of the system.

[0059] EDF: EDF (Earliest Deadline First) real-time task management in intelligent scheduling is a dynamic priority scheduling algorithm, which is widely used in scenarios that require strict task timeliness. EDF real-time task management has become one of the core technologies of intelligent scheduling through dynamic priority allocation and data-driven decision-making mechanisms.

[0060] Core board: A customized NVIDIA AGX Orin 64G is used as the core board to process the image information received by the camera.

[0061] Light guide beam: The light guide beam in a camera is a component used to transmit light. It is mainly composed of an optical fiber bundle and can efficiently transmit the light emitted by the light source to the object to be measured. In the imaging process, the light guide beam plays a crucial role. It ensures that the light can be projected onto the object surface along a predetermined path and forms a specific light spot or light strip pattern on the object surface.

[0062] Hardware Technical Specifications Table 1 Camera Host

[0063] Table 2 Cold Light Source Technical Specifications

[0064] The applicant has conducted tests, "4K Medical Endoscope Camera System Test Report" Conclusion

[0065] (1) The noise reduction performance meets the image quality requirements.

[0066] (2) The system delay (98.8 ms) meets the real-time surgical interaction requirements (≤100 ms).

[0067] Special Note: The applicant's requirement for system delay is within ≤100 ms. While the industry's general requirement is only ≤150 ms.

[0068] The significant progress and creative technical features of the 4K abdominal endoscope imaging device and image optimization method of the present invention are as follows: The technical route adopted: The implementation process of the 4K endoscope camera solution: The CMOS converts the collected image into a digital signal and transmits the parallel signal to the serial board; the gyroscope collects the status information of the CMOS and outputs it to the signal output port of the serial board; the serial board compiles the parallel signal into a serial signal and transmits it at high speed to the deserialization board, enabling high-speed transmission of more than ten meters; at the same time, the gyroscope also transmits the status information to the backend at high speed. The host synthesizes the gyroscope information and processes the image. Because the image information is completely retained during the entire image transmission, higher-quality images can be obtained. It mainly overcomes the problem of image information loss caused by image processing at the camera module end, and also overcomes the problems such as possible increase in module temperature, damage to components, and impact on imaging quality caused by image processing at the camera module end; The technical objectives achieved: The main purpose is to meet the requirements of low-latency and high-pixel transmission, and the image information is completely retained. The powerful host at the backend can better complete the image processing, thus ensuring the final image quality. The data of this patent directly reaches the GPU, and the camera end does not perform processing. The GPU can obtain the most original image data; then the GPU completes the ISP image processing, and all image data can be implanted with AI algorithms, and the image algorithms can be updated in real time without the need for reprogramming; adopting the backend processing method preserves the image information. Other patents perform image processing at the camera front end, which may lead to the loss of image information.

[0069] Achieved technical effects: The image data is transmitted to the display screen for real-time display and playback for medical staff to view. The image data can also be processed in the GPU chip processor, and the processed results can be superimposed on the optimized image data and transmitted to the display screen for real-time display; the optimized image data can also be copied to an external storage device, such as a USB flash drive, for backup or transmitted over the network to a remote location. Description of the Drawings

[0070] Figure 1 It is a schematic structural diagram of the present invention.

[0071] Figure 2 It is a schematic diagram of the data flow of the present invention.

[0072] Figure 3 It is a schematic diagram of the module structure in the present invention.

[0073] Figure 4 It is a schematic diagram of the module flowchart in the present invention.

[0074] Figure 5 It is a processing flowchart from image state 1 to image state 2 in the present invention.

[0075] Figure 6 It is an image on a medical display before noise reduction processing in the present invention.

[0076] Figure 7 It is an image on a medical display after noise reduction processing in the present invention.

[0077] Figure 8 It is a low-latency processing flowchart in the present invention.

[0078] Figure 9 It is a noise reduction processing flowchart in the present invention.

[0079] In the figure: cold light source 4-1, cold light source touch screen 4-1-1, cold light source module 4-1-2, cold light source main board 4-1-3, cold light source power supply 4-1-4, light guide beam 4-1-5, insufflator 4-2, gas source management module 4-2-1, flow control module 4-2-2, pressure monitoring module 4-2-3, insufflator main board 4-2-4, safety protection module 4-2-5, power module 4-2-6, exhaust system 4-2-7, 4K camera system 1, optical rigid endoscope 1-1, camera 1-2, optical bayonet 1-2-1, filter 1-2-2, photosensitive element 1-2-3, gyroscope 1-2-4, keypad 1-2-5, camera main board 1-2-6, serial board 1-2-7, camera cable 1-3, camera host 1-4, host main board 1-4-1, video output board 1-4-1-1, core board 1-4-1-2, hard disk 1-4-1-3, deserialization board 1-4-1-4, host touch screen 1-4-2, host power supply 1-4-3, video expansion board 1-4-4, network port 1-4-5, USB port 1-4-6, RS232 port 1-4-7, signal output port 1-4-8, display 1-5, signal connection cable 1-5-1, signal input interface 1-5-2. Specific embodiments

[0080] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

[0081] Embodiment 1

[0082] A 4K abdominal endoscope imaging device includes an endoscope cannula, a cold light source 4-1, an insufflation 4-2, a photosensitive element 1-2-3, a camera front end, a camera host, and a rear end device; The cold light source 4-1 includes: a cold light source touch screen 4-1-1, a cold light source module 4-1-2, a cold light source main board 4-1-3, a cold light source power supply 4-1-4, and a light guide beam 4-1-5; the cold light source main board 4-1-3 is respectively connected to the cold light source touch screen 4-1-1, the cold light source module 4-1-2, and the cold light source power supply 4-1-4, and the light guide beam 4-1-5 is connected to the cold light source 4-1; The insufflator 4-2 described above includes: a gas source management module 4-2-1, a flow control module 4-2-2, a pressure monitoring module 4-2-3, an insufflator main board 4-2-4, a safety protection module 4-2-5, a power supply module 4-2-6, an exhaust system 4-2-7. The insufflator main board is respectively connected to the gas source management module 4-2-1, the flow control module 4-2-2, the pressure monitoring module 4-2-3, the safety protection module 4-2-5, the power supply module 4-2-6, and the exhaust system 4-2-7; The endoscope cannula integrates an optical lens imaging part and an optical fiber light transmission part; One end of the light guide beam 4-1-5 is connected to the output port of the cold light source, and the other end of the light guide beam 4-1-5 is arranged at the front end of the endoscope cannula; The inflation port of the insufflator 4-2 is arranged at the front end of the accessory instrument of the endoscope cannula; The camera front end includes: an optical rigid endoscope 1-1, a camera 1-2, an optical bayonet 1-2-1, a filter 1-2-2, a photosensitive element 1-2-3, a gyroscope 1-2-4, a keypad 1-2-5, a camera main board 1-2-6, a serial board 1-2-7, a camera cable 1-3; The gyroscope 1-2-4 is connected to the keypad 1-2-5; The keypad 1-2-5 is connected to the camera main board 1-2-6; The filter 1-2-2 is arranged at the front end of the photosensitive element 1-2-3; The gyroscope 1-2-4 is built-in on the keypad 1-2-6; The filter 1-2-2 filters out interfering miscellaneous colors and transmits the filtered image to the photosensitive element 1-2-3; The photosensitive element 1-2-3 transmits image data to the input interface of the camera host 1-4 through the camera cable 1-3, and the gyroscope 1-2-4 also transmits status information to the camera host 1-4. The camera host 1-4 comprehensively processes the information of the gyroscope 1-2-4 and processes the image; The camera host 1-4 includes: a host main board 1-4-1, a video output board 1-4-1-1, a core board 1-4-1-2, a hard disk 1-4-1-3, a deserialization board 1-4-1-4, a host touch screen 1-4-2, a host power supply 1-4-3, a fan, a video expansion board 1-4-4, a network port 1-4-5, a USB port 1-4-6, an RS232 port 1-4-7, a signal output port 1-4-8, a display 1-5, a signal connection cable 1-5-1, a signal input interface 1-5-2; The camera connection cable 1-3 refers to a GMSL3 data cable; The photosensitive element 1-2-3 is used to collect the image obtained by the camera 1-2; The host main board 1-4-1 described above includes: a core board 1-4-1-2; the core board 1-4-1-2 includes a fusion algorithm module, and the fusion algorithm module includes an acquisition module, a DPC module, a BLC module, a WB module, a Demosaic module, a Gammacorrect module, a CCM module, a Contrast / Brightness Enhancement module, an Edge Enhancement module, an output module, an AE Stat module, an AE Control module, a denoising algorithm module, and a total delay algorithm module; the host main board 1-4-1 is respectively connected to a touch screen 1-4-2, a host power supply 1-4-3, a fan, a video adapter board 1-4-4, a signal output interface 1-5-2 or other host interfaces; The output module described above includes a video output board 1-4-1-1, a video adapter board 1-4-4, and signal output terminals; the signal output terminals include a signal output port 1-4-8, network ports 1-4-5 1-4-6, USB ports, and an RS232 port 1-4-7; The backend device includes: a monitor 1-5, or a remote receiving device; The hardware connection relationship of the 4K laparoscope imaging device is as follows: the lens of the optical rigid endoscope 1 / 1 collects the image information of the target area. The optical rigid endoscope 1 / 1 is adaptively connected to the optical bayonet 1-2-1, and the collected image information is optically transmitted to the filter 1-2-2. The photosensitive element 1-2-3 is connected to the camera main board 1-2-6; the gyroscope 1-2-4 is connected to the camera main board 1-2-6; the keypad 1-2-5 is connected to the camera main board 1-2-6; the camera main board 1-2-6 is connected to the serial board 1-2-7; the serial board 1-2-7 is connected to the deserialization board 1-4-1-4 through the camera cable 1-3, and the deserialization board 1-4-1-4 is connected to the core board 1-4-1-2. The core board 1-4-1-2 is simultaneously connected to the hard disk 1-4-1-3 and the video output board 1-4-1-1; the video output board 1-4-1-1 is connected to the video expansion board 1-4-4, and the video expansion board 1-4-4 is connected to the backend device through a wired or wireless network. The monitor 1-5 in the backend device displays the optimized medical image; The image data transmission relationship of the 4K laparoscope imaging device is as follows: the camera 1-2 converts the collected optical signal into an electrical signal through the photosensitive element 1-2-3 and transmits it to the camera main board 1-2-6. The gyroscope 1-2-4 collects the status information of the photosensitive element 1-2-3 and outputs it to the camera main board 1-2-6. The serial board 1-2-7 converts the data of the camera main board into serial data and transmits it to the deserialization board 1-4-1-4 at high speed through the camera cable 1-3. The deserialization board 1-4-1-4 converts the serial data into parallel data and inputs it into the camera host 1-4; The camera converts the collected image information into digital signals, and the camera main board 1-2-6 transmits the parallel signals to the serial board 1-2-7; the gyroscope 1-2-4 collects the status information of the photosensitive element 1-2-3 and outputs it to the signal input port in the serial board 1-2-7; the serial board compiles the parallel signals into serial signals and transmits them at high speed to the deserialization board 1-4-1-4; at the same time, the gyroscope 1-2-4 also transmits the status information to the input interface of the camera host 1-4, and the camera cable 1-3 transmits the video data to the input interface of the camera host 1-4; the camera host 1-4 synthesizes the information of the gyroscope 1-2-4 and processes the image; the fusion algorithm module in the core board 1-4-1-2 of the camera host 1-4 reads the number and pixel of the connected camera 1-2, automatically matches the built-in image processing algorithm module, and performs preprocessing through the exposure algorithm module, denoising algorithm module, and contrast enhancement algorithm module to obtain a single or multiple images as Status 1; extracts the image features of Status 1, automatically selects a single-image algorithm module or a multiple-image fusion algorithm module, and obtains Status 2 through the fusion algorithm module; distributes the Status 2 image, identifies it as a single-image, and outputs the spectrally filtered image and the normal image respectively after processing. The core board 1-4-1-2 of the camera host is connected to the image ISP processing module, the ISP processing module is connected to the host main board 1-4-1, the host main board 1-4-1 is connected to the video output board 1-4-1-1, the video output board 1-4-1-1 is connected to the video expansion board 1-4-4, the video expansion board 1-4-4 is connected to the host signal output port 1-4-8, and the host signal output port 1-4-8 is connected to the monitor 1-5, or is transmitted to the remote device through the signal output terminal interface by wire or wireless network, and the monitor 1-5 in the backend device displays the optimized image. The lens of the optical rigid endoscope 1 / 1 collects the image information of the target area. The optical rigid endoscope 1 / 1 is adaptively connected to the optical bayonet 1-2-1, and the collected image information is optically transmitted to the filter 1-2-2. The photosensitive element 1-2-3 is connected to the camera main board 1-2-6; the gyroscope 1-2-4 is connected to the camera main board 1-2-6; the keypad 1-2-5 is connected to the camera main board 1-2-6; the camera main board 1-2-6 is connected to the serial board 1-2-7; the serial board 1-2-7 is connected to the deserialization board 1-4-1-4 through the camera connection cable 1-3, and the deserialization board 1-4-1-4 is connected to the core board 1-4-1-2. The core board 1-4-1-2 is simultaneously connected to the hard disk 1-4-1-3 and the video output board 1-4-1-1; the video output board 1-4-1-1 is connected to the video adapter board 1-4-4, and the video adapter board 1-4-4 is connected to the backend device by wire or wireless network. The monitor 1-5 in the backend device displays the optimized medical image; The image data transmission relationship of the described 4K laparoscope imaging device is as follows: The camera converts the collected image information into digital signals, and the camera main board 1-2-6 transmits the parallel signals to the serial board 1-2-7; the gyroscope 1-2-4 collects the status information of the photosensitive element 1-2-3 and outputs it to the signal input port in the serial board 1-2-7; the serial board compiles the parallel signals into serial signals and transmits them at high speed to the deserialization board 1-4-1-4; at the same time, the gyroscope 1-2-4 also transmits the status information to the input interface of the camera host 1-4, and the camera connection line 1-3 transmits the image data to the input interface of the camera host 1-4. The camera host 1-4 synthesizes the information of the gyroscope 1-2-4 and processes the image; the fusion algorithm module set on the core board 1-4-1-2 of the camera host 1-4 automatically identifies the number and pixels of the connected cameras 1-2, automatically matches the built-in image processing algorithm module, and performs preprocessing through the exposure algorithm module, noise reduction algorithm module, total delay algorithm module, and contrast enhancement algorithm module to obtain a single or multiple images, which is regarded as State 1; extract the image features of State 1, automatically select the single-image algorithm module or the multi-image fusion algorithm module, and obtain State 2 through the fusion algorithm module; distribute the images in State 2, identify them as single images, and after processing, respectively output the spectrally filtered images and normal images. The core board 1-4-1-2 of the camera host is connected to the image ISP processing module, the ISP processing module is connected to the host main board 1-4-1, the host main board 1-4-1 is connected to the video output board 1-4-1-1, the video output board 1-4-1-1 is connected to the video adapter board 1-4-4, the video adapter board 1-4-4 is connected to the signal output port 1-4-8, the signal output port 1-4-8 is connected to the monitor 1-5, or is transmitted to the remote device through the signal output terminal interface by wire or wireless network, and the monitor 1-5 in the back-end device displays the optimized image.

[0083] The described camera 1-2 captures a single channel.

[0084] The described filter 1-2-2 is used to filter stray light and only allows light with a wavelength of 400-650nm to enter the photosensitive element 1-2-3.

[0085] The described photosensitive element 1-2-3 is a CMOS.

[0086] The described endoscope is a digestive tract endoscope.

[0087] The described camera 1-2 is adaptively connected to the endoscope.

[0088] The described image information includes the original medical image data of the medical endoscope.

[0089] The described camera 1-2 refers to reading the original medical image data with a resolution of 4K and a frame rate of 60 frames per channel.

[0090] The host touch screen 1-4-2 mentioned above is a touch screen switch.

[0091] For the total delay algorithm module mentioned above, the delay time of image data from the front-end device to the back-end device is within 50 - 100 ms.

[0092] For the total delay algorithm module mentioned above, the delay time of image data from the front-end device to the back-end device is within 50 - 100 ms.

[0093] The denoising algorithm module: X1. Pixel classification and detection This algorithm first classifies each pixel. If a pixel is determined to be noise, noise reduction is performed on this pixel; otherwise, enhancement is performed. Noise point judgment condition: The brightness value of the current pixel is less than or greater than the brightness values of all 8 adjacent points The values of the 4 adjacent pixels above, below, left, and right are denoted as n1, n2, n3, n4 The pixel values of the 4 adjacent diagonal points are denoted as d1, d2, d3, d4 ; v ≤ (n1, n2, n3, n4, d1, d2, d3, d4) or v ≥ (n1, n2, n3, n3, d1, d2, d3, d4), the current point is a noise point, otherwise it is a non-noise point, If the current point is a noise point, go to step 5; if the current point is a non-noise point, go to step 2; X2. Calculate the high-frequency mask (mask) Calculate the total difference between the current pixel point and the 8 adjacent points, dv = v * 8 – ((n1 + n2 + n3 + n4) + (d1 + d2 + d3 + d4)); X3. Calculate the high-frequency superposition component Calculate three enhanced superposition values: Z1. Direct superposition value: dv1 = dv * c1; Z2. Square root superposition value; dv2 = sqrt(|dv|) * c2; Z3. Cube root superposition value: dv3 = cbrt(dv) * c3; Z4. High-frequency component superposition v’ = v + dv1 + dv2 + dv3;; Z5. Noise reduction processing The noise reduction value v' of the pixel point = ((v * 8) + (n1 + n2 + n3 + n4) + (d1 + d2 + d3 + d4)) / 16.

[0094] The total delay algorithm module described: Y1. Total delay algorithm module of the 4K laparoscope imaging device The total delay of the 4K laparoscope imaging device is the superposition of the delays of each link: ;

[0095] Y2. Real-time scheduling algorithm EDF: Earliest Deadline First The task is completed within the deadline, and the scheduling condition is: ;

[0096] Among them, C i is the task calculation time, and T i is the task period; Y3. Parallel block processing The image is divided into K×K blocks for parallel processing, and the total delay is: ;

[0097] Among them, T merge is the result merging time; Y4. Display synchronization technology Adaptive synchronization (such as dynamic refresh rate) reduces display delay: ;

[0098] Among them, f content is the content frame rate, and T processing is the processing delay.

[0099] The total delay algorithm module described, combined with GPU parallel computing and EDF real-time task processing, is jointly optimized to achieve the delay time of image data from the front-end device to the back-end device; the specific steps are as follows: (1) GPU acceleration deployment: Transplant the image processing algorithm to the GPU and optimize the memory access mode; Verify the performance improvement and ensure that the single-frame processing time ≤ 5 ms; (2) EDF scheduling integration: Configure the EDF scheduler in the real-time operating system (Linux); Define the task parameters (C i , T i ) and test the scheduling feasibility; (3) Joint debugging: Monitor the latency from the front-end device to the back-end device and adjust the reserved GPU resource ratio; (4)Handle extreme situations (such as sudden high load) to ensure system robustness; (5)Clinical verification: Test the impact of latency on operations in a simulation environment.

[0100] The low-latency implementation process is as follows: GPU acceleration deployment: Transplant the image processing algorithm to the GPU and optimize the memory access pattern.

[0101] Verify the performance improvement and ensure that the single-frame processing time ≤ 5 ms.

[0102] EDF scheduling integration: Configure the EDF scheduler in a real-time operating system (Linux).

[0103] Define task parameters (C i 、T i ) and test the scheduling feasibility.

[0104] Joint debugging: Monitor the latency from the front-end device to the back-end device and adjust the reserved GPU resource ratio.

[0105] Handle extreme situations (such as sudden high load) to ensure system robustness.

[0106] Clinical verification: Test the impact of latency on operations (such as the doctor's manipulation feedback time) in a simulation environment.

[0107] Goal: The latency from the front-end device to the back-end device ≤ 100 ms.

[0108] Embodiment 2

[0109] The host touch screen 1-4-2 is a push-button switch.

[0110] The rest is the same as above.

[0111] The specific embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to the above embodiments, and various changes can be made without departing from the spirit of the present invention within the scope of knowledge possessed by those of ordinary skill in the art.

Claims

1. A 4K laparoscope imaging device, characterized in that: It includes an endoscope cannula, a cold light source, a pneumoperitoneum machine, a photosensitive element, a front camera, a camera host, and a rear-end device; The cold light source includes: a cold light source touch screen, a cold light source module, a cold light source main board, a cold light source power supply, and a light guide beam; the cold light source main board is respectively connected to the cold light source touch screen, the cold light source module, and the cold light source power supply; the light guide beam is connected to the cold light source host; The pneumoperitoneum machine includes: a gas source management module, a flow control module, a pressure monitoring module, a pneumoperitoneum machine main board, a safety protection module, a power supply module, and an exhaust system; the pneumoperitoneum machine main board is respectively connected to the gas source management module, the flow control module, the pressure monitoring module, the safety protection module, the power supply module, and the exhaust system; The endoscope cannula integrates an optical lens imaging part and an optical fiber light-passing part; one end of the light guide beam is connected to the output port of the cold light source, and the other end of the light guide beam is arranged at the front end of the endoscope cannula; the inflation port of the pneumoperitoneum machine is arranged at the front end of the supporting instrument of the endoscope cannula; The front camera includes a camera, and the camera includes: an optical rigid endoscope, an optical bayonet, a filter, a photosensitive element, a gyroscope, a keypad, a camera main board, a serial board, and a camera cable; The gyroscope is connected to the keypad; the keypad is connected to the camera main board; The filter is arranged in front of the photosensitive element; the gyroscope is built into the camera keypad; The filter filters out interfering miscellaneous colors and transmits the filtered image to the photosensitive element; The photosensitive element transmits image data to the input interface of the camera host through the camera cable, and the gyroscope also transmits status information to the camera host. The camera host synthesizes the gyroscope information and processes the image; The camera host includes: a host main board, a video output board, a core board, a hard disk, a deserialization board, a host touch screen, a host power supply, a fan, a video expansion board, a network port, a USB port, an RS232 port, a signal output port, a display, a signal connection line, and a signal input interface; The camera connection cable refers to a GMSL3 data cable; The photosensitive element is used to collect the image obtained by the camera; The core board includes a fusion algorithm module, and the fusion algorithm module includes an acquisition module, a DPC module, a BLC module, a WB module, a Demosaic module, a Gamma correct module, a CCM module, a Contrast / Brightness Enhancement module, an Edge Enhancement module, an output module, an AE Stat module, an AE Control module, a denoising algorithm module, and a total delay algorithm module; the host main board is respectively connected to the touch screen, the host power supply, the fan, the video transfer board, the signal output port, or other host interfaces; The output module includes a video output board, a video transfer board, and a signal output end; The main board is respectively connected to the host touch screen, the video transfer board, and the signal output end; the signal output end includes a signal output port, a network port, a USB port, and an RS232 port; The rear-end device includes: a display, or a remote device; The hardware connection relationship of the described 4K laparoscope imaging device is as follows: The lens of the optical rigid endoscope collects the image information of the target area. The optical rigid endoscope is adaptively connected to the optical bayonet, and the collected image information is optically transmitted to the filter. The photosensitive element is connected to the camera main board; the gyroscope is connected to the button board; the button board is connected to the camera main board; the camera main board is connected to the serial board; the serial board is connected to the deserialization board through the camera cable, and the deserialization board is connected to the core board. The core board is simultaneously connected to the hard disk and the video output board; the video output board is connected to the video expansion board, and the video expansion board is connected to the signal output port. The signal input interface is connected to the monitor; The image data transmission relationship of the described 4K laparoscope imaging device is as follows: The camera converts the collected optical signal into an electrical signal through the photosensitive element and transmits it to the camera main board. The gyroscope collects the status information of the photosensitive element and outputs it to the camera main board. The serial board converts the data of the camera main board into serial data and transmits it at high speed to the deserialization board through the camera cable. The deserialization board converts the serial data into parallel data and inputs it into the imaging host; The core board of the imaging host fusion algorithm module reads the number and pixels of the connected cameras; matches the built-in image processing algorithm module, and through the exposure algorithm module, noise reduction algorithm module, and contrast enhancement algorithm module for preprocessing to obtain a single or multiple images, as state 1; extracts the image features of state 1, automatically selects the single-image algorithm module or the multi-image fusion algorithm module, and obtains state 2 through the fusion algorithm module; distributes the state 2 image, identifies it as a single image, and after processing, outputs the spectrally filtered image and the normal image respectively. The core board is connected to the image ISP processing module, the ISP processing module is connected to the host main board, the host main board outputs the image data to the signal output port, the signal output port outputs to the monitor, and the monitor displays the optimized image or transmits it to the remote device through the signal output port via wired or wireless network.

2. The 4K laparoscope imaging device according to claim 1, wherein: The host touch screen is a touch screen switch or a button switch.

3. The 4K laparoscope imaging device according to claim 1, characterized in that: The color rendering index of the described cold light source ≥90; the correlated color temperature of the cold light source is 5200 - 6200K; the radiant flux ratio of red, green, and blue light of the cold light source (with the green radiant flux Φeg in the wavelength range of 515nm - 545nm as the reference) in the wavelength range of 630nm - 660nm, the ratio of the red radiant flux Φer to Φeg should be 0.67, and the tolerance is ±20; in the wavelength range of 435nm - 465nm, the ratio of the blue radiant flux Φeb to Φeg should be 1.3, and the tolerance is ±20%; the ratio of the radiant flux and luminous flux of the infrared cut-off performance product of the cold light source in the wavelength range of 300nm - 1700nm should not be greater than 6mW / lm.

4. The 4K laparoscope imaging device according to claim 1, wherein: The technical parameters of the described imaging host are: Signal-to-noise ratio (dB): The nominal value is 50 dB, the peak signal-to-noise ratio is 82 dB, the tolerance should be -20%, and the upper limit is not counted; Minimum illumination (Lux): ≤0.1 Lx; Image sensor ADC bit width: 12bit; Luminance response characteristic: The linear fitting coefficient is not less than 0.98; Spatial frequency response: When the SFR value of the imaging system is 50%, the corresponding nominal spatial frequency is 48 lp / (°); when the SFR value of the imaging system is 30%, the corresponding nominal spatial frequency is 57 lp / (°); the tolerance of the spatial frequency response shall be -20%, and the upper limit is not counted; Static image latitude: The nominal value of the static image latitude is 180, and the tolerance shall be -20%, and the upper limit is not counted; Horizontal resolution: The system horizontal resolution is 2000 lines, and the tolerance shall be -20%, and the upper limit is not counted; Field of view angle: The field of view angle is 20°, and the tolerance shall be within ±10%; Image pixels: 3840×2160; Overall machine noise: ≤45dB.

5. The image optimization method of the 4K laparoscope imaging device according to claim 1, characterized in that: The denoising algorithm module: Pixel classification detection This algorithm first classifies each pixel. If a pixel is judged as noise, noise reduction is performed on this pixel, otherwise enhancement is performed; Noise point judgment condition: The luminance value of the current pixel is less than or greater than the luminance values of all adjacent 8 points The values of the 4 adjacent pixels above, below, left, and right are denoted as n1, n2, n3, n4 The pixel values of the 4 adjacent diagonal points are denoted as d1, d2, d3, d4 ； v ≤ (n1, n2, n3, n4, d1, d2, d3, d4) or v ≥ (n1, n2, n3, n3, d1, d2, d3, d4), the current point is a noise point, Otherwise it is a non-noise point, If the current point is a noise point, go to step 5; if the current point is a non-noise point, go to step 2; Calculate the high-frequency mask (mask) Calculate the sum of the differences between the current pixel point and the adjacent 8 points, dv = v*8– ((n1 + n2 + n3 + n4) + (d1 + d2 + d3 + d4)); Calculate the high-frequency superposition component Calculate three enhancement superposition values: Z1. Direct superposition value: dv1 = dv* c1; Z2. Square root superposition value; dv2 = sqrt(|dv|) * c2; Z3. Cube root superposition value: dv3 = cbrt(dv) * c3; Z4. High-frequency component superposition v’ = v + dv1 + dv2 + dv3; Z5. Noise reduction processing The noise reduction value v’ of the pixel point = ((v*8) + (n1 + n2 + n3 + n4)+(d1 + d2 + d3 + d4)) / 16.

6. The image optimization method of the 4K laparoscope imaging device according to claim 1, characterized in that: The total delay algorithm module, The total delay algorithm module of the 4K laparoscope imaging device The total delay of the 4K laparoscope imaging device is the superposition of the delays of each link: ； [[ID=…]]Real-time scheduling algorithm EDF: Earliest Deadline First The task is completed within the deadline, and the scheduling condition is: ； Among them, C i is the task calculation time, and T i is the task period; Parallel block processing The image is divided into K×K blocks for parallel processing, and the total delay is: ； Among them, T merge is the result merging time; Display synchronization technology Adaptive synchronization (such as dynamic refresh rate) reduces display delay: ； Among them, f content is the content frame rate, and T processing is the processing delay.

7. The image optimization method of the 4K laparoscope imaging device according to claim 6, characterized in that: For the total delay algorithm module, the delay time of image data from the front-end device to the back-end device is ≤ 100 ms.

8. The image optimization method of the 4K laparoscope imaging device according to claim 6, characterized in that: The total delay algorithm module is combined with GPU parallel computing and EDF real-time task processing for joint optimization to achieve the delay time of image data from the front-end device to the back-end device; the specific steps are as follows: (1) GPU acceleration deployment: Transplant the image processing algorithm to the GPU and optimize the memory access mode; Verify the performance improvement and ensure that the single-frame processing time is ≤ 5 ms; (2) EDF scheduling integration: Configure the EDF scheduler in the real-time operating system (Linux); Define task parameters (C i , T i ) and test the scheduling feasibility; (3) Joint debugging: Monitor the delay from the front-end device to the back-end device and adjust the reserved ratio of GPU resources; (4) Handle extreme situations (such as sudden high load) to ensure the robustness of the system; (5) Clinical verification: Test the impact of delay on operation in a simulated environment.

Images