Image processing method, device and computer readable storage medium

Abstract

The application discloses an image processing method and device and a computer readable storage medium, wherein the method comprises the following steps: acquiring a plurality of to-be-processed polarization images corresponding to a plurality of polarization angles of a polarization light image sensor based on polarization information pixel data collected by the polarization light image sensor; determining a best polarization state image and a worst polarization state image based on the to-be-processed polarization images; and determining a target clear image based on the best polarization state image and the worst polarization state image. The application realizes the function that a polarization light endoscope device can clearly image in an underwater environment (blood water, turbidity, tissue crumbs, fog), so that when blood water, tissue crumbs, fog and the like appear in a surgical environment, the clear processing of the polarization information pixel data solves the problem that the output image of the polarization light endoscope device is not clear.

Description

Technical Field

[0001] This invention relates to the field of endoscopy technology, and more particularly to an image processing method, apparatus, and computer-readable storage medium. Background Technology

[0002] An endoscope is a medical electro-optical instrument that can be inserted into the cavities of the human body and organs for direct observation, diagnosis, and treatment. It uses a tiny optical lens to optically image the objects inside the cavity to be observed through a miniature objective lens imaging system. The optical image is then sent to an image processing host, and finally the processed image is output on the display screen for doctors to observe and diagnose.

[0003] Currently, endoscopes mainly include ultra-high-definition 4K endoscopes, three-chip endoscopes, fluorescence endoscopes, narrow-band light endoscopes, and 3D endoscopes. These endoscopes have excellent functions and performance in their respective fields. Among them, 4K endoscopes have ultra-high resolution and excellent detail reproduction; three-chip endoscopes have three image sensors that each collect a monochromatic light signal from RGB, obtaining the RGB color value of each pixel, with high color reproduction accuracy and better performance than ordinary single-chip endoscopes; fluorescence endoscopes are a new imaging technology that uses fluorescent molecules to image in a special spectral environment. The principle is to label tumor cells with fluorescent agents to increase the contrast between diseased tissue and normal tissue, which can lead to earlier detection of cancer and tumors; narrowband endoscopes use filters to filter out the broadband spectrum of red, blue and green light waves emitted by the endoscope light source, leaving only the narrowband spectrum for precise observation of the morphology of the digestive tract mucosal epithelium, such as the epithelial glandular concave structure, and can also observe the morphology of the epithelial vascular network; 3D endoscopes display camera scenes with depth information, allowing doctors to perform surgery and treatment on lesions more precisely during surgery.

[0004] Although the above-mentioned endoscopes have good functions and performance, they cannot output clear images in surgical environments with blood, turbid water, tissue debris, or fog. Unclear images can interfere with the surgeon's work. Sometimes, the interfering environment (blood, turbid water, tissue debris, fog) must be cleared before the surgery can be performed, resulting in lower surgical efficiency.

[0005] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] The main objective of this invention is to provide an image processing method, apparatus, and computer-readable storage medium, which aims to solve the technical problem that polarized light endoscope devices cannot output clear images in underwater environments (blood, turbidity, tissue debris, fog).

[0007] To achieve the above objectives, the present invention provides an image processing method applied to a polarized light endoscope device including a camera, wherein the camera is equipped with a polarized light image sensor; the image processing method includes the following steps:

[0008] Based on the polarization information pixel data collected by the polarization light image sensor, obtain the polarization image to be processed corresponding to multiple polarization angles of the polarization light image sensor;

[0009] Based on the polarization image to be processed, determine the best polarization state image and the worst polarization state image;

[0010] Based on the best and worst polarization state images, a clear image of the target is determined.

[0011] Further, the step of determining the best polarization state image and the worst polarization state image based on the polarization image to be processed includes:

[0012] Based on the polarization image to be processed, the total light intensity vector, the linearly polarized light component in the horizontal axis direction, and the linearly polarized light component in the 45° direction are determined.

[0013] Based on the total light intensity vector, the linearly polarized light component along the horizontal axis, and the linearly polarized light component along the 45° direction, the optimal polarization state image and the worst polarization state image are determined respectively.

[0014] Further, the polarization image to be processed includes a polarization image with a 0° polarization angle, a polarization image with a 45° polarization angle, a polarization image with a 90° polarization angle, and a polarization image with a 135° polarization angle; the step of determining the total light intensity vector, the linearly polarized light component in the horizontal axis direction, and the linearly polarized light component in the 45° direction based on the polarization image to be processed includes:

[0015] Based on polarization images at 0° and 90° polarization angles, the total light intensity vector and the transversely linearly polarized light component are determined.

[0016] Based on the polarization images at a 45° polarization angle and a 135° polarization angle, the linearly polarized light component in the 45° direction is determined.

[0017] Furthermore, the step of determining the sharp target image based on the best polarization state image and the worst polarization state image includes:

[0018] Obtain the polarization degree image corresponding to the polarization image to be processed, and the pixel value of the brightest pixel in the worst polarization state image;

[0019] The target clear image is determined based on the polarization degree image, the pixel value, the best polarization state image, and the worst polarization state image.

[0020] Further, the step of obtaining the polarization degree image corresponding to the polarization image to be processed includes:

[0021] Based on the total light intensity vector, the linearly polarized light component in the horizontal direction, and the linearly polarized light component in the 45° direction corresponding to the polarization image to be processed, the polarization degree image corresponding to the polarization image to be processed is determined.

[0022] Further, the step of obtaining the polarization image to be processed corresponding to multiple polarization angles of the polarization image sensor based on the polarization information pixel data collected by the polarization image sensor includes:

[0023] Based on the polarization information pixel data collected by the polarization light image sensor, Bayer format polarization images corresponding to multiple polarization angles are obtained;

[0024] Color interpolation is performed on the Bayer format polarization image to obtain the RGB format polarization image to be processed.

[0025] Furthermore, the camera is equipped with a Bayer sensor; the image processing method further includes:

[0026] Based on the pixel data collected by the Bayer sensor, a Bayer image is obtained;

[0027] The Bayer image is preprocessed to obtain an RGB image;

[0028] The RGB image is converted to obtain a YUV image.

[0029] Furthermore, the polarized light endoscope device also includes a first display and a second display. The camera contains an optical prism, which splits a single beam of light output from the optical lens into two beams. The Bayer sensor acquires the light signal from one of the beams output from the optical prism and converts the light signal into pixel data. The polarized light image sensor acquires the polarized light signal from the other beam output from the optical prism and converts the polarized light signal into polarization information pixel data. The image processing method further includes:

[0030] The YUV image is displayed on the first display, and the target image is displayed on the second display.

[0031] In addition, to achieve the above objectives, the present invention also provides an image processing apparatus, the image processing apparatus comprising: a memory, a processor, and an image processing program stored in the memory and executable on the processor, wherein the image processing program, when executed by the processor, implements the steps of the aforementioned image processing method.

[0032] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing an image processing program, which, when executed by a processor, implements the steps of the aforementioned image processing method.

[0033] This invention acquires polarization images corresponding to multiple polarization angles of a polarization image sensor by collecting polarization information pixel data. Then, based on these polarization images, it determines the optimal and worst polarization state images. Finally, based on these optimal and worst polarization state images, it determines a clear target image. This enables the polarization-guided endoscope to achieve clear imaging in underwater environments (blood, turbidity, tissue debris, fog). In surgical environments with blood, tissue debris, fog, etc., the clear processing of polarization information pixel data solves the problem of unclear output images from the polarization-guided endoscope. This allows for direct surgery without needing to clean up interfering environments (blood, turbidity, tissue debris, fog), thereby improving surgical efficiency. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the structure of the image processing device in the hardware operating environment involved in the embodiments of the present invention;

[0035] Figure 2 This is a flowchart illustrating the first embodiment of the image processing method of the present invention;

[0036] Figure 3 This is a schematic flowchart of another embodiment of the image processing method of the present invention;

[0037] Figure 4 This is a flowchart illustrating another embodiment of the image processing method of the present invention.

[0038] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0039] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0040] like Figure 1 As shown, Figure 1 This is a schematic diagram of the structure of the image processing device in the hardware operating environment involved in the embodiments of the present invention.

[0041] In embodiments of the present invention, the image processing device can be located in the host unit of the endoscopic camera system. For example... Figure 1As shown, the image processing device may include: a processor 1001, such as a CPU; a network interface 1004; a user interface 1003; a memory 1005; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0042] Those skilled in the art will understand that Figure 1 The terminal structure shown does not constitute a limitation on the image processing device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0043] like Figure 1 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an image processing program.

[0044] exist Figure 1 In the image processing apparatus shown, the network interface 1004 is mainly used to connect to the backend server and communicate data with the backend server; the user interface 1003 is mainly used to connect to the client (user terminal) and communicate data with the client; and the processor 1001 can be used to call the image processing program stored in the memory 1005.

[0045] In this embodiment, the image processing apparatus includes: a memory 1005, a processor 1001, and an image processing program stored in the memory 1005 and executable on the processor 1001. When the processor 1001 calls the image processing program stored in the memory 1005, it executes the steps of the image processing methods in the following embodiments.

[0046] The present invention also provides an image processing method, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the image processing method of the present invention.

[0047] In this embodiment, the image processing method is applied to a polarized light endoscope device including a camera, wherein the camera is equipped with a polarized light image sensor.

[0048] Specifically, the polarized light endoscope device electrically includes a camera for acquiring pixel data of the target object and an image processing system for processing the pixel data. The camera and the image processing system transmit signals through a transmission cable.

[0049] The camera includes an optical lens, an optical prism, a Bayer sensor, a polarized light image sensor, an image transmission module, and a function selection module.

[0050] An optical lens is an optical path system composed of multiple optical mirrors. It provides an optical path for the white light and / or polarized light generated by the light source to be directed toward the target. After the light source is directed toward the target, it will generate a return light. The optical path system also provides an optical path for the return light. It can guide the return light to an optical prism. The optical prism splits the return light into two paths. One path is transmitted to the light field range where the Bayer sensor collects the light signal, and the other path is transmitted to the light field range where the polarized light image sensor collects the polarized light signal.

[0051] The Bayer sensor acquires the light signal from one of the beams output by the optical prism and converts the light signal into pixel data. In this embodiment, the Bayer sensor can simultaneously acquire red (R), green (G), and blue (B) light signals and convert them into RGB pixel data. The Bayer sensor in this embodiment can be a 4K resolution image sensor, such as a CCD image sensor or a CMOS image sensor. Furthermore, the final pixel data output by the Bayer sensor has 4K resolution, while the three-chip endoscope uses three independent monochrome image sensors, which acquire red (R), green (G), and blue (B) light signals respectively, converting them into R pixel data, G pixel data, and B pixel data respectively, and then combining the R pixel data, G pixel data, and B pixel data into pixel data.

[0052] A polarized light image sensor is used to acquire the polarized light signal of another beam output from the optical prism and convert the polarized light signal into polarization information pixel data. In this embodiment, the polarized light image sensor can acquire the polarized light signal from the returned light and convert the polarized light signal into polarization information pixel data, that is, convert the optical signal (polarized light) into an electrical signal (polarization information pixel data).

[0053] The image transmission module transmits pixel data and polarization information pixel data to the transmission cable, which contains transmission lines for various signal data transmissions. In this embodiment, the input terminal of the image transmission module is connected to the output terminals of the Bayer sensor and the polarized light image sensor, and the output terminal of the image transmission module is connected to the transmission cable.

[0054] The function selection module is located inside the camera and connected to the image processor. The module includes a button assembly located outside the camera housing. When triggered by the button assembly, the function selection module outputs at least one function selection signal to the image processor. The image processor controls the polarized light endoscope device to perform at least one corresponding function based on the function selection signal. For example, it can perform functions such as screenshotting, freezing, and recording of the acquired and processed real-time image data. When the buttons on the button assembly are pressed, they trigger the polarized light endoscope device to perform the corresponding function. In this embodiment, the image transmission module transmits pixel data and polarization information pixel data to the image receiving module via a serial data line in the transmission cable.

[0055] Specifically, the image processing method includes:

[0056] Step S101: Based on the polarization information pixel data collected by the polarization light image sensor, obtain the polarization image to be processed corresponding to multiple polarization angles of the polarization light image sensor.

[0057] In this embodiment, the polarized light image sensor can collect polarized light signals from the returned light and convert the polarized light signals into polarization information pixel data. The image processing system then obtains the polarization information pixel data collected by the polarized light image sensor and, based on the polarization information pixel data, obtains polarized images to be processed corresponding to multiple polarization angles of the polarized light image sensor. Specifically, it extracts image data of consecutive frames from the polarization information pixel data. The image data of consecutive frames includes multiple corresponding polarized images to be processed. Specifically, the polarization angles include 0°, 45°, 90°, and 135°. Therefore, the polarized images to be processed include images with a 0° polarization angle, images with a 45° polarization angle, images with a 90° polarization angle, and images with a 135° polarization angle.

[0058] In this embodiment, the image processing system can first preprocess the polarization information pixel data using existing image processing algorithms, such as linear correction, bad pixel removal, white balance, gamma, automatic exposure control, and also adjust parameters such as brightness, saturation, contrast, and sharpness of the output image data. Then, based on the preprocessed polarization information pixel data, multiple polarization images corresponding to different polarization angles are obtained.

[0059] Step S102: Based on the polarization image to be processed, determine the best polarization state image and the worst polarization state image;

[0060] In this embodiment, multiple Stokes vectors are calculated for each polarization image to be processed at each polarization angle. The degree of polarization image corresponding to the polarization image to be processed is determined by each Stokes vector. Then, the best polarization state image and the worst polarization state image are determined based on the degree of polarization image.

[0061] Step S103: Based on the best polarization state image and the worst polarization state image, determine the clear image of the target.

[0062] In this embodiment, when the best polarization state image and the worst polarization state image are obtained, the target clear image is calculated based on the best polarization state image and the worst polarization state image. Specifically, the underwater transmittance corresponding to the polarization image to be processed is first obtained, and then the target clear image is calculated using a formula based on the underwater transmittance, the best polarization state image, and the worst polarization state image, thereby obtaining a clear target polarization image. This realizes the function of clear imaging of the polarization endoscope device in underwater environments (blood, turbidity, tissue debris, fog), so that when blood, tissue debris, fog, etc. occur in the surgical environment, the problem of unclear output image of the polarization endoscope device is solved by clarifying the polarization information pixel data.

[0063] The image processing method proposed in this embodiment obtains polarization images corresponding to multiple polarization angles of the polarization image sensor based on polarization information pixel data collected by the polarization image sensor. Then, based on the polarization images to be processed, the best polarization state image and the worst polarization state image are determined. Subsequently, based on the best polarization state image and the worst polarization state image, a clear target image is determined. This enables the polarization endoscope device to achieve clear imaging in underwater environments (blood, turbidity, tissue debris, fog). When blood, tissue debris, fog, etc. are present in the surgical environment, the problem of unclear output images of the polarization endoscope device is solved by clarifying the polarization information pixel data. It can avoid cleaning up the interfering environment (blood, turbidity, tissue debris, fog) and directly perform surgery, thereby improving surgical efficiency.

[0064] Reference Figure 3 Based on the first embodiment, a second embodiment of the image processing method of the present invention is proposed. In this embodiment, as shown... Figure 3 As shown, step S102 includes:

[0065] Step S201: Based on the polarization image to be processed, determine the total light intensity vector, the linearly polarized light component in the horizontal axis direction, and the linearly polarized light component in the 45° direction.

[0066] Step S202: Based on the total light intensity vector, the linearly polarized light component along the horizontal axis, and the linearly polarized light component along the 45° direction, determine the optimal polarization state image and the worst polarization state image, respectively.

[0067] In this embodiment, when the polarization images to be processed at each polarization angle are acquired, the corresponding Stokes vector is calculated using the polarization images to be processed at each polarization angle. The Stokes vector includes four components, specifically including the polarization image at a 0° polarization angle, the polarization image at a 45° polarization angle, the polarization image at a 90° polarization angle, and the polarization image at a 135° polarization angle. The total light intensity vector, the linearly polarized light component in the horizontal axis direction, and the linearly polarized light component in the 45° direction are determined using the components of the Stokes vector.

[0068] Further, in one embodiment, the polarization images to be processed include polarization images with a 0° polarization angle, polarization images with a 45° polarization angle, polarization images with a 90° polarization angle, and polarization images with a 135° polarization angle. Step S201 includes:

[0069] Step a: Based on the polarization images at 0° and 90° polarization angles, determine the total light intensity vector and the transverse linearly polarized light component.

[0070] Step b: Based on the polarization images at a 45° polarization angle and the polarization images at a 135° polarization angle, determine the linearly polarized light component in the 45° direction.

[0071] In this embodiment, the polarization angles of the polarized light image sensor include 0°, 45°, 90°, and 135°. Therefore, the polarized images to be processed include polarized images with a 0° polarization angle, a 45° polarization angle, a 90° polarization angle, and a 135° polarization angle. When the polarized images to be processed are acquired, the total light intensity vector and the transverse linearly polarized light component are determined based on the polarized images with the 0° and 90° polarization angles. Specifically, the formulas for the total light intensity vector I and the transverse linearly polarized light component Q are: I = I... 0° +-I 90° Q = I 0° -I 90° .

[0072] Where I is the total light intensity vector, and Q is the linearly polarized light component along the X-axis, i.e., the transversely linearly polarized light component. 0° I is the three-dimensional matrix parameter of the polarization image at a 0° polarization angle. 90° The vector three-dimensional matrix parameters are for a polarized image with a 90° polarization angle. The three dimensions of the three-dimensional matrix parameters are R, G, and B, respectively. The parameters in each column of the three-dimensional matrix parameters are the R, G, and B values ​​of a pixel in the polarized image.

[0073] Simultaneously, based on the polarization images at 45° and 135° polarization angles, the linearly polarized light component in the 45° direction is determined. Specifically, the formula for the linearly polarized light component U in the 45° direction is: U = I45° +I 135° Where U is the linearly polarized light component at 45°, and I 45° I is the three-dimensional matrix parameter of a polarization image with a 45° polarization angle. 135° The vector three-dimensional matrix parameters of the polarization image at a polarization angle of 135°.

[0074] It should be noted that the circular polarization component V can also be determined based on polarization images at 45° and 135° polarization angles. The formula for the circular polarization component V is: V = I 45° -I 135° .

[0075] Next, based on the total light intensity vector, the linearly polarized light component along the horizontal axis, and the linearly polarized light component along the 45° direction, the best polarization state image and the worst polarization state image are determined respectively.

[0076] When acquiring the polarization image, the total light intensity vector, the linearly polarized light component along the horizontal axis, and the linearly polarized light component along the 45° direction corresponding to the polarization image to be processed are obtained. The methods for obtaining the total light intensity vector, the linearly polarized light component along the horizontal axis, and the linearly polarized light component along the 45° direction are similar to those in the previous embodiment and will not be repeated here. Based on the total light intensity vector, the linearly polarized light component along the horizontal axis, and the linearly polarized light component along the 45° direction, the optimal polarization state image Imax and the worst polarization state image Imin are calculated.

[0077] The formulas for the optimal polarization state image Imax and the worst polarization state image Imin are as follows:

[0078]

[0079] Where Imax is the best polarization state image, Imin is the worst polarization state image, I is the total light intensity vector, Q is the transverse linearly polarized light component, and U is the linearly polarized light component in the 45° direction.

[0080] This embodiment can accurately obtain the total light intensity vector, the transverse linear polarized light component, and the 45° linear polarized light component based on the Stokes vector. Furthermore, it can accurately obtain the best polarization state image and the worst polarization state image based on the Stokes vector, thereby improving the accuracy of the target clear image and enabling the polarized light endoscope device to achieve clear imaging in underwater environments (blood, turbidity, tissue debris, fog).

[0081] The image processing method proposed in this embodiment determines the total light intensity vector, the linearly polarized light component along the horizontal axis, and the linearly polarized light component along the 45° direction based on the polarized image to be processed. Then, based on the total light intensity vector, the linearly polarized light component along the horizontal axis, and the linearly polarized light component along the 45° direction, the optimal polarization state image and the worst polarization state image are determined respectively. The optimal and worst polarization state images can be accurately obtained according to the Stokes vector, thereby improving the clarity of the target image. This enables the polarized light endoscope device to achieve clear imaging in underwater environments (blood, turbidity, tissue debris, fog), avoiding the need to clean up the interfering environment (blood, turbidity, tissue debris, fog) and allowing for direct surgery, thus improving surgical efficiency.

[0082] Reference Figure 3 Based on the first embodiment, a third embodiment of the image processing method of the present invention is proposed. In this embodiment, as shown... Figure 3 As shown, step S103 includes:

[0083] Step S301: Obtain the polarization degree image corresponding to the polarization image to be processed, and the pixel value of the brightest pixel in the worst polarization state image;

[0084] Step S302: Based on the polarization image, the pixel value, the best polarization state image, and the worst polarization state image, determine the target clear image.

[0085] In this embodiment, when the best polarization state image and the worst polarization state image are obtained, the pixel value of the brightest pixel in the worst polarization state image is obtained. Specifically, based on the pixel values ​​(R / G / B values) of each pixel in the worst polarization state image, the brightness value of each pixel in the worst polarization state image is calculated using an existing brightness value algorithm. Then, the brightness values ​​are compared to obtain the pixel with the largest brightness value in the worst polarization state image, i.e., the brightest pixel, and the pixel value of the brightest pixel in the worst polarization state image is obtained.

[0086] Simultaneously, the polarization degree image corresponding to the polarization image to be processed is obtained. Specifically, the total light intensity vector, the linearly polarized light component along the horizontal axis, and the linearly polarized light component along the 45° direction corresponding to the polarization image to be processed are first obtained, and then the polarization degree image is determined. The method for obtaining the total light intensity vector, the linearly polarized light component along the horizontal axis, and the linearly polarized light component along the 45° direction is the same as in the second embodiment and will not be repeated here. Further, in one embodiment, step S301 includes:

[0087] Based on the total light intensity vector, the linearly polarized light component in the horizontal direction, and the linearly polarized light component in the 45° direction corresponding to the polarization image to be processed, the polarization degree image corresponding to the polarization image to be processed is determined.

[0088] This polarization degree image is the current underwater polarization degree image. Specifically, the formula for the polarization degree image P is:

[0089]

[0090] Where P is the polarization degree image, I is the total light intensity vector, Q is the transverse linearly polarized light component, and U is the linearly polarized light component in the 45° direction.

[0091] In this embodiment, the Stokes vectors I, Q, and U are first obtained from the polarization image to be processed. Then, the polarization degree image is calculated using I, Q, and U. This allows for the accurate acquisition of the polarization degree image corresponding to the polarization image to be processed, further improving the accuracy of the best and worst polarization state images. This enables the polarized endoscope device to achieve clear imaging in underwater environments (blood, turbidity, tissue debris, fog).

[0092] It should be noted that the circular polarization component V can also be calculated based on the polarization images at 45° and 135° polarization angles. The degree of polarization image P can then be calculated based on the total light intensity vector I, the transverse linearly polarized light component Q, the linearly polarized light component U at 45°, and the circular polarization component V. In this case, the formula for the degree of polarization image P is:

[0093]

[0094] Since the circular polarization component is minimal in the polarization effect of light incident between underwater targets and the background, V=0 can be set, and the polarization degree image can be calculated based solely on I, Q, and U, thereby improving image processing efficiency.

[0095] Next, based on the polarization degree image, pixel values, the best polarization state image, and the worst polarization state image, a clear target image is determined. The formula for the clear target image is:

[0096]

[0097] Among them, H OBJ P is the image of the target with a clear view, and A is the polarization image. ∞ Imax represents the pixel value of the brightest pixel in the worst polarization state image, Imin represents the best polarization state image, and Imin represents the worst polarization state image.

[0098] The image processing method proposed in this embodiment obtains the polarization degree image corresponding to the polarization image to be processed, and the pixel value of the brightest pixel in the worst polarization state image; then, based on the polarization degree image, the pixel value, the best polarization state image, and the worst polarization state image, the target clear image is determined. This method can accurately obtain a clear target polarization image through the polarization degree image, the best polarization state image, and the worst polarization state image, enabling the polarized light endoscope device to achieve clear imaging in underwater environments (blood, turbidity, tissue debris, fog). This solves the problem of unclear output images from the polarized light endoscope device by clarifying the polarization information pixel data in situations such as blood, tissue debris, and fog in the surgical environment, further improving surgical efficiency.

[0099] Reference Figure 3 Based on the first embodiment, a fourth embodiment of the image processing method of the present invention is proposed. In this embodiment, as shown... Figure 3 As shown, step S101 includes:

[0100] Step S401: Based on the polarization information pixel data collected by the polarization light image sensor, obtain Bayer format polarization images corresponding to multiple polarization angles;

[0101] Step S402: Perform color interpolation processing on the Bayer format polarization image to obtain the polarization image to be processed in RGB format.

[0102] In this embodiment, the polarized light image sensor is a Bayer format image sensor. When acquiring the polarization information pixel data collected by the polarized light image sensor, multiple Bayer format polarized images corresponding to polarization angles are acquired based on the polarization information pixel data, namely, Bayer format images with a 0° polarization angle, Bayer format images with a 45° polarization angle, Bayer format images with a 90° polarization angle, and Bayer format images with a 135° polarization angle.

[0103] Next, color interpolation processing is performed on the Bayer format polarization image to obtain the polarization image to be processed in RGB format. Specifically, color interpolation processing is performed on the Bayer format image with a 0° polarization angle to obtain a polarization image with a 0° polarization angle, on the Bayer format image with a 45° polarization angle to obtain a polarization image with a 45° polarization angle, on the Bayer format image with a 90° polarization angle to obtain a polarization image with a 90° polarization angle, and on the Bayer format image with a 135° polarization angle to obtain a polarization image with a 135° polarization angle. Thus, the polarization image to be processed in RGB format can be accurately obtained.

[0104] In this embodiment, the image processing system can first preprocess the polarization information pixel data using existing image processing algorithms, such as linear correction, bad pixel removal, white balance, gamma, automatic exposure control, and also adjust parameters such as brightness, saturation, contrast, and sharpness of the output image data. Then, based on the preprocessed polarization information pixel data, Bayer format polarization images corresponding to multiple polarization angles are obtained.

[0105] The image processing method proposed in this embodiment obtains Bayer format polarized images corresponding to multiple polarization angles by acquiring polarization information pixel data based on polarization light image sensor; then, it performs color interpolation processing on the Bayer format polarized images to obtain the RGB format polarized image to be processed. The RGB format polarized image to be processed is obtained based on the polarization information pixel data, so as to facilitate subsequent processing of the polarized image to be processed, improve the computational efficiency of the target clear image, and further improve the efficiency of surgery.

[0106] Please see Figure 4 Based on the above embodiments, a fifth embodiment of the image processing method of the present invention is proposed, as follows: Figure 4 As shown, in this embodiment, the image processing method further includes:

[0107] Step S501: Obtain a Bayer image based on the pixel data collected by the Bayer sensor;

[0108] Step S502: Preprocess the Bayer image to obtain an RGB image;

[0109] Step S503: Perform image conversion on the RGB image to obtain a YUV image.

[0110] In this embodiment, a Bayer image is obtained based on the pixel data collected by the Bayer sensor. The image processing system preprocesses the pixel data collected by the Bayer sensor using existing image processing algorithms, such as linear correction, bad pixel removal, white balance, gamma, and automatic exposure control. It also adjusts parameters such as brightness, saturation, contrast, and sharpness of the output image data. Then, the Bayer image is obtained based on the preprocessed pixel data collected by the Bayer sensor.

[0111] Next, the Bayer image is preprocessed to obtain an RGB image. Specifically, the Bayer image is color interpolated to obtain a first RGB image. The first RGB image is then CCM (Color Correction Matrix) color correction to obtain a second RGB image that is closest to the real colors of the surgical scene. Finally, the second RGB image is Gamma corrected to obtain an RGB image, so that the RGB image is more consistent with the human visual system.

[0112] Then, the RGB image is converted to a YUV image, that is, the RGB image in the RGB domain is converted to a YUV image in the YUV domain. Wide dynamic range processing is used to solve the problem of overexposure in bright areas and underexposure in dark areas in surgical scene images. In order to eliminate the loss of image details during noise reduction, the image needs to be sharpened to restore and enhance the image's detail information. After processing, traditional endoscopic imaging (any resolution) is achieved, clearly restoring the surgical scene and further improving surgical efficiency. Among them, YUV is divided into three components: "Y" represents luminance (Luminance or Luma), which is the grayscale value; while "U" and "V" represent chrominance (Chrominance or Chroma), which describes the color and saturation of the image and is used to specify the color of the pixel.

[0113] Furthermore, in one embodiment, the polarized light endoscope device further includes a first display and a second display. The camera is equipped with an optical prism, which splits a beam of light output from the optical lens into two beams. The Bayer sensor acquires the light signal of one beam output from the optical prism and converts the light signal into pixel data. The polarized light image sensor acquires the polarized light signal of the other beam output from the optical prism and converts the polarized light signal into polarization information pixel data. The image processing method further includes displaying the YUV image on the first display and displaying the clear target image on the second display. This achieves simultaneous display of the clear target image and the YUV image, further improving surgical efficiency.

[0114] The image processing method proposed in this embodiment acquires a Bayer image based on pixel data collected by the Bayer sensor; then, the Bayer image is preprocessed to obtain an RGB image; and then the RGB image is converted to obtain a YUV image. The surgical scene can then be reconstructed using the YUV image and a clear image of the target, enabling the polarized light endoscope device to achieve clear imaging in underwater environments (blood, turbidity, tissue debris, fog), further improving surgical efficiency.

[0115] The present invention also provides a computer-readable storage medium.

[0116] The present invention provides a computer-readable storage medium storing an image processing program, which, when executed by a processor, implements the steps of the image processing method described above.

[0117] The method implemented when the image processing program running on the processor is executed can be referred to in various embodiments of the image processing method of the present invention, and will not be repeated here.

[0118] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0119] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0120] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0121] The above are merely preferred embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. An image processing method, characterized in that, A polarized light endoscope device including a camera, wherein the camera is equipped with a polarized light image sensor; the image processing method includes the following steps: Based on the polarization information pixel data collected by the polarization light image sensor, obtain the polarization image to be processed corresponding to multiple polarization angles of the polarization light image sensor; Based on the polarization image to be processed, determine the best polarization state image and the worst polarization state image; Obtain the polarization degree image corresponding to the polarization image to be processed, and the pixel value of the brightest pixel in the worst polarization state image; Based on the polarization image, the pixel value, the best polarization state image, and the worst polarization state image, a clear target image is determined.

2. The image processing method as described in claim 1, characterized in that, The step of determining the best polarization state image and the worst polarization state image based on the polarization image to be processed includes: Based on the polarization image to be processed, determine the total light intensity vector, the linearly polarized light component along the horizontal axis, and the 45° angle. ° Direction of linearly polarized light component; Based on the total light intensity vector, the linearly polarized light component along the horizontal axis, and the 45°... ° The linearly polarized light components are used to determine the optimal polarization state image and the worst polarization state image, respectively.

3. The image processing method as described in claim 2, characterized in that, The polarization images to be processed include polarization images with a 0° polarization angle, polarization images with a 45° polarization angle, polarization images with a 90° polarization angle, and polarization images with a 135° polarization angle. Based on the polarization image to be processed, the total light intensity vector, the linearly polarized light component along the horizontal axis, and the 45° angle are determined. ° The steps for directional linearly polarized light components include: Based on polarization images at 0° and 90° polarization angles, the total light intensity vector and the transversely linearly polarized light component are determined. Based on polarization images at 45° and 135° polarization angles, the polarization angle at 45° is determined. ° The linearly polarized light component.

4. The image processing method as described in claim 1, characterized in that, The step of obtaining the polarization degree image corresponding to the polarization image to be processed includes: Based on the total light intensity vector corresponding to the polarization image to be processed, the linearly polarized light component in the horizontal axis direction, and 45° ° The linearly polarized light component is used to determine the degree of polarization image corresponding to the polarization image to be processed.

5. The image processing method as described in claim 1, characterized in that, The step of obtaining the polarization image to be processed corresponding to multiple polarization angles of the polarization image sensor based on the polarization information pixel data collected by the polarization image sensor includes: Based on the polarization information pixel data collected by the polarization light image sensor, Bayer format polarization images corresponding to multiple polarization angles are obtained; Color interpolation is performed on the Bayer format polarization image to obtain the RGB format polarization image to be processed.

6. The image processing method according to any one of claims 1 to 5, characterized in that, The camera is equipped with a Bayer sensor; the image processing method further includes: Based on the pixel data collected by the Bayer sensor, a Bayer image is obtained; The Bayer image is preprocessed to obtain an RGB image; The RGB image is converted to obtain a YUV image.

7. The image processing method as described in claim 6, characterized in that, The polarized light endoscope device further includes a first display and a second display. The camera is equipped with an optical prism, which is used to split a beam of light output from the optical lens into two beams. The Bayer sensor collects the light signal of one of the beams output from the optical prism and converts the light signal into pixel data. The polarized light image sensor collects the polarized light signal of the other beam output from the optical prism and converts the polarized light signal into polarization information pixel data. The image processing method further includes: The YUV image is displayed on the first display, and the target image is displayed on the second display.

8. An image processing apparatus, characterized in that, The image processing apparatus includes: a memory, a processor, and an image processing program stored in the memory and executable on the processor, wherein the image processing program, when executed by the processor, implements the steps of the image processing method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an image processing program, which, when executed by a processor, implements the steps of the image processing method as described in any one of claims 1 to 7.

Images