Endoscope imaging system and image processing method thereof

By identifying and removing the reflective areas in the endoscope image, and using the mirror and diffuse reflection components for de-reflection processing, the complexity and lack of real-timeness of the reflection problem in the prior art are solved, and a low-cost and efficient image processing effect is achieved. .

CN119006401APending Publication Date: 2024-11-22SHENZHEN MINDRAY BIO MEDICAL ELECTRONICS CO LTD +1
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202411074779.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-06
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The prior art reduces the reflection in the endoscope image, hardware complexity and cost increase, and the reflection problem of different areas or objects cannot be completely eliminated, and the image processing method has a long calculation time, limiting real-time applications.

Method used

By identifying and removing the reflective areas in the endoscope image, determining the mirror and diffuse reflection components, and only the reflective areas are dereflective, simplifying the calculation process and improving the processing speed.

Benefits of technology

It realizes low-cost, high real-time and stable reflection removal, ensuring the clarity and accuracy of the endoscopic image, reducing the amount of calculation, avoiding the mishandling of non-reflective areas, and improving the accuracy of diagnosis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006401A_ABST
    Figure CN119006401A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses an endoscope camera system and an image processing method thereof, and relates to the technical field of medical equipment, and the method comprises the steps: obtaining endoscope image data which comprises an endoscope image; performing reflective area identification processing on the endoscope image, and determining a reflective area in the endoscope image; determining brightness information of the reflective area from the endoscope image; determining a specular reflection component of the reflective area from the endoscope image; determining a diffuse reflection component of the reflective area according to the brightness information of the reflective area and the specular reflection component; and according to the diffuse reflection component, carrying out light reflection removal processing on the light reflection area of the endoscope image to obtain a target endoscope image, so that light reflection in the endoscope image can be effectively removed, and low cost, high real-time performance and stability can be maintained.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of medical equipment, and in particular to an endoscopic image processing method and an endoscopic camera system thereof. Background Art

[0002] Medical endoscopes are indispensable tools in clinical diagnosis and surgery, which help doctors observe the internal tissues of the human body through real-time images. However, during the endoscopic imaging process, the humid environment inside the human body can cause reflection problems, which in turn interferes with the display clarity of the endoscopic image and affects the doctor's accurate judgment of the tissue structure.

[0003] To solve this problem, existing technologies use a variety of methods to reduce or eliminate reflections in endoscopic images. One of the methods is to achieve a de-reflection effect by setting cross-polarizers. Although this method can reduce reflections to a certain extent, it also increases the hardware complexity and cost of the system, limiting its wide applicability in different application scenarios. In addition, this method cannot completely eliminate the reflection problem in different areas or objects in the field of view. Another method is to use image processing technology to remove highlights from endoscopic images through iterative calculations. Although this method can theoretically provide a more ideal de-reflection effect, its calculation process often takes a long time, which limits its use in clinical applications that require real-time feedback. Summary of the invention

[0004] The following is a summary of the subject matter described in detail in this application. This summary is not intended to limit the scope of the claims.

[0005] The embodiments of the present application provide an endoscopic camera system and an image processing method thereof, which can effectively remove reflections in endoscopic images while maintaining low cost, high real-time performance and stability.

[0006] On the one hand, an embodiment of the present application provides an endoscopic image processing method, the method comprising:

[0007] Acquiring endoscopic image data, wherein the endoscopic image data includes an endoscopic image;

[0008] Performing a reflective region recognition process on the endoscopic image to determine a reflective region in the endoscopic image;

[0009] determining brightness information of the reflective area from the endoscopic image;

[0010] determining a specular reflection component of the reflective area from the endoscopic image;

[0011] Determine the diffuse reflection component of the reflective area according to the brightness information of the reflective area and the specular reflection component;

[0012] The reflective area of ​​the endoscopic image is de-reflected according to the diffuse reflection component to obtain a target endoscopic image.

[0013] An endoscopic image processing method provided in accordance with an embodiment of the present invention has at least the following beneficial effects: performing reflective area recognition processing on the endoscopic image, determining the reflective area in the endoscopic image, avoiding erroneous processing of non-reflective areas, and ensuring the naturalness and accuracy of the processing effect, determining the mirror reflection component of the reflective area from the endoscopic image, further acquiring brightness information of the reflective area, and determining the diffuse reflection component of the reflective area based on the brightness information and the mirror reflection component, and finally, performing de-reflection processing on the reflective area of ​​the endoscopic image based on the diffuse reflection component to obtain a target endoscopic image. In addition, since de-reflection processing is only performed on the reflective area in the image instead of performing complex iterative calculations on the entire image, this method can greatly reduce the amount of calculation, improve the processing speed, and ensure that the endoscope can output clear, non-reflection images in a timely manner during operation.

[0014] In some embodiments, determining the specular reflection component of the reflective area from the endoscopic image comprises one of the following:

[0015] Acquire the neighborhood image information of the reflective area in the endoscopic image, and compare the image information in the reflective area with the neighborhood image information to obtain the specular reflection component of the reflective area;

[0016] Alternatively, the endoscope image and the image information of the reflective area are input into a specular reflection component recognition model for prediction processing to obtain the specular reflection component;

[0017] Alternatively, the image information of the reflective area is input into a specular reflection component recognition model for prediction processing to obtain the specular reflection component.

[0018] In some embodiments, the image information of the reflective area includes at least one of the following:

[0019] Image data of the reflective area;

[0020] Brightness information of the reflective area;

[0021] The reflective degree information of the reflective area.

[0022] In some embodiments, the endoscopic image data includes a plurality of frames of endoscopic images, and determining the specular reflection component of the reflective area from the endoscopic images includes:

[0023] Acquire historical image information of historical endoscopic images, wherein the historical endoscopic images are historical frame endoscopic images in the endoscopic image data;

[0024] Determine the specular reflection component of the reflective area according to the historical image information and the image information of the reflective area;

[0025] The historical image information includes at least one of the following:

[0026] a specular component of the historical endoscopic image;

[0027] image data of the historical endoscopic images;

[0028] image data of the reflective area in the historical endoscopic image;

[0029] a non-reflective endoscopic image after the historical endoscopic image has been subjected to de-reflection processing;

[0030] Information on the degree of reflection in the historical endoscopic images;

[0031] Information about the degree of reflection of the reflective area in the historical endoscopic image.

[0032] In some embodiments, determining the specular reflection component of the reflective area according to the historical image information and the image information of the reflective area includes:

[0033] Comparing and obtaining inter-frame difference information between the endoscopic image and the historical endoscopic image;

[0034] Determine the specular reflection component of the reflective area according to the image information of the reflective area, the inter-frame difference information and / or the historical image information;

[0035] The inter-frame difference information includes at least one of the following:

[0036] first brightness difference information between the endoscopic image and the historical endoscopic image;

[0037] second brightness difference information of the reflective area between the endoscopic image and the historical endoscopic image;

[0038] The reflection degree difference information between the reflection degree information of the endoscopic image and the reflection degree information of the historical endoscopic image.

[0039] In some embodiments, determining the specular reflection component of the reflective area according to the historical image information and the image information of the reflective area includes one of the following:

[0040] The historical image information and the image information of the reflective area are input into a specular reflection component recognition model for prediction processing to obtain the specular reflection component;

[0041] or,

[0042] determining a first specular reflection component of the reflective region from the endoscopic image;

[0043] determining a second specular reflection component of the reflective region from the historical endoscopic image;

[0044] Based on the historical image information, the first specular reflection component and the second specular reflection component are fused to obtain a specular reflection component.

[0045] In some embodiments, the fusion processing includes performing a weighted summation processing according to a weighting coefficient, wherein the weighting coefficient is obtained according to at least one of the following parameters:

[0046] brightness information of the endoscopic image;

[0047] brightness information of the reflective area in the endoscopic image;

[0048] historical brightness information of the historical endoscopic images;

[0049] historical brightness information of the reflective area in the historical endoscopic image;

[0050] first brightness difference information between the endoscopic image and the historical endoscopic image;

[0051] second brightness difference information of the reflective area between the endoscopic image and the historical endoscopic image;

[0052] Information on the degree of reflection of the endoscopic image;

[0053] Information on the degree of reflectivity of the historical endoscopic images;

[0054] Reflectivity difference information between the reflectivity information of the endoscopic image and the reflectivity information of the historical endoscopic images;

[0055] Reflection intensity distribution information of the endoscopic image;

[0056] Reflection intensity distribution information of the reflective area of ​​the endoscopic image;

[0057] The historical endoscopic image includes historical reflection intensity distribution information of the reflection area.

[0058] In some embodiments, determining the first specular reflection component of the reflective area from the endoscopic image includes:

[0059] After inputting the image information of the reflective area into a specular reflection component recognition model for prediction processing, the first specular reflection component is obtained;

[0060] Determining the second specular reflection component of the reflective area from the historical endoscopic image comprises one of the following:

[0061] After inputting the image information of the reflective area of ​​the historical endoscopic image into a specular reflection component recognition model for prediction processing, the second specular reflection component is obtained;

[0062] Alternatively, the second specular reflection component saved after the reflective area of ​​the historical endoscopic image has been predictively processed by a specular reflection component recognition model is obtained.

[0063] In some embodiments, performing de-reflection processing on the reflective area of ​​the endoscopic image according to the diffuse reflection component to obtain a target endoscopic image includes:

[0064] performing de-reflection processing on the reflective area of ​​the endoscopic image according to the diffuse reflection component to obtain a local endoscopic image corresponding to the reflective area;

[0065] The reflective area in the endoscopic image is replaced by the local endoscopic image to obtain a target endoscopic image.

[0066] In some embodiments, determining the specular reflection component of the reflective area from the endoscopic image comprises:

[0067] After downsampling the image information of the reflective area, first image information of the reflective area is obtained, and a specular reflection component of the reflective area is determined from the first image information;

[0068] Replacing the reflective area in the endoscopic image with the local endoscopic image to obtain a target endoscopic image includes:

[0069] After upsampling the local endoscopic image, the reflective area in the endoscopic image is replaced to obtain a target endoscopic image.

[0070] In some embodiments, performing reflective region recognition processing on the endoscopic image to determine the reflective region in the endoscopic image includes one of the following:

[0071] Determine the reflective points of the endoscopic image, and determine the reflective area in the endoscopic image according to the distribution of the reflective points in the endoscopic image;

[0072] Alternatively, determining the reflective area in the endoscopic image according to the distribution of brightness values ​​of the endoscopic image;

[0073] Alternatively, determining the reflective area in the endoscopic image according to the distribution of values ​​of different color channels in the endoscopic image;

[0074] After the endoscopic image is input into the reflective region recognition model for prediction processing, the reflective region in the endoscopic image is obtained.

[0075] On the other hand, an embodiment of the present application provides an endoscopic image processing method, the method comprising:

[0076] Acquiring endoscopic image data, wherein the endoscopic image data includes an endoscopic image;

[0077] determining a specular component of the endoscopic image from the endoscopic image;

[0078] Acquiring brightness information of the endoscopic image, and determining a diffuse reflection component of the endoscopic image according to the brightness information and the specular reflection component;

[0079] The endoscopic image is subjected to de-reflection processing according to the diffuse reflection component to obtain a target endoscopic image.

[0080] An endoscopic image processing method provided in accordance with an embodiment of the present invention has at least the following beneficial effects: first, endoscopic image data is acquired, wherein the endoscopic image data includes an endoscopic image, thereby ensuring that subsequent processing has a reliable data basis; second, a mirror reflection component of the endoscopic image is determined from the endoscopic image; then, brightness information of the endoscopic image is acquired, and a diffuse reflection component of the endoscopic image is determined based on the brightness information and the mirror reflection component; finally, the endoscopic image is de-reflected based on the diffuse reflection component to obtain a target endoscopic image; since the mirror reflection component of the endoscopic image is determined directly from the endoscopic image, the image processing steps are simplified, thereby ensuring that the endoscope can output clear, non-reflective images in a timely manner during operation.

[0081] In some embodiments, determining the specular reflection component of the endoscopic image from the endoscopic image comprises one of the following:

[0082] Acquire reflection intensity distribution information from the endoscopic image, determine neighborhood image information of the reflective area according to the reflection intensity distribution information, and compare the image information in the reflective area with the neighborhood image information to obtain the specular reflection component of the reflective area;

[0083] Alternatively, the endoscope image is input into a specular reflection component recognition model for prediction processing to obtain the specular reflection component.

[0084] In some embodiments, determining the specular reflection component of the endoscopic image from the endoscopic image comprises:

[0085] Acquire historical image information of historical endoscopic images, wherein the historical endoscopic images are historical frame endoscopic images in the endoscopic image data;

[0086] determining a specular reflection component of the endoscopic image based on the historical image information and the endoscopic image;

[0087] The historical image information includes at least one of the following:

[0088] a specular component of the historical endoscopic image;

[0089] image data of the historical endoscopic images;

[0090] a non-reflective endoscopic image after the historical endoscopic image has been subjected to de-reflection processing;

[0091] Information about the degree of reflection in the historical endoscopic images.

[0092] On the other hand, an embodiment of the present application provides an endoscopic image processing method, the method comprising:

[0093] Acquiring endoscopic image data, wherein the endoscopic image data includes an endoscopic image;

[0094] After performing de-reflection processing on the endoscopic image, a first endoscopic image is obtained;

[0095] Acquire historical image information of historical endoscopic images, wherein the historical endoscopic images are historical frame endoscopic images in the endoscopic image data;

[0096] Based on the historical image information of the historical endoscopic image, the first endoscopic image is smoothed to obtain a target endoscopic image.

[0097] An endoscopic image processing method provided in accordance with an embodiment of the present invention has at least the following beneficial effects: acquiring endoscopic image data, the endoscopic image data including an endoscopic image, performing de-reflection processing on the endoscopic image to obtain a first endoscopic image, acquiring historical image information of historical endoscopic images, the historical endoscopic images being historical frame endoscopic images in the endoscopic image data, and performing smoothing processing on the first endoscopic image based on the historical image information of the historical endoscopic images to obtain a target endoscopic image, wherein the target endoscopic image finally obtained is clearer and smoother than the original endoscopic image, and is more helpful for diagnosis and analysis, and in addition, smoothing processing based on historical data helps to reduce mutations between image frames, thereby avoiding flickering in the video.

[0098] In some embodiments, after performing de-reflection processing on the endoscopic image, obtaining a first endoscopic image includes:

[0099] After the endoscopic image is input into a de-reflection model for de-reflection processing, a first endoscopic image is obtained.

[0100] In some embodiments, the historical image information includes image data of the historical endoscopic image or a historical non-reflective endoscopic image after the historical endoscopic image is de-reflected; the smoothing of the first endoscopic image based on the historical image information of the historical endoscopic image to obtain the target endoscopic image includes one of the following:

[0101] fusing the first endoscopic image with the historical endoscopic image to obtain a target endoscopic image;

[0102] Alternatively, the first endoscopic image and the historical non-reflective endoscopic image are fused to obtain a target endoscopic image.

[0103] In some embodiments, the fusion processing includes performing a weighted summation processing according to a weighting coefficient, wherein the weighting coefficient is obtained according to at least one of the following parameters:

[0104] brightness information of the first endoscopic image;

[0105] historical brightness information of the historical endoscopic images;

[0106] first brightness difference information between the first endoscopic image and the historical endoscopic image;

[0107] Information about the degree of reflection of the first endoscopic image;

[0108] Reflectivity difference information between the reflectivity information of the first endoscopic image and the reflectivity information of the historical endoscopic image;

[0109] Reflection intensity distribution information of the first endoscopic image;

[0110] Historical reflection intensity distribution information of the historical endoscopic images;

[0111] Information about the degree of reflection in the historical endoscopic images.

[0112] In some embodiments, in the fusion process, the target endoscopic image is calculated by the following formula:

[0113] y′=ypre*W1+y*W2

[0114] Among them, y' is the RGB information of the unit pixel of the target endoscopic image, y is the RGB information of the endoscopic image in the current frame, ypre is the RGB information of the historical endoscopic image or the historical non-reflective endoscopic image in the previous frame, and W1 and / or W2 are the weighting coefficients.

[0115] On the other hand, an embodiment of the present application provides an endoscopic image processing method, the method comprising:

[0116] Acquiring endoscopic image data, wherein the endoscopic image data includes an endoscopic image;

[0117] Acquiring a reflective area in the endoscopic image;

[0118] After performing de-reflection processing on the endoscopic image, a first endoscopic image is obtained;

[0119] Acquire historical image information of historical endoscopic images, wherein the historical endoscopic images are historical frame endoscopic images in the endoscopic image data;

[0120] Based on the historical image information of the historical endoscopic image, smoothing the first endoscopic image to obtain a second endoscopic image;

[0121] extracting a local endoscopic image of the reflective area from the second endoscopic image;

[0122] The reflective area in the endoscopic image is replaced by the local endoscopic image to obtain a target endoscopic image.

[0123] On the other hand, an embodiment of the present application provides an endoscope camera system, comprising:

[0124] Endoscope;

[0125] A light source device, used to provide lighting for the endoscope;

[0126] A camera host is connected to the endoscope, and is used to obtain an image signal of a target object photographed by the endoscope and display an endoscopic image of the target object through a display, wherein the camera host executes an endoscopic image processing method of any one of the above embodiments.

[0127] On the other hand, an embodiment of the present application provides a camera host, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the endoscopic image processing method of any one of the above embodiments is implemented.

[0128] On the other hand, an embodiment of the present application further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the endoscopic image processing method of any one of the above embodiments is implemented.

[0129] On the other hand, an embodiment of the present application further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is executed by a processor to implement the endoscopic image processing method of any one of the above embodiments.

[0130] On the other hand, the embodiment of the present application further provides a computer program product, which includes a computer program, and the computer program is stored in a computer storable medium. The processor of the computer device reads the computer program from the computer readable storage medium, and the processor executes the computer program, so that the computer device executes the endoscopic image processing method of any one of the above embodiments.

[0131] According to the embodiments of the present invention, at least the following beneficial effects are achieved: obtaining the reflective area in the endoscopic image, laying a data foundation for subsequently replacing the reflective area in the endoscopic image with the local endoscopic image to obtain a target endoscopic image, and further, obtaining a first endoscopic image after de-reflection processing of the endoscopic image; obtaining historical image information of a historical endoscopic image, wherein the historical endoscopic image is a historical frame endoscopic image in the endoscopic image data; and smoothing the first endoscopic image based on the historical image information of the historical endoscopic image. Processing is performed to obtain a second endoscopic image, wherein, compared with the unprocessed endoscopic image, the clarity of the local endoscopic image of the reflective area in the second endoscopic image obtained after de-reflection processing and smoothing processing is greatly improved, and the local endoscopic image of the reflective area is extracted from the second endoscopic image; the reflective area in the endoscopic image is replaced by the local endoscopic image to obtain a target endoscopic image, thereby ensuring that only the area affected by the reflection is further processed and replaced, thereby improving the overall quality of the image.

[0132] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent from the description, or may be understood by practicing the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0133] The accompanying drawings are used to provide further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0134] Figure 1 A system principle block diagram of an endoscope camera system provided in an embodiment of the present application;

[0135] Figure 2 A logical schematic diagram of an endoscopic image processing method provided in an embodiment of the present application;

[0136] Figure 3 A logical schematic diagram of another endoscopic image processing method provided in an embodiment of the present application;

[0137] Figure 4 A logical schematic diagram of another endoscopic image processing method provided in an embodiment of the present application;

[0138] Figure 5 A logical schematic diagram of another endoscopic image processing method provided in an embodiment of the present application;

[0139] Figure 6 A logical schematic diagram of another endoscopic image processing method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0140] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0141] It should be noted that in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the characteristics of the target object such as target object attribute information or attribute information set, the permission or consent of the target object will be obtained first, and the collection, use and processing of these data will comply with relevant laws, regulations and standards. Among them, the target object can be a user. In addition, when the embodiment of the present application needs to obtain the target object attribute information, the target object's separate permission or separate consent will be obtained by means of a pop-up window or jumping to a confirmation page, etc., and after the separate permission or separate consent of the target object is clearly obtained, the necessary target object-related data for enabling the normal operation of the embodiment of the present application will be obtained.

[0142] To facilitate understanding of the technical solutions provided in the embodiments of the present application, some key terms used in the embodiments of the present application are explained here:

[0143] Multilayer Perceptron (MLP): also called Artificial Neural Network (ANN). In addition to the input and output layers, it can have multiple hidden layers in the middle. The simplest MLP contains only one hidden layer, that is, a three-layer structure.

[0144] Leaky ReLU (Leaky Rectified Linear Unit) is an activation function used to introduce nonlinearity in neural networks.

[0145] Convolutional Neural Networks (CNN) are a type of feedforward neural networks that include convolution calculations and have a deep structure. They are one of the representative algorithms of deep learning.

[0146] Medical endoscopes are indispensable tools in clinical diagnosis and surgery, which help doctors observe the internal tissues of the human body through real-time images. However, during the endoscopic imaging process, the humid environment inside the human body can cause reflection problems, which in turn interferes with the display clarity of the endoscopic image and affects the doctor's accurate judgment of the tissue structure.

[0147] To solve this problem, existing technologies use a variety of methods to reduce or eliminate reflections in endoscopic images. One of the methods is to achieve a de-reflection effect by setting cross-polarizers. Although this method can reduce reflections to a certain extent, it also increases the hardware complexity and cost of the system, limiting its wide applicability in different application scenarios. In addition, this method cannot completely eliminate the reflection problem in different areas or objects in the field of view. Another method is to use image processing technology to remove highlights from endoscopic images through iterative calculations. Although this method can theoretically provide a more ideal de-reflection effect, its calculation process often takes a long time, which limits its use in clinical applications that require real-time feedback.

[0148] Based on this, the embodiments of the present application provide an endoscopic image processing method and an endoscopic camera system thereof, which can effectively remove reflections in endoscopic images while maintaining low cost, high real-time performance and stability.

[0149] Reference Figure 1, is a schematic diagram of the structure of an endoscopic camera system, the endoscopic camera system 1000 includes but is not limited to a light source device 10, a light guide beam 20, and an endoscope, the endoscope includes a mirror tube 30, an optical mount 40, and a camera assembly 50, and the endoscopic camera system 1000 also includes a camera host 60, a display 70 and a communication cable 80.

[0150] One end of the light guide 20 is connected to the light output end of the light source device 10, and the other end is connected to the light source interface of the endoscope. The light guide 20 is used to provide an illumination light signal for the endoscope to illuminate the target object (eg, the part to be observed of the observation body).

[0151] One end of the camera assembly 50 is connected to the image signal input interface of the camera host 60 through a communication cable 80, and the other end is connected to the lens tube 30 through an optical bayonet 40, so as to obtain the optical signal of the target object 90 transmitted by the lens tube 30 for imaging, obtain the image signal, and transmit the obtained image signal to the camera host 60 for processing. The camera assembly 50 includes a handle, and a plurality of buttons are arranged on the handle. The user can call the function corresponding to the button by pressing the button. The target object 90 can be a human body.

[0152] The display 70 is connected to the camera host 60 via a communication cable 80, and is used to display the image output by the camera host 60. The display 70 may include a liquid crystal display, an LED display, or an OLED display, etc., and of course, it may also be a display on an electronic device such as a mobile phone, a tablet computer, or a personal computer connected to the camera host 60, etc.

[0153] In one embodiment, the camera host 60 and the light source device 10 are connected in communication so that the camera host 60 and the light source device 10 can be linked and synchronized to complete specific functions. For example, during the use of the endoscope, the camera host 60 links the light source device 10 to increase or decrease the brightness according to the parameters of the endoscopic image captured by the endoscope, or, during the white balance correction function, the camera host 60 links and controls the light source device 10 to start when correcting the white balance to provide the illumination light required for white balance. Alternatively, the camera host 60 controls the light source device to link and output test light with varying light intensity according to the light transmission capability requirements of the endoscope camera system. The camera host 60 and the light source device 10 can be communicatively connected via wired or wireless means. In some embodiments, the camera host 60 and the light source device 10 can communicate via a server or a main control center. For example, the camera host 60 feeds back information to the server or the main control center, and then the server or the main control center issues a control instruction (such as a test instruction for light transmission capacity) to the light source device 10. Alternatively, the light source device 10 feeds back information to the server or the main control center, and then the server or the main control center issues a control instruction to the camera host 60.

[0154] It should be understood by those skilled in the art that Figure 1 The endoscope camera system 1000 is merely an example and does not constitute a limitation on the endoscope camera system 1000. The endoscope camera system 1000 may include Figure 1 More or fewer components as shown, or a combination of certain components, or different components, for example, the endoscope camera system 1000 may also include a dilator, a smoke control device, input and output devices, a network access device, etc.

[0155] Reference Figure 2 On the one hand, the embodiment of the present application provides an endoscopic image processing method, the method comprising steps S100-S600:

[0156] Step S100: Acquire endoscopic image data, where the endoscopic image data includes an endoscopic image;

[0157] Step S200: performing a reflective region recognition process on the endoscope image to determine the reflective region in the endoscope image;

[0158] Step S300: determining brightness information of the reflective area from the endoscopic image;

[0159] Step S400: determining the specular reflection component of the reflective area from the endoscopic image;

[0160] Step S500: determining the diffuse reflection component of the reflective area according to the brightness information of the reflective area and the specular reflection component;

[0161] Step S600: performing de-reflection processing on the reflective area of ​​the endoscopic image according to the diffuse reflection component to obtain a target endoscopic image.

[0162] It should be noted that the method can be applied to an endoscope host or other terminals for processing images.

[0163] According to an endoscopic image processing method provided by an embodiment of the present invention, reflective area recognition processing is performed on the endoscopic image, the reflective area in the endoscopic image is determined, the misprocessing of the non-reflective area is avoided, the naturalness and accuracy of the processing effect are ensured, the mirror reflection component of the reflective area is determined from the endoscopic image, the brightness information of the reflective area is further obtained, and the diffuse reflection component of the reflective area is determined based on the brightness information and the mirror reflection component. Finally, the reflective area of ​​the endoscopic image is de-reflected according to the diffuse reflection component to obtain the target endoscopic image. In addition, since the de-reflection processing is only performed on the reflective area in the image instead of performing complex iterative calculations on the entire image, this method can greatly reduce the amount of calculation, improve the processing speed, and ensure that the endoscope can output clear, non-reflective images in a timely manner during operation.

[0164] It is understandable that after removing the reflection, the details of the lesions, blood vessels, tissues, etc. in the image can be displayed more clearly, providing doctors with a more accurate basis for diagnosis and helping to improve the accuracy and reliability of diagnosis.

[0165] It can be understood that the diffuse reflection component represents the actual reflection condition of the object surface. De-reflection processing of the reflective area of ​​the endoscope image using the diffuse reflection component can enhance the contrast and clarity of the image.

[0166] It is understandable that when de-reflecting the entire image, the details of other non-reflective areas in the image may be over-processed or lost due to algorithm complexity or improper parameter settings. However, processing only the reflective areas can better preserve the original details and color information of other areas in the image, making the processed image more natural and realistic, reducing the amount of calculation, and increasing the processing speed, while maintaining low cost, high real-time performance and stability.

[0167] It should be noted that, in step S500, the difference between the brightness information of the reflective area and the specular reflection component is the diffuse reflection component of the reflective area.

[0168] In some embodiments, the step S200 performs reflective region recognition processing on the endoscopic image to determine the reflective region in the endoscopic image, including one of the following:

[0169] First, the reflective points of the endoscopic image are determined, and the reflective area in the endoscopic image is determined according to the distribution of the reflective points in the endoscopic image.

[0170] Secondly, according to the distribution of the brightness value of the endoscopic image, the reflective area in the endoscopic image is determined.

[0171] It should be noted that by extracting the brightness value of the endoscopic image, the endoscopic image can be converted from the RGB color space to the grayscale space or the V channel in the HSV (hue, saturation, brightness) space, which makes it easier to extract the brightness value. Furthermore, the pixels with brightness values ​​higher than the set threshold are regarded as reflective points, and the complete reflective area is determined according to the distribution of the reflective points.

[0172] Third, the reflective area in the endoscopic image is determined according to the distribution of the values ​​of different color channels in the endoscopic image.

[0173] It is understandable that different color channels have different sensitivities to reflections, and according to the distribution of values ​​of different color channels in the endoscopic image, corresponding thresholds are set to identify reflection points to determine the reflection area in the endoscopic image.

[0174] Fourthly, the endoscopic image is input into the reflective area recognition model for prediction processing to obtain the reflective area in the endoscopic image.

[0175] It should be noted that the reflective area recognition model is obtained by collecting a large number of endoscopic images containing reflective area annotations as a training set, and training them using a machine learning or deep learning algorithm (such as a convolutional neural network (CNN)).

[0176] It can be understood that the reflective area recognition model can handle complex and varied reflective situations, including reflective areas of different shapes, sizes and positions.

[0177] In some embodiments, determining the brightness information of the reflective area from the endoscopic image in step S300 includes: calculating the brightness value of each pixel in the reflective area, and performing statistics on the brightness values ​​of the reflective area.

[0178] In some embodiments, determining the specular reflection component of the reflective area from the endoscopic image in step S400 includes one of the following:

[0179] First, obtain the neighborhood image information of the reflective area in the endoscope image, and compare the image information in the reflective area with the neighborhood image information to obtain the specular reflection component of the reflective area;

[0180] It should be noted that the image in the reflective area is mainly caused by specular reflection, while the image information in its neighborhood, i.e. the non-reflective area, reflects more of the diffuse reflection component information of the object. Therefore, by comparing the image information in the reflective area with its neighborhood image information, the specular reflection component of the reflective area can be separated.

[0181] Secondly, the endoscope image and the image information of the reflective area are input into the specular reflection component recognition model for prediction processing to obtain the specular reflection component;

[0182] It should be noted that the image information of the reflective area may be the image of the reflective area itself, or the image information of the reflective area may be the position information (eg boundary information) of the reflective area.

[0183] It should be noted that after the endoscopic image and the image of the reflective area are input into the specular reflection component recognition model for predictive processing, the specular reflection component is obtained, wherein the image portion containing the reflective area retains the complete image data of the reflective area, including detailed information such as brightness, color, and texture, so as to more comprehensively understand the characteristics of the reflective area. This method is suitable for situations where the characteristics of the reflective area are complex and difficult to accurately describe only by position information.

[0184] It should be noted that after the endoscopic image and the position information of the reflective area (such as boundary information) are input into the mirror reflection component recognition model for predictive processing, the mirror reflection component is obtained. It is only necessary to obtain the position and shape of the reflective area, which reduces the amount of data processing and improves the processing speed.

[0185] Thirdly, the image information of the reflective area is input into the specular reflection component recognition model for prediction processing to obtain the specular reflection component.

[0186] It should be noted that the image information of the reflective area may be the image of the reflective area itself, or the image information of the reflective area may be the brightness information of the reflective area.

[0187] It can be understood that inputting the image information of the reflective area into the mirror reflection component recognition model for predictive processing reduces the data processing amount of the mirror reflection component recognition model and improves the processing speed compared to the method that requires adding endoscopic images for processing.

[0188] In some embodiments, the image information of the reflective area includes at least one of the following: image data of the reflective area; brightness information of the reflective area; and reflectivity information of the reflective area.

[0189] It can be understood that when the image information of the reflective area is the reflectivity information of the reflective area, different image processing strategies can be adopted according to the different degrees of reflectivity. For example, for areas with higher degrees of reflectivity, more complex de-reflection processing may be required; while for areas with lower degrees of reflectivity, only simple brightness adjustment or enhancement may be required.

[0190] Reference Figure 3 In some embodiments, the endoscopic image data includes multiple frames of endoscopic images, and step S400 of determining the specular reflection component of the reflective area from the endoscopic image includes steps S410-S420:

[0191] Step S410: acquiring historical image information of historical endoscopic images, where the historical endoscopic images are historical frame endoscopic images in the endoscopic image data;

[0192] It can be understood that an endoscopic image of a historical frame is extracted from the endoscopic image data, at least one frame is extracted, and historical image information related to the historical endoscopic image is obtained, which information includes at least one of the following: a mirror reflection component of the historical endoscopic image; image data of the historical endoscopic image; image data of the reflective area in the historical endoscopic image; a non-reflective endoscopic image after de-reflection processing of the historical endoscopic image; information on the degree of reflection in the historical endoscopic image; information on the degree of reflection in the reflective area in the historical endoscopic image.

[0193] It should be noted that the specular reflection component is obtained by previously identifying and processing the specular reflection component of the historical endoscopic image, and is saved as reference data for subsequent processing.

[0194] It should be noted that the image data of the historical endoscopic image may be the image data of the previous endoscopic image frame of the current frame. Since the endoscopic image of the current frame and the previous endoscopic image are images of the same tissue at different times, there are differences in the image data of the historical endoscopic image and the current frame, which can be used for comparison and analysis.

[0195] It should be noted that the image data of the reflective area in the historical endoscopic image is the image data of the same position as the reflective area identified in the current frame. There is a difference between the image data of the reflective area in the historical endoscopic image and the image data of the reflective area in the endoscopic image of the current frame, which can be used for comparison and analysis.

[0196] Step S420: determining the specular reflection component of the reflective area according to the historical image information and the image information of the reflective area.

[0197] It should be noted that the traditional endoscopic image processing method is only based on the current frame image, without taking into account the temporal correlation and spatial continuity between consecutive frames in the video sequence. When the reflective area is large or the changes are complex, it is difficult to accurately estimate the specific degree and distribution of the reflection. The mirror reflection component of the reflective area obtained based on the current frame image information may not be calculated accurately, which can easily cause the picture displayed on the endoscopic image display device to flicker before and after.

[0198] It can be understood that by introducing historical image information and determining the mirror reflection component of the reflective area based on the historical image information and the image information of the reflective area, even if the reflective information in the current frame is not sufficient or there are errors, it can be compensated and corrected through the information of the historical frame, so that the mirror reflection component of the reflective area can be estimated more accurately. The changes in the reflective area in consecutive frames have a certain regularity and continuity, thereby avoiding the occurrence of video flickering.

[0199] Reference Figure 4 In some embodiments, determining the specular reflection component of the reflective area according to the historical image information and the image information of the reflective area in step S420 includes steps S421-S422:

[0200] Step S421: comparing and obtaining inter-frame difference information between the endoscopic image and the historical endoscopic image, wherein the inter-frame difference information includes at least one of the following: first brightness difference information between the endoscopic image and the historical endoscopic image; second brightness difference information of the reflective area between the endoscopic image and the historical endoscopic image; reflective degree difference information between the reflective degree information of the endoscopic image and the reflective degree information of the historical endoscopic image;

[0201] It should be noted that the first brightness difference information reflects the brightness difference between the current frame and the historical frame in the entire endoscopic image area, providing a global perspective to understand the changes in the overall brightness of the image, where the historical endoscopic image can be the previous frame image of the current frame image.

[0202] It should be noted that the second brightness difference information reflects the brightness difference of the reflective area between the endoscopic image and the historical endoscopic image, focuses on the reflective area, and directly reflects the change of the reflective phenomenon, wherein the historical endoscopic image may be the previous frame image of the current frame image.

[0203] It should be noted that comparing the information on the degree of reflection in the current frame with that in the historical frames and quantifying the degree of change in the reflection phenomenon can help to more accurately evaluate the impact of reflection on image quality.

[0204] Step S422: determining the specular reflection component of the reflective area according to the image information of the reflective area, and the inter-frame difference information and / or the historical image information.

[0205] Preferably, the specular reflection component of the reflective area can be obtained by inputting the image information of the reflective area, and the inter-frame difference information and / or the historical image information into the specular reflection component recognition model for prediction processing.

[0206] In some embodiments, determining the specular reflection component of the reflective area according to the historical image information and the image information of the reflective area in step S420 includes one of the following:

[0207] First, the historical image information and the image information of the reflective area are input into the specular reflection component recognition model for prediction processing to obtain the specular reflection component;

[0208] It should be noted that before inputting the image information into the specular reflection component recognition model, the historical endoscopic images (including their specular reflection components, image data, image data of the reflective area, etc.) and the image information of the reflective area in the current endoscopic image are also organized into a data format acceptable to the model.

[0209] It can be understood that by simultaneously inputting the historical image information and the image information of the reflective area into the specular reflection component recognition model, the specular reflection component can be automatically and quickly determined, thereby improving the processing speed.

[0210] Second: First, determine the first specular reflection component of the reflective area from the endoscopic image; second, determine the second specular reflection component of the reflective area from the historical endoscopic image; finally, based on the historical image information, fuse the first specular reflection component and the second specular reflection component to obtain the specular reflection component.

[0211] It should be noted that the first specular reflection component represents the light intensity distribution generated by specular reflection in the reflective area of ​​the current endoscopic image.

[0212] It should be noted that the second specular reflection component represents the light intensity distribution generated by specular reflection in the determined reflective area in the historical endoscopic image.

[0213] It can be understood that based on the historical image information, the first and second mirror reflection components are fused, and the fusion process takes into account the changes between consecutive frames. Therefore, a smoother transition effect can be achieved when processing reflective areas, which helps to reduce the occurrence of video flicker and improve the visual quality and diagnostic accuracy of endoscopic images.

[0214] It can be understood that the step of fusing the first specular reflection component and the second specular reflection component based on the historical image information to obtain the specular reflection component is a closed-loop image processing step with feedback during the entire endoscope operation process. Specifically, the endoscopic image of the current frame is a historical endoscopic image of the endoscopic image of the next frame. The first specular reflection component determined from the endoscopic image of the current frame will be used as the second specular reflection component in the de-reflection processing of the endoscopic image of the next frame, which can ensure that a clear and stable video image is output during the entire endoscope use process.

[0215] In some embodiments, the fusion processing includes weighted summation processing according to a weighting coefficient, wherein the weighting coefficient is obtained according to at least one of the following parameters: brightness information of the endoscopic image; brightness information of the reflective area in the endoscopic image; historical brightness information of historical endoscopic images; historical brightness information of the reflective area in historical endoscopic images; first brightness difference information between the endoscopic image and the historical endoscopic images; second brightness difference information of the reflective area between the endoscopic image and the historical endoscopic images; reflectivity information of the endoscopic image; reflectivity information of the historical endoscopic images; reflectivity difference information between the reflectivity information of the endoscopic image and the reflectivity information of the historical endoscopic images; reflection intensity distribution information of the endoscopic image; reflection intensity distribution information of the reflective area of ​​the endoscopic image; historical reflection intensity distribution information of the reflective area of ​​the historical endoscopic images.

[0216] In some embodiments, determining a first specular reflection component of a reflective region from an endoscopic image includes:

[0217] After the image information of the reflective area is input into the specular reflection component recognition model for prediction processing, the first specular reflection component is obtained.

[0218] In some embodiments, determining the second specular reflection component of the reflective region from the historical endoscopic image comprises one of the following:

[0219] First, the image information of the reflective area of ​​the historical endoscope image is input into the specular reflection component recognition model for prediction processing to obtain the second specular reflection component;

[0220] Second, obtaining the second specular reflection component saved after the reflective area of ​​the historical endoscopic image has been predicted and processed by the specular reflection component recognition model.

[0221] It should be noted that if the specular reflection component of the historical endoscopic image has been predicted and processed before, and the prediction results have been saved, then these pre-stored results can be directly obtained as the second specular reflection component, avoiding repeated calculations and improving processing efficiency.

[0222] Reference Figure 5 In some embodiments, step S600 performs de-reflection processing on the reflective area of ​​the endoscopic image according to the diffuse reflection component to obtain a target endoscopic image, including steps S610-S620:

[0223] Step S610: performing de-reflection processing on the reflective area of ​​the endoscope image according to the diffuse reflection component to obtain a local endoscope image corresponding to the reflective area;

[0224] Step S620: Replace the reflective area in the endoscopic image with the local endoscopic image to obtain a target endoscopic image.

[0225] Preferably, the local endoscopic image is replaced by the reflective area in the endoscopic image through image fusion technology. Specifically, the boundary pixels are gradually processed and an image fusion algorithm is used to achieve a smooth transition of the replaced image at the boundary to avoid obvious stitching marks.

[0226] It can be understood that after the replacement and adjustment are completed, the target endoscopic image can be output, and the reflective area in the image has been effectively removed or weakened, thereby improving the clarity and readability of the image.

[0227] In some embodiments, determining the specular reflection component of the reflective area from the endoscopic image in step S400 includes:

[0228] After down-sampling the image information of the reflective area, first image information of the reflective area is obtained, and the specular reflection component of the reflective area is determined from the first image information.

[0229] It can be understood that, through downsampling processing, the resolution of the image in the reflective area can be reduced, the amount of processed data can be reduced, and the processing speed can be increased, while the main features of the image can be retained.

[0230] It should be noted that determining the specular reflection component of the reflective area from the first image information can be achieved through a variety of methods, such as using a specular reflection component recognition model for prediction, or analyzing and calculating based on image statistical characteristics (such as brightness, contrast, etc.).

[0231] Furthermore, after determining the mirror reflection component of the reflective area from the first image information, the brightness information of the reflective area is obtained, and the diffuse reflection component of the reflective area is determined based on the brightness information and the mirror reflection component; the reflective area of ​​the endoscopic image is de-reflected based on the diffuse reflection component to obtain a local endoscopic image corresponding to the reflective area; the local endoscopic image replaces the reflective area in the endoscopic image to obtain a target endoscopic image.

[0232] It should be noted that after downsampling the image information of the reflective area, first image information of the reflective area is obtained, and the first image information is input into the specular reflection component recognition model to obtain the real-time specular reflection component of the reflective area end-to-end.

[0233] It should be noted that different network structures can also be designed to obtain the specular reflection component, such as using MLP, Transformer and other network structures, adding Attention mechanism network structure to the network structure, using LeakyReLU and other activation units in the network to improve the stability and performance of the model; in addition, the de-reflection algorithm can be combined with other imaging to obtain the specular reflection component, such as narrow-band light reflection imaging.

[0234] In some embodiments, replacing the reflective area in the endoscopic image with the local endoscopic image to obtain the target endoscopic image includes:

[0235] After the local endoscopic image is upsampled, the reflective area in the endoscopic image is replaced to obtain the target endoscopic image.

[0236] It is understandable that since some detail information may be lost during the downsampling process, the determined specular reflection component may be an approximation or a generalized representation. Therefore, the local endoscopic image needs to be upsampled to match the size and resolution of the reflective area in the original endoscopic image.

[0237] It is understandable that some interpolation errors or blurring effects may be introduced during the upsampling process, so it is necessary to select appropriate upsampling algorithms and parameters according to actual conditions.

[0238] On the other hand, an embodiment of the present application provides another endoscopic image processing method, the method comprising the following steps:

[0239] First, endoscopic image data is acquired, wherein the endoscopic image data includes an endoscopic image;

[0240] secondly, determining the specular component of the endoscopic image from the endoscopic image;

[0241] Next, the brightness information of the endoscope image is obtained, and the diffuse reflection component of the endoscope image is determined according to the brightness information and the specular reflection component;

[0242] Finally, the endoscopic image is de-reflected according to the diffuse reflection component to obtain the target endoscopic image.

[0243] According to an endoscopic image processing method provided by an embodiment of the present invention, first, endoscopic image data is acquired, and the endoscopic image data includes an endoscopic image, ensuring that subsequent processing has a reliable data basis. Secondly, the mirror reflection component of the endoscopic image is determined from the endoscopic image. Then, the brightness information of the endoscopic image is acquired, and the diffuse reflection component of the endoscopic image is determined based on the brightness information and the mirror reflection component. Finally, the endoscopic image is de-reflected based on the diffuse reflection component to obtain a target endoscopic image. Since the mirror reflection component of the endoscopic image is determined directly from the endoscopic image, the image processing steps are simplified, ensuring that the endoscope can output clear, non-reflective images in a timely manner during operation.

[0244] It should be noted that the endoscopic image data is original image data containing target tissues or organs captured and obtained by an endoscopic device.

[0245] It can be understood that the diffuse reflection component represents the actual reflection condition of the object surface. De-reflection processing of the reflective area of ​​the endoscope image using the diffuse reflection component can enhance the contrast and clarity of the image.

[0246] It should be noted that the difference between the brightness information of the reflective area and the specular reflection component is the diffuse reflection component of the reflective area.

[0247] In some embodiments, determining a specular component of the endoscopic image from the endoscopic image comprises one of the following:

[0248] First, the reflection intensity distribution information is obtained from the endoscopic image, the neighborhood image information of the reflective area is determined according to the reflection intensity distribution information, and the image information in the reflective area is compared with the neighborhood image information to obtain the specular reflection component of the reflective area.

[0249] It should be noted that, first, a global brightness analysis is performed on the endoscopic image to obtain the distribution of reflection intensity in the image, for example, by calculating the brightness value of each pixel or area in the image and generating a corresponding brightness distribution map. Further, the area with stronger reflection is determined to be the reflective area, and the image information of one or more non-reflective or weakly reflective areas adjacent to the reflective area is selected, and the image information in the reflective area is compared with the neighboring image information to obtain the mirror reflection component of the reflective area.

[0250] It should be noted that by comparing the image information in the reflective area with the image information of the selected reflective area, the part of the reflective area that is distorted due to specular reflection, i.e., the specular reflection component, can be inferred by comparing the brightness, color, texture and other features of the two.

[0251] Second, the endoscope image is input into the specular reflection component recognition model for prediction processing to obtain the specular reflection component.

[0252] Preferably, the model can be constructed based on a deep learning algorithm, such as a convolutional neural network (CNN).

[0253] It can be understood that in the model training stage, a large amount of labeled endoscopic image data is used as a training set, and the model is trained through supervised learning to identify the specular reflection component in the image. Once the model training is completed, the specular reflection component of the new image can be automatically identified without human intervention. It has a fast processing speed and can complete the processing of a large number of images in a short time. Moreover, with the increase of training data and the continuous optimization of the model, the recognition accuracy and generalization ability can be continuously improved.

[0254] It can be understood that when the endoscope image to be processed is input into the trained specular reflection component recognition model, the specular reflection component recognition model will automatically recognize and predict the specular reflection component in the image.

[0255] In some embodiments, determining a specular component of an endoscopic image from an endoscopic image comprises the following steps:

[0256] First, historical image information of historical endoscopic images is acquired, where the historical endoscopic images are historical frame endoscopic images in the endoscopic image data.

[0257] It can be understood that an endoscopic image of a historical frame is extracted from the endoscopic image data, at least one frame is extracted, and historical image information related to the historical endoscopic image is obtained, and the historical image information includes at least one of the following: a mirror reflection component of the historical endoscopic image; image data of the historical endoscopic image; a non-reflective endoscopic image after de-reflection processing of the historical endoscopic image; and information on the degree of reflection in the historical endoscopic image.

[0258] It should be noted that the specular reflection component is obtained by previously identifying and processing the specular reflection component of the historical endoscopic image, and is saved as reference data for subsequent processing.

[0259] It should be noted that the image data of the historical endoscopic image may be the image data of the previous endoscopic image frame of the current frame, and the endoscopic image of the current frame and the previous endoscopic image are images of the same tissue at different times. There are differences in the image data between the historical endoscopic image and the current frame, which can be used for comparison and analysis.

[0260] Secondly, the specular reflection component of the reflective area is determined based on the historical image information and the endoscopic image.

[0261] It should be noted that the traditional endoscopic image processing method is only based on the current frame image, without taking into account the temporal correlation and spatial continuity between consecutive frames in the video sequence. When the reflective area in the endoscopic image is large or the changes are complex, it is difficult to accurately estimate the specific degree and distribution of the reflection. The mirror reflection component of the reflective area obtained based on the current frame image information is not calculated accurately, which can easily cause the picture displayed on the endoscopic image display device to flicker before and after.

[0262] It can be understood that by introducing historical image information and determining the mirror reflection component of the endoscopic image based on the historical image information and the image information of the endoscopic image, even if the reflection information in the current frame is not sufficient or there are errors, it can be compensated and corrected through the information of the historical frame. The mirror reflection component of the reflective area can be estimated more accurately, and the changes in the reflective area in consecutive frames have a certain regularity and continuity, thereby avoiding the occurrence of video flicker.

[0263] In some embodiments, determining the specular reflection component of the endoscopic image based on the historical image information and the endoscopic image includes:

[0264] First, inter-frame difference information between the endoscopic image and the historical endoscopic image is compared and obtained, wherein the inter-frame difference information includes at least one of the following:

[0265] first brightness difference information between the endoscopic image and the historical endoscopic image;

[0266] The reflection degree difference information between the reflection degree information of the endoscopic image and the reflection degree information of the historical endoscopic images.

[0267] It should be noted that the first brightness difference information reflects the brightness difference between the current frame and the historical frame in the entire endoscopic image area, providing a global perspective to understand the changes in the overall brightness of the image, where the historical endoscopic image can be the previous frame image of the current frame image.

[0268] It should be noted that comparing the information on the degree of reflection in the current frame with that in the historical frames and quantifying the degree of change in the reflection phenomenon can help to more accurately evaluate the impact of reflection on image quality.

[0269] Secondly, the specular reflection component is determined based on the endoscopic image, and inter-frame difference information and / or historical image information.

[0270] Preferably, the specular reflection component can be obtained by inputting the endoscopic image, inter-frame difference information and / or historical image information into a specular reflection component recognition model for prediction processing.

[0271] In some embodiments, determining the specular reflection component of the reflective area based on the historical image information and the endoscopic image includes at least one of the following:

[0272] First, the historical image information and the endoscope image are input into the specular reflection component recognition model for prediction processing to obtain the specular reflection component;

[0273] It should be noted that before inputting the image information into the specular reflection component recognition model, the historical endoscopic images (including their specular reflection components, image data, image data of the reflective area, etc.) and the image information of the reflective area in the current endoscopic image are also organized into a data format acceptable to the model.

[0274] It can be understood that by simultaneously inputting the historical image information and the image information of the reflective area into the specular reflection component recognition model, the specular reflection component can be automatically and quickly determined, thereby improving the processing speed. Second: First, determine the first specular reflection component from the endoscopic image;

[0275] Secondly, a second specular reflection component is determined from the historical endoscopic images;

[0276] Finally, based on the historical image information of the historical endoscopic images, the first specular reflection component and the second specular reflection component are fused to obtain the specular reflection component.

[0277] It should be noted that the first specular reflection component represents the light intensity distribution generated by specular reflection in the current endoscopic image.

[0278] It should be noted that the second specular reflection component represents the light intensity distribution determined in the historical endoscopic image to be generated due to specular reflection.

[0279] It can be understood that based on historical image information (such as the degree of reflection, brightness changes, etc.), the first and second mirror reflection components are fused, and the fusion process takes into account the changes between consecutive frames. Therefore, a smoother transition effect can be achieved when processing reflective areas, which helps to reduce the occurrence of video flicker and improve the visual quality and diagnostic accuracy of endoscopic images.

[0280] It can be understood that the step of fusing the first specular reflection component and the second specular reflection component based on historical image information to obtain the specular reflection component is a closed-loop image processing step with feedback during the entire endoscopic operation process. Specifically, the endoscopic image of the current frame is a historical endoscopic image of the endoscopic image of the next frame, and the first specular reflection component determined from the endoscopic image of the current frame will be used as the second specular reflection component in the de-reflection processing of the endoscopic image of the next frame.

[0281] In some embodiments, the fusion process includes performing a weighted summation process according to a weighting coefficient, wherein the weighting coefficient is obtained according to at least one of the following parameters:

[0282] Brightness information of endoscopic images;

[0283] Historical brightness information of historical endoscopic images;

[0284] first brightness difference information between the endoscopic image and the historical endoscopic image;

[0285] Information on the reflectivity of the endoscope image;

[0286] Reflectivity difference information between the reflectivity information of the endoscopic image and the reflectivity information of the historical endoscopic images;

[0287] Reflection intensity distribution information of endoscope images

[0288] Historical reflection intensity distribution information of historical endoscopic images.

[0289] In some embodiments, determining a first specular reflection component from an endoscopic image includes:

[0290] After the endoscope image is input into the specular reflection component recognition model for prediction processing, the first specular reflection component is obtained.

[0291] In some embodiments, determining the second specular component from the historical endoscopic image comprises one of:

[0292] First, the historical endoscope image is input into the specular reflection component recognition model for prediction processing to obtain the second specular reflection component;

[0293] Secondly, the second specular reflection component saved after the historical endoscopic image has been predictively processed by the specular reflection component recognition model is obtained.

[0294] It should be noted that if the specular reflection component of the historical endoscopic image has been predicted and processed before, and the prediction results have been saved, then these pre-stored results can be directly obtained as the second specular reflection component, avoiding repeated calculations and improving processing efficiency.

[0295] On the other hand, an embodiment of the present application provides another endoscopic image processing method, the method comprising:

[0296] Acquiring endoscopic image data, the endoscopic image data comprising an endoscopic image;

[0297] After performing de-reflection processing on the endoscopic image, a first endoscopic image is obtained;

[0298] Acquire historical image information of historical endoscopic images, where the historical endoscopic images are historical frame endoscopic images in the endoscopic image data;

[0299] Based on the historical image information of the historical endoscopic images, the first endoscopic image is smoothed to obtain a target endoscopic image.

[0300] According to an endoscopic image processing method provided by an embodiment of the present invention, endoscopic image data is acquired, the endoscopic image data includes an endoscopic image, a first endoscopic image is obtained after de-reflection processing is performed on the endoscopic image, historical image information of historical endoscopic images is acquired, the historical endoscopic images are historical frame endoscopic images in the endoscopic image data, and based on the historical image information of the historical endoscopic images, the first endoscopic image is smoothed to obtain a target endoscopic image, wherein the final endoscopic image is clearer and smoother than the original endoscopic image, and is more helpful for diagnosis and analysis, and in addition, the smoothing processing based on the historical data helps to reduce mutations between image frames, thereby avoiding flickering in the video.

[0301] It should be noted that smoothing can be used to reduce noise and incoherent pixel changes in an image, making the image appear smoother and more coherent.

[0302] It can be understood that an endoscopic image of a historical frame is extracted from the endoscopic image data, at least one frame is extracted, and historical image information related to the historical endoscopic image is obtained, and the historical image information includes at least one of the following: a mirror reflection component of the historical endoscopic image; image data of the historical endoscopic image; a non-reflective endoscopic image after de-reflection processing of the historical endoscopic image; reflection degree information in the historical endoscopic image; first brightness difference information between the endoscopic image and the historical endoscopic image; reflection degree difference information between the reflection degree information of the endoscopic image and the reflection degree information of the historical endoscopic image.

[0303] It should be noted that the specular reflection component is obtained by previously identifying and processing the specular reflection component of the historical endoscopic image, and is saved as reference data for subsequent processing.

[0304] It should be noted that the image data of the historical endoscopic image may be the image data of the previous endoscopic image frame of the current frame, and the endoscopic image of the current frame and the previous endoscopic image are images of the same tissue at different times. There are differences in the image data between the historical endoscopic image and the current frame, which can be used for comparison and analysis.

[0305] It should be noted that the first brightness difference information reflects the brightness difference between the current frame and the historical frame in the entire endoscopic image area, providing a global perspective to understand the changes in the overall brightness of the image, where the historical endoscopic image can be the previous frame image of the current frame image.

[0306] It should be noted that comparing the information on the degree of reflection in the current frame with that in the historical frames and quantifying the degree of change in the reflection phenomenon can help to more accurately evaluate the impact of reflection on image quality.

[0307] In some embodiments, after performing de-reflection processing on the endoscopic image, a first endoscopic image is obtained, including:

[0308] After the endoscopic image is input into the de-reflection model for de-reflection processing, a first endoscopic image is obtained.

[0309] It should be noted that the endoscopic image is input into the de-reflection model for processing. The de-reflection model identifies the image that may contain highlight areas or glare areas caused by light source reflection, and de-reflects the corresponding reflective areas to obtain a first endoscopic image without reflective areas.

[0310] In some embodiments, the historical image information includes image data of a historical endoscopic image or a historical non-reflective endoscopic image after de-reflection processing of the historical endoscopic image; based on the historical image information of the historical endoscopic image, the first endoscopic image is smoothed to obtain a target endoscopic image, including one of the following:

[0311] First, the first endoscopic image and the historical endoscopic image are fused to obtain a target endoscopic image;

[0312] Preferably, the fusion processing may utilize image weighting or image superposition techniques to combine the current endoscopic image with the historical endoscopic image to generate a composite image.

[0313] It can be understood that by fusing the current endoscopic image with the historical endoscopic image, random noise and incoherent changes in the image can be reduced, thereby improving the smoothness and consistency of the image.

[0314] Second, the first endoscopic image and the historical non-reflective endoscopic image are fused to obtain a target endoscopic image.

[0315] Preferably, the fusion process may employ image weighting or image superposition techniques to combine the current endoscopic image with the historical non-reflective endoscopic image to generate a composite image.

[0316] It should be noted that the image quality problem caused by reflection can be further reduced by fusing the historical non-reflection endoscopic image after de-reflection processing with the endoscopic image using the historical endoscopic image.

[0317] In some embodiments, the fusion processing includes performing weighted summation processing according to a weighting coefficient, wherein the weighting coefficient is obtained according to at least one of the following parameters: brightness information of the first endoscopic image; historical brightness information of the historical endoscopic image; first brightness difference information between the first endoscopic image and the historical endoscopic image; reflection degree information of the first endoscopic image; reflection degree difference information between the reflection degree information of the first endoscopic image and the reflection degree information of the historical endoscopic image; reflection intensity distribution information of the first endoscopic image; historical reflection intensity distribution information of the historical endoscopic image;

[0318] Information on the degree of reflection in historical endoscopic images.

[0319] It can be understood that by combining the above parameters to determine the weighting coefficient, the fusion processing can more intelligently balance the difference between the current image and the historical image, while reducing image quality problems caused by reflections and other factors, which can improve the stability and coherence of the image, reduce flicker, and improve the accuracy of diagnosis. In addition, the weighting coefficient can be dynamically adjusted according to the actual situation to adapt to different inspection conditions and image characteristics.

[0320] In some embodiments, during the fusion process, the target endoscopic image is calculated using the following formula:

[0321] y=ypre*W1+y*W2

[0322] Among them, y' is the RGB information of the unit pixel of the target endoscopic image, y is the RGB information of the current frame endoscopic image, ypre is the RGB information of the previous frame historical endoscopic image or historical non-reflective endoscopic image, and W1 and / or W2 are weighting coefficients.

[0323] It should be noted that the sum of the weighting coefficient W1 and the weighting coefficient W2 is 1.

[0324] It can be understood that W1 and W2 can be dynamically adjusted according to the difference between the current frame and the historical frame. If the RGB information of the current frame endoscopic image is significantly different from the RGB information of the previous frame historical endoscopic image or the historical non-reflective endoscopic image, W2 can be appropriately reduced and W1 can be increased, so that the historical image accounts for a larger proportion in the fusion process, thereby achieving a smooth transition.

[0325] It should be noted that the step of fusing the endoscopic image with the historical endoscopic image or the historical non-reflective endoscopic image to obtain the target endoscopic image is a closed-loop image processing step with feedback during the entire endoscopic operation process. Specifically, the RGB information y' of the unit pixel of the target endoscopic image of the current frame is the RGB information ypre of the historical endoscopic image or the historical non-reflective endoscopic image of the next frame of the endoscopic image.

[0326] Reference Figure 6 On the other hand, the embodiment of the present application provides another endoscopic image processing method, the method comprising steps S710 to S770:

[0327] Step S710: Acquire endoscopic image data, where the endoscopic image data includes an endoscopic image;

[0328] Step S720: Acquire the reflective area in the endoscope image;

[0329] Step S730: performing de-reflection processing on the endoscopic image to obtain a first endoscopic image;

[0330] Step S740: acquiring historical image information of historical endoscopic images, where the historical endoscopic images are historical frame endoscopic images in the endoscopic image data;

[0331] Step S750: Based on the historical image information of the historical endoscopic image, the first endoscopic image is smoothed to obtain a second endoscopic image;

[0332] Step S760: extracting a local endoscopic image of the reflective area from the second endoscopic image;

[0333] Step S770: Replace the reflective area in the endoscopic image with the local endoscopic image to obtain the target endoscopic image.

[0334] According to an endoscopic image processing method provided by an embodiment of the present invention, a reflective area in an endoscopic image is obtained to lay a data foundation for subsequently replacing the reflective area in the endoscopic image with a local endoscopic image to obtain a target endoscopic image. Furthermore, after de-reflection processing is performed on the endoscopic image, a first endoscopic image is obtained; historical image information of historical endoscopic images is obtained, and the historical endoscopic images are historical frame endoscopic images in the endoscopic image data; based on the historical image information of the historical endoscopic images, the first endoscopic image is smoothed to obtain a second endoscopic image, wherein, compared with the unprocessed endoscopic image, the clarity of the local endoscopic image of the reflective area in the second endoscopic image obtained after the de-reflection processing and smoothing processing is well improved, and the local endoscopic image of the reflective area is extracted from the second endoscopic image; the local endoscopic image of the reflective area in the endoscopic image is replaced by the local endoscopic image to obtain the target endoscopic image, ensuring that only the area affected by the reflection is further processed and replaced, thereby improving the overall quality of the image.

[0335] In some embodiments, after de-reflection processing is performed on the endoscopic image in step S730, a first endoscopic image is obtained, including:

[0336] After the endoscopic image is input into the de-reflection model for de-reflection processing, a first endoscopic image is obtained.

[0337] It should be noted that the endoscopic image is input into the de-reflection model for processing. The de-reflection model identifies the image that may contain highlight areas or glare areas caused by light source reflection, and de-reflects the corresponding reflective areas to obtain a first endoscopic image without reflective areas.

[0338] In some embodiments, after inputting the endoscopic image into a de-reflection model for de-reflection processing, obtaining a first endoscopic image includes the following steps:

[0339] First, after downsampling the endoscopic image, first image information of the endoscopic image is obtained;

[0340] It can be understood that through downsampling processing, the resolution of the image in the reflective area can be reduced, the amount of processed data can be reduced, and the processing speed can be increased while retaining the main features of the image.

[0341] Secondly, the first image information is input into a de-reflection model for de-reflection processing to obtain a first endoscopic image.

[0342] It should be noted that after obtaining the first endoscopic image, historical image information of the historical endoscopic image is obtained, and the historical endoscopic image is the historical frame endoscopic image in the endoscopic image data; then, based on the historical image information of the historical endoscopic image, the first endoscopic image is smoothed to obtain the second endoscopic image, and a local endoscopic image of the reflective area is extracted from the second endoscopic image; finally, the local endoscopic image replaces the reflective area in the endoscopic image to obtain the target endoscopic image.

[0343] In some embodiments, in step S770, replacing the reflective area in the endoscopic image with the local endoscopic image to obtain the target endoscopic image includes:

[0344] After the local endoscopic image is upsampled, the reflective area in the endoscopic image is replaced to obtain the target endoscopic image.

[0345] It is understandable that, since some detail information may be lost during the downsampling process, the local endoscopic image needs to be upsampled to match the size and resolution of the reflective area in the original endoscopic image.

[0346] It is understandable that some interpolation errors or blurring effects may be introduced during the upsampling process, so it is necessary to select appropriate upsampling algorithms and parameters according to actual conditions.

[0347] In some embodiments, the historical image information includes at least one of the following: a mirror reflection component of a historical endoscopic image; image data of a historical endoscopic image; a reflection-free endoscopic image after de-reflection processing of the historical endoscopic image; reflection degree information in the historical endoscopic image; first brightness difference information between an endoscopic image and a historical endoscopic image; reflection degree difference information between the reflection degree information of an endoscopic image and the reflection degree information of a historical endoscopic image.

[0348] In some embodiments, the historical image information includes image data of a historical endoscopic image or a historical non-reflective endoscopic image after de-reflection processing of the historical endoscopic image; based on the historical image information of the historical endoscopic image, the first endoscopic image is smoothed to obtain a target endoscopic image, including one of the following:

[0349] First, the endoscopic image and the historical endoscopic image are fused to obtain the target endoscopic image;

[0350] Preferably, the fusion processing may utilize image weighting or image superposition techniques to combine the current endoscopic image with the historical endoscopic image to generate a composite image.

[0351] It can be understood that by fusing the current endoscopic image with the historical endoscopic image, random noise and incoherent changes in the image can be reduced, thereby improving the smoothness and consistency of the image.

[0352] Second, the endoscopic image and the historical non-reflective endoscopic image are fused to obtain the target endoscopic image.

[0353] Preferably, the fusion process may employ image weighting or image superposition techniques to combine the current endoscopic image with the historical non-reflective endoscopic image to generate a composite image.

[0354] It should be noted that the image quality problem caused by reflection can be further reduced by fusing the historical non-reflection endoscopic image after de-reflection processing with the endoscopic image using the historical endoscopic image.

[0355] In some embodiments, the fusion processing includes weighted summation processing according to a weighting coefficient, wherein the weighting coefficient is obtained according to at least one of the following parameters: brightness information of the endoscopic image; historical brightness information of historical endoscopic images; first brightness difference information between the endoscopic image and the historical endoscopic images; reflection degree information of the endoscopic image; reflection degree difference information between the reflection degree information of the endoscopic image and the reflection degree information of the historical endoscopic images; reflection intensity distribution information of the endoscopic image; historical reflection intensity distribution information of historical endoscopic images; reflection degree information in historical endoscopic images.

[0356] It can be understood that by combining the above parameters to determine the weighting coefficient, the fusion processing can more intelligently balance the difference between the current image and the historical image, while reducing image quality problems caused by reflections and other factors, which can improve the stability and coherence of the image, reduce flicker, and improve the accuracy of diagnosis. In addition, the weighting coefficient can be dynamically adjusted according to the actual situation to adapt to different inspection conditions and image characteristics.

[0357] In some embodiments, during the fusion process, the target endoscopic image is calculated using the following formula:

[0358] y'=ypre*W1+y*W2

[0359] Among them, y' is the RGB information of the unit pixel of the target endoscopic image, y is the RGB information of the current frame endoscopic image, ypre is the RGB information of the previous frame historical endoscopic image or historical non-reflective endoscopic image, and W1 and / or W2 are weighting coefficients.

[0360] It should be noted that the sum of the weighting coefficient W1 and the weighting coefficient W2 is 1.

[0361] It can be understood that W1 and W2 can be dynamically adjusted according to the difference between the current frame and the historical frame. If the RGB information of the current frame endoscopic image is significantly different from the RGB information of the previous frame historical endoscopic image or the historical non-reflective endoscopic image, W2 can be appropriately reduced and W1 can be increased, so that the historical image accounts for a larger proportion in the fusion process, thereby achieving a smooth transition.

[0362] It should be noted that the step of fusing the endoscopic image with the historical endoscopic image or the historical non-reflective endoscopic image to obtain the target endoscopic image is a closed-loop image processing step with feedback during the entire endoscopic operation process. Specifically, the RGB information y' of the unit pixel of the target endoscopic image of the current frame is the RGB information ypre of the historical endoscopic image or the historical non-reflective endoscopic image of the next frame of the endoscopic image.

[0363] In some embodiments, after de-reflection processing is performed on the endoscopic image in step S730, a first endoscopic image is obtained, including:

[0364] Determining the number of reflective areas in the endoscopic image, and when the number of reflective areas is greater than a preset value, performing de-reflection processing on the endoscopic image to obtain a first endoscopic image;

[0365] When the number of reflective areas is less than or equal to a preset value, the reflective areas of the endoscopic image are extracted and de-reflected to obtain a first partial endoscopic image of the reflective areas.

[0366] It is understandable that when the number of reflective areas is greater than a preset value, it indicates that there is a large amount of reflective interference in the image, and global de-reflection processing is required, that is, de-reflection processing is performed on the endoscopic image to reduce or eliminate the reflection phenomenon in the image.

[0367] It can be understood that when the number of reflective areas is less than or equal to the preset value, it means that the reflection phenomenon in the image is relatively light. These reflective areas can be extracted and subjected to separate de-reflection processing, which can reduce unnecessary calculations and improve processing efficiency. In addition, the first local endoscopic image can be viewed separately or recombined into the original image to improve the overall image quality.

[0368] In some embodiments, when the number of reflective areas is less than or equal to a preset value, the reflective areas of the endoscopic image are extracted for de-reflection processing to obtain a first local endoscopic image of the reflective area, and then, first, based on the local historical image information of the reflective area in the historical endoscopic image, the first local endoscopic image is smoothed to obtain a second local endoscopic image; secondly, the reflective area in the endoscopic image is replaced by the second local endoscopic image to obtain a target endoscopic image.

[0369] It can be understood that since de-reflection processing is only performed on the image information of the reflective area in the endoscopic image and the image information of the reflective area in the historical endoscopic image, rather than performing complex iterative calculations on the entire image, this method can greatly reduce the amount of calculation, improve the processing speed, and ensure that the endoscope can output clear, non-reflection images in a timely manner during operation.

[0370] In some embodiments, local historical image information includes at least one of the following: a mirror reflection component of a reflective area of ​​a historical endoscopic image; image data of a reflective area of ​​a historical endoscopic image; a non-reflective endoscopic image of a reflective area of ​​a historical endoscopic image after de-reflection processing; information on the degree of reflection of the reflective area in the historical endoscopic image; first brightness difference information between the reflective area of ​​the endoscopic image and the reflective area of ​​the historical endoscopic image; and information on the difference in degree of reflection between the reflective area of ​​the endoscopic image and the reflective area of ​​the historical endoscopic image.

[0371] In some embodiments, obtaining the reflective area of ​​the endoscopic image in step S720 includes: performing reflective area recognition processing on the endoscopic image to determine the reflective area in the endoscopic image.

[0372] In some implementations, the endoscopic image is subjected to a reflective region recognition process to determine that the reflective region in the endoscopic image includes one of the following:

[0373] First, determining the reflective points of the endoscopic image, and determining the reflective area in the endoscopic image according to the distribution of the reflective points in the endoscopic image;

[0374] Second, the endoscopic image is input into the reflective area recognition model for prediction processing to obtain the reflective area in the endoscopic image.

[0375] It should be noted that the reflective area recognition model is obtained by collecting a large number of endoscopic images containing reflective area annotations as a training set, and training them using a machine learning or deep learning algorithm (such as a convolutional neural network (CNN)).

[0376] On the other hand, an embodiment of the present application provides an endoscope camera system, comprising:

[0377] Endoscope;

[0378] A light source device, used to provide lighting for the endoscope;

[0379] A camera host is connected to the endoscope, and is used to obtain an image signal of a target object photographed by the endoscope and display an endoscopic image of the target object through a display, wherein the camera host executes an endoscopic image processing method of any one of the above embodiments.

[0380] On the other hand, an embodiment of the present application provides a camera host, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the endoscopic image processing method of any one of the above embodiments is implemented.

[0381] On the other hand, an embodiment of the present application further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned endoscopic image processing method when executing the computer program.

[0382] It should be noted that it also includes a light source module, an endoscope module, an image sensor module, a light source regulator module and an image processor module. The light source module is used to provide an illumination light source; the endoscope module is used to transmit illumination and excitation light and realize endoscopic imaging; the image sensor module is used to collect reflected light signals; the light source regulator module adjusts and controls the light source intensity; and the image processor module processes the collected video signals in real time.

[0383] It can be understood that the processor is used to control the light source regulator and the image processor module, and present the collected and processed video signal to the display device.

[0384] On the other hand, an embodiment of the present application further provides a computer-readable storage medium, the storage medium stores a computer program, and the computer program is executed by a processor to implement the above-mentioned endoscopic image processing method.

[0385] On the other hand, the embodiment of the present application further provides a computer program product, which includes a computer program, and the computer program is stored in a computer-readable storage medium. The processor of the computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the above-mentioned endoscopic image processing method.

[0386] It is to be understood that, although the steps in the above-mentioned various flow charts are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in the present embodiment, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the above-mentioned flow charts can include a plurality of steps or a plurality of stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of the steps or stages in other steps or other steps.

[0387] An embodiment of the present application also provides a computer-readable storage medium, which is used to store a computer program, and the computer program is used to execute the endoscopic image processing method of each of the aforementioned embodiments.

[0388] The embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. The processor of the computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the above-mentioned endoscopic image processing method.

[0389] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate to describe the embodiments of the present application, for example, it can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0390] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0391] It should be understood that in the description of the embodiments of the present application, the meaning of multiple (or multiple items) is more than two, greater than, less than, exceed, etc. are understood to not include the number, and above, below, within, etc. are understood to include the number.

[0392] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0393] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0394] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0395] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0396] It should also be understood that the various implementations provided in the embodiments of the present application can be combined arbitrarily to achieve different technical effects.

[0397] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the above-mentioned implementation mode. Technical personnel familiar with the field can also make various equivalent modifications or substitutions under the shared conditions that do not violate the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. An endoscopic image processing method, characterized in that: The method comprises: Acquiring endoscopic image data, wherein the endoscopic image data includes an endoscopic image; Performing a reflective region recognition process on the endoscopic image to determine a reflective region in the endoscopic image; determining brightness information of the reflective area from the endoscopic image; determining a specular reflection component of the reflective area from the endoscopic image; Determine the diffuse reflection component of the reflective area according to the brightness information of the reflective area and the specular reflection component; The reflective area of ​​the endoscopic image is de-reflected according to the diffuse reflection component to obtain a target endoscopic image.

2. The endoscopic image processing method according to claim 1, characterized in that: Determining the specular reflection component of the reflective area from the endoscopic image comprises one of the following: Acquire neighborhood image information of the reflective area in the endoscopic image, and compare the image information in the reflective area with the neighborhood image information to obtain the specular reflection component of the reflective area; Alternatively, the endoscope image and the image information of the reflective area are input into a specular reflection component recognition model for prediction processing to obtain the specular reflection component; Alternatively, the image information of the reflective area is input into a specular reflection component recognition model for prediction processing to obtain the specular reflection component.

3. The endoscopic image processing method according to claim 2, characterized in that: The image information of the reflective area includes at least one of the following: Image data of the reflective area; Brightness information of the reflective area; The reflective degree information of the reflective area.

4. The endoscopic image processing method according to claim 1, characterized in that: The endoscopic image data includes a plurality of frames of the endoscopic image, and determining the specular reflection component of the reflective area from the endoscopic image includes: Acquire historical image information of historical endoscopic images, wherein the historical endoscopic images are historical frame endoscopic images in the endoscopic image data; Determine the specular reflection component of the reflective area according to the historical image information and the image information of the reflective area; The historical image information includes at least one of the following: a specular component of the historical endoscopic image; image data of the historical endoscopic images; image data of the reflective area in the historical endoscopic image; a non-reflective endoscopic image after the historical endoscopic image has been subjected to de-reflection processing; Information on the degree of reflection in the historical endoscopic images; Information about the degree of reflection of the reflective area in the historical endoscopic image.

5. The endoscopic image processing method according to claim 4, characterized in that: The determining the specular reflection component of the reflective area according to the historical image information and the image information of the reflective area comprises: Comparing and obtaining inter-frame difference information between the endoscopic image and the historical endoscopic image; Determine the specular reflection component of the reflective area according to the image information of the reflective area, the inter-frame difference information and / or the historical image information; The inter-frame difference information includes at least one of the following: first brightness difference information between the endoscopic image and the historical endoscopic image; second brightness difference information of the reflective area between the endoscopic image and the historical endoscopic image; The reflection degree difference information between the reflection degree information of the endoscopic image and the reflection degree information of the historical endoscopic image.

6. The endoscopic image processing method according to claim 4, characterized in that:The determining the specular reflection component of the reflective area according to the historical image information and the image information of the reflective area comprises one of the following: The historical image information and the image information of the reflective area are input into a specular reflection component recognition model for prediction processing to obtain the specular reflection component; or, determining a first specular reflection component of the reflective region from the endoscopic image; determining a second specular reflection component of the reflective region from the historical endoscopic image; Based on the historical image information, the first specular reflection component and the second specular reflection component are fused to obtain the specular reflection component.

7. The endoscopic image processing method according to claim 6, characterized in that: The fusion process includes performing a weighted summation process according to a weighting coefficient, wherein the weighting coefficient is obtained according to at least one of the following parameters: brightness information of the endoscopic image; brightness information of the reflective area in the endoscopic image; historical brightness information of the historical endoscopic images; historical brightness information of the reflective area in the historical endoscopic image; first brightness difference information between the endoscopic image and the historical endoscopic image; second brightness difference information of the reflective area between the endoscopic image and the historical endoscopic image; Information on the degree of reflection of the endoscope image; Information on the degree of reflectivity of the historical endoscopic images; Reflectivity difference information between the reflectivity information of the endoscopic image and the reflectivity information of the historical endoscopic images; Reflection intensity distribution information of the endoscopic image; Reflection intensity distribution information of the reflective area of ​​the endoscopic image; The historical endoscopic image includes historical reflection intensity distribution information of the reflection area.

8. The endoscopic image processing method according to claim 6, characterized in that: The step of determining the first specular reflection component of the reflective area from the endoscopic image comprises: After inputting the image information of the reflective area into a specular reflection component recognition model for prediction processing, the first specular reflection component is obtained; Determining the second specular reflection component of the reflective area from the historical endoscopic image comprises one of the following: After inputting the image information of the reflective area of ​​the historical endoscopic image into a specular reflection component recognition model for prediction processing, the second specular reflection component is obtained; Alternatively, the second specular reflection component saved after the reflective area of ​​the historical endoscopic image has been predictively processed by a specular reflection component recognition model is obtained.

9. The endoscopic image processing method according to any one of claims 1 to 8, characterized in that: The step of performing de-reflection processing on the reflective area of ​​the endoscopic image according to the diffuse reflection component to obtain a target endoscopic image includes: Performing de-reflection processing on the reflective area of ​​the endoscopic image according to the diffuse reflection component to obtain a local endoscopic image corresponding to the reflective area; The reflective area in the endoscopic image is replaced by the local endoscopic image to obtain a target endoscopic image.

10. The endoscopic image processing method according to claim 9, characterized in that: Determining the specular reflection component of the reflective area from the endoscopic image comprises: After downsampling the image information of the reflective area, first image information of the reflective area is obtained, and a specular reflection component of the reflective area is determined from the first image information; The step of replacing the reflective area in the endoscopic image with the local endoscopic image to obtain a target endoscopic image includes: After upsampling the local endoscopic image, the reflective area in the endoscopic image is replaced to obtain a target endoscopic image.

11. The endoscopic image processing method according to any one of claims 1 to 8, characterized in that: Performing a reflective region recognition process on the endoscopic image to determine the reflective region in the endoscopic image includes one of the following: Determine the reflective points of the endoscopic image, and determine the reflective area in the endoscopic image according to the distribution of the reflective points in the endoscopic image; Alternatively, determining the reflective area in the endoscopic image according to the distribution of brightness values ​​of the endoscopic image; Alternatively, determining the reflective area in the endoscopic image according to the distribution of values ​​of different color channels in the endoscopic image; After the endoscopic image is input into the reflective region recognition model for prediction processing, the reflective region in the endoscopic image is obtained.

12. An endoscopic image processing method, characterized in that: The method comprises: Acquiring endoscopic image data, wherein the endoscopic image data includes an endoscopic image; determining brightness information of the endoscopic image from the endoscopic image; determining a specular component of the endoscopic image from the endoscopic image; determining a diffuse reflection component of the endoscopic image according to the brightness information and the specular reflection component of the endoscopic image; The endoscopic image is subjected to de-reflection processing according to the diffuse reflection component to obtain a target endoscopic image.

13. The endoscopic image processing method according to claim 12, characterized in that: Determining the specular reflection component of the endoscopic image from the endoscopic image comprises one of the following: Acquire reflection intensity distribution information from the endoscopic image, determine neighborhood image information of the reflective area according to the reflection intensity distribution information, and compare the image information in the reflective area with the neighborhood image information to obtain the specular reflection component of the reflective area; Alternatively, the endoscope image is input into a specular reflection component recognition model for prediction processing to obtain the specular reflection component.

14. The endoscopic image processing method according to claim 12, characterized in that: Determining the specular reflection component of the endoscopic image from the endoscopic image comprises: Acquire historical image information of historical endoscopic images, wherein the historical endoscopic images are historical frame endoscopic images in the endoscopic image data; determining a specular reflection component of the endoscopic image based on the historical image information and the endoscopic image; The historical image information includes at least one of the following: a specular component of the historical endoscopic image; image data of the historical endoscopic images; a non-reflective endoscopic image after the historical endoscopic image has been subjected to de-reflection processing; Information about the degree of reflection in the historical endoscopic images.

15. An endoscopic image processing method, characterized in that: The method comprises: Acquiring endoscopic image data, wherein the endoscopic image data includes an endoscopic image; After performing de-reflection processing on the endoscopic image, a first endoscopic image is obtained; Acquire historical image information of historical endoscopic images, wherein the historical endoscopic images are historical frame endoscopic images in the endoscopic image data; Based on the historical image information of the historical endoscopic image, the first endoscopic image is smoothed to obtain a target endoscopic image.

16. The endoscopic image processing method according to claim 15, characterized in that: After performing de-reflection processing on the endoscopic image, a first endoscopic image is obtained, including: After the endoscopic image is input into a de-reflection model for de-reflection processing, a first endoscopic image is obtained.

17. The endoscopic image processing method according to claim 15, characterized in that: The historical image information includes image data of the historical endoscopic image, or a historical non-reflective endoscopic image after the historical endoscopic image is de-reflected; the first endoscopic image is smoothed based on the historical image information of the historical endoscopic image to obtain a target endoscopic image, including one of the following: fusing the first endoscopic image with the historical endoscopic image to obtain a target endoscopic image; Alternatively, the first endoscopic image and the historical non-reflective endoscopic image are fused to obtain a target endoscopic image.

18. The endoscopic image processing method according to claim 17, characterized in that: The fusion process includes performing a weighted summation process according to a weighting coefficient, wherein the weighting coefficient is obtained according to at least one of the following parameters: brightness information of the first endoscopic image; historical brightness information of the historical endoscopic images; first brightness difference information between the first endoscopic image and the historical endoscopic image; Information about the degree of reflection of the first endoscopic image; Reflectivity difference information between the reflectivity information of the first endoscopic image and the reflectivity information of the historical endoscopic image; Reflection intensity distribution information of the first endoscopic image; Historical reflection intensity distribution information of the historical endoscopic images; Information about the degree of reflection in the historical endoscopic images.

19. The endoscopic image processing method according to claim 18, characterized in that: In the fusion process, the target endoscopic image is calculated by the following formula: y′=ypre*W1+y*W2 Among them, y′ is the RGB information of the unit pixel of the target endoscopic image, y is the RGB information of the endoscopic image in the current frame, ypre is the RGB information of the historical endoscopic image in the previous frame or the historical non-reflective endoscopic image, and W1 and / or W2 are the weighting coefficients.

20. An endoscopic image processing method, characterized in that: The method comprises: Acquiring endoscopic image data, wherein the endoscopic image data includes an endoscopic image; Acquiring a reflective area in the endoscopic image; After performing de-reflection processing on the endoscopic image, a first endoscopic image is obtained; Acquire historical image information of historical endoscopic images, wherein the historical endoscopic images are historical frame endoscopic images in the endoscopic image data; Based on the historical image information of the historical endoscopic image, smoothing the first endoscopic image to obtain a second endoscopic image; extracting a local endoscopic image of the reflective area from the second endoscopic image; The reflective area in the endoscopic image is replaced by the local endoscopic image to obtain a target endoscopic image.

Citation Information

Cited By

  • Endoscope image reflection point identification method and device, equipment and storage medium

    CN120070995A

  • Endoscope image imaging method and endoscope

    CN120510211A

  • Endoscope image imaging method and endoscope

    CN120510211B