Methods, devices, electronic equipment, and storage media for processing endoscopic images

Abstract

The application discloses an endoscope image processing method and device, electronic equipment and storage medium, and relates to the technical field of endoscopes. In the processing method, the pixels of a first a channel image are subjected to a mapping operation based on a preset mapping table, which can map a red region in the image into a preset color; each pixel of a first b channel image is subjected to color offset, which can reduce the value of each pixel of the first b channel image, thereby reducing the color of the background in the current frame image to obtain a second b channel. The first HSV image is adjusted by using historical HSV adjustment parameters corresponding to the previous frame image, which can improve the smoothness of the video when the video composed of the continuous multiple frames of images is displayed by the display module. The application realizes electronic dyeing of the endoscope image under white light irradiation, facilitates medical practitioners to observe the endoscope image, and can reduce the equipment cost and the occupied volume of the endoscope.

Description

Technical Field

[0001] This application relates to the field of endoscopy technology, and in particular to a method, apparatus, electronic device and storage medium for processing endoscopic images. Background Technology

[0002] Endoscopic electronic staining is not true staining, but rather a process that processes endoscopic images based on different principles to create differences between diseased and normal tissues, facilitating observation and highlighting lesions. Current staining methods for endoscopic images typically involve adding a narrow-band filter in front of the endoscopic light source or using tunable spectral imaging, allowing only wavelengths strongly absorbed / reflected by hemoglobin, such as 400-430 nm (blue light) and 530-550 nm (green light), to reach the sensor. This makes microvessels on the mucosal surface appear "brown" and deep vessels appear "cyan," achieving a contrast effect similar to "iodine staining" or "indigo carmine."

[0003] However, the use of spectral imaging or the addition of narrowband filters in related technologies increases the cost of endoscopes and results in a larger volume of endoscopes. Summary of the Invention

[0004] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a method, apparatus, electronic device, and storage medium for processing endoscopic images, which can electronically stain endoscopic images illuminated by white light, thereby reducing equipment costs and the volume occupied by the endoscope.

[0005] According to a first aspect of the present application, a method for processing endoscopic images is characterized in that it is applied to an electronic endoscope, the electronic endoscope including a display module and a sensing module, the sensing module including a white light illumination component and a camera component;

[0006] The method includes:

[0007] The white light illumination component is controlled to illuminate the image, and the camera component is controlled to capture the image of the current frame.

[0008] Based on the current frame image, the first L channel image, the first a channel image, and the first b channel image of the CIELab color system are obtained;

[0009] Based on a preset mapping table, a mapping operation is performed on each pixel of the first a-channel image to obtain the second a-channel image;

[0010] Color shift processing is performed on each pixel of the first b-channel image to reduce the value of each pixel of the first b-channel image, thus obtaining the second b-channel image;

[0011] Based on the first L-channel image, the second a-channel image, and the second b-channel image, a second CIELab image is obtained, and the second CIELab image is converted to HSV space to obtain a first HSV image;

[0012] If no image adjustment operation command is detected for the current frame image, the previous frame image captured by the camera component is acquired, and it is detected whether the screen area of ​​the current frame image and the previous frame image are the same, and the detection result is obtained.

[0013] If the detection result indicates that the current frame image and the previous frame image have the same picture area, the historical HSV adjustment parameters corresponding to the previous frame image are obtained.

[0014] The first HSV image is adjusted based on the historical HSV adjustment parameters to obtain the second HSV image;

[0015] The second HSV image is converted to RGB space to obtain the image to be displayed in the current frame, and the image to be displayed in the current frame is displayed through the display module.

[0016] The endoscopic image processing method according to the embodiments of this application has at least the following beneficial effects: In the process of using an electronic endoscope, after illumination by a white light illumination component, a current frame image based on white light is obtained by capturing an image using a camera component. The current frame image is converted to the CIELab color system to obtain a first L-channel image, a first a-channel image, and a first b-channel image. The first a-channel image can represent the position of a color in the red-green spectrum. Therefore, based on a preset mapping table, the pixels of the first a-channel image are mapped to obtain a second a-channel image, which can map red areas in the image to preset colors, such as mapping blood vessel areas in the image to other preset colors. Since the first b-channel image can represent the position of a color in the yellow-blue spectrum, color shifting is performed on each pixel of the first b-channel image to reduce the value of each pixel, thereby reducing the background color in the current frame image to obtain the second b-channel. The first L-channel image, the second a-channel image, and the second b-channel image are then merged to obtain a second CIELab image, which is then converted to the HSV space to obtain a first HSV image. If no image adjustment operation command is detected, the previous frame image captured by the camera component based on white light is acquired. If the image area of ​​the current frame image is the same as that of the previous frame image, it indicates that the current shooting area of ​​the camera component has not changed compared to the shooting area of ​​the previous frame image. That is, the position of the camera component has not changed during the shooting process from the previous frame image to the current frame image. Therefore, the historical HSV adjustment parameters corresponding to the previous frame image can be used to adjust the first HSV image so that the adjustment process of the current frame image is the same as that of the previous frame image. In this way, when displaying a video composed of multiple consecutive frames of images through the display module, the smoothness of the video can be improved, making it easier to observe. Furthermore, this application realizes electronic staining of endoscope images illuminated by white light, which facilitates the observation of endoscope images by medical practitioners, reduces equipment costs, reduces the volume occupied by the endoscope, and improves the smoothness of the video when displaying a video composed of multiple consecutive frames of images through the display module.

[0017] According to some embodiments of this application, before converting the second HSV image to RGB space to obtain the current frame image to be displayed, and displaying the current frame image to be displayed through the display module, the method further includes:

[0018] Upon detecting the image adjustment operation command for the current frame image, the target HSV adjustment parameters are determined based on the image adjustment operation command;

[0019] The first HSV image is adjusted based on the target HSV adjustment parameters to obtain the second HSV image.

[0020] According to some embodiments of this application, before converting the second HSV image to RGB space to obtain the current frame image to be displayed, and displaying the current frame image to be displayed through the display module, the method further includes:

[0021] If the detection result indicates that the image area of ​​the current frame image is not the same as that of the previous frame image, the default HSV adjustment parameters are obtained.

[0022] The first HSV image is adjusted based on the default HSV adjustment parameters to obtain the second HSV image.

[0023] According to some embodiments of this application, obtaining the first L-channel image, the first a-channel image, and the first b-channel image of the CIELab color system based on the current frame image includes:

[0024] The green and blue channels of the current frame image are enhanced to obtain an enhanced image;

[0025] The enhanced image is converted to the CIELab color system to obtain the first CIELab image;

[0026] Based on the first CIELab image, channel splitting is performed to obtain the first L-channel image, the first a-channel image, and the first b-channel image.

[0027] According to some embodiments of this application, the step of detecting whether the current frame image and the previous frame image have the same screen area, and obtaining the detection result, includes:

[0028] The current frame image is converted to grayscale to obtain a first grayscale image, and the previous frame image is converted to grayscale to obtain a second grayscale image;

[0029] Calculate the structural similarity index between the first grayscale image and the second grayscale image;

[0030] If the structural similarity index is greater than a preset threshold, the detection result indicates that the current frame image and the previous frame image have the same picture area;

[0031] If the structural similarity index is less than the preset threshold, the detection result indicates that the image regions of the current frame image and the previous frame image are not the same.

[0032] According to some embodiments of this application, the sensing module is further provided with a gyroscope, which is used to acquire the attitude information of the camera component;

[0033] The step of detecting whether the current frame image and the previous frame image have the same image area, and obtaining the detection result, includes:

[0034] The gyroscope is used to obtain the first posture information of the camera component when capturing the current frame image, and the gyroscope is used to obtain the second posture information of the camera component when capturing the previous frame image;

[0035] If the first pose information is the same as the second pose information, then the detection result indicates that the current frame image and the previous frame image have the same picture area;

[0036] If the first pose information and the second pose information are not the same, then the detection result indicates that the current frame image and the previous frame image have different screen areas.

[0037] According to some embodiments of this application, the display module is further configured to display an adjustment screen, which includes a hue adjustment area, a saturation adjustment area, and a brightness adjustment area;

[0038] The image adjustment operation command is obtained through the following steps:

[0039] In response to detecting a first operation command for the hue adjustment region, a target hue parameter is obtained based on the first operation command;

[0040] In response to detecting a second operation command for the saturation adjustment region, a target saturation parameter is obtained based on the second operation command;

[0041] In response to the detection of a third operation command for the brightness adjustment area, a target brightness parameter is obtained based on the third operation command;

[0042] The image adjustment operation command is constructed based on the target hue parameter, the target saturation parameter, and the target brightness parameter.

[0043] A second aspect of this application provides an endoscope image processing apparatus for use in an electronic endoscope, the electronic endoscope including a display module and a sensing module, the sensing module including a white light illumination component and a camera component;

[0044] The device includes:

[0045] The shooting unit is used to control the white light illumination component to illuminate the image and to control the camera component to capture the image of the current frame.

[0046] The first image processing unit is used to obtain the first L-channel image, the first a-channel image, and the first b-channel image of the CIELab color system based on the current frame image;

[0047] The mapping unit is used to perform a mapping operation on each pixel of the first a-channel image based on a preset mapping table to obtain the second a-channel image.

[0048] An offset unit is used to perform color offset processing on each pixel of the first b-channel image to reduce the value of each pixel of the first b-channel image and obtain a second b-channel image.

[0049] The second image processing unit is used to obtain a second CIELab image based on the first L-channel image, the second a-channel image, and the second b-channel image, and to convert the second CIELab image to HSV space to obtain a first HSV image;

[0050] The detection unit is used to acquire the previous frame image captured by the camera component when no image adjustment operation command for the current frame image is detected, detect whether the screen area of ​​the current frame image and the previous frame image are the same, and obtain the detection result.

[0051] The acquisition unit is used to acquire the historical HSV adjustment parameters corresponding to the previous frame image when the detection result indicates that the current frame image and the previous frame image have the same picture area;

[0052] An adjustment unit is used to adjust the first HSV image based on the historical HSV adjustment parameters to obtain a second HSV image;

[0053] The display unit is used to convert the second HSV image to RGB space to obtain the current frame image to be displayed, and to display the current frame image to be displayed through the display module.

[0054] A third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the endoscopic image processing method described in any one of the first aspects of the embodiment.

[0055] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the endoscopic image processing method described in any one of the first aspects of the embodiment.

[0056] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0057] The present application will be further described below with reference to the accompanying drawings and embodiments, wherein:

[0058] Figure 1 This is a flowchart illustrating the steps of an endoscopic image processing method according to an embodiment of this application.

[0059] Figure 2 This is a schematic diagram of a sub-process before step S190 in an embodiment of this application;

[0060] Figure 3 This is a simplified schematic diagram of the adjustment screen according to an embodiment of this application;

[0061] Figure 4 This is a schematic diagram of a sub-process before step S190 in an embodiment of this application;

[0062] Figure 5 This is a schematic flowchart of step S160 in an embodiment of this application;

[0063] Figure 6 This is a detailed flowchart illustrating step S160 of another embodiment of this application;

[0064] Figure 7 This is a schematic diagram of the structure of an endoscope image processing device according to an embodiment of this application;

[0065] Figure 8 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0066] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0067] In the description of this application, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0068] In the description of this application, "several" means one or more, "multiple" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0069] In the description of this application, unless otherwise expressly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0070] In the description of this application, the terms "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0071] The first aspect of this application provides a method for processing endoscopic images. This method can be applied to a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the endoscopic image processing method, but is not limited to the above forms.

[0072] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0073] The endoscopic image processing method of the first aspect of this application is applied to an electronic endoscope, which includes a display module and a sensing module. The sensing module includes a white light illumination component and a camera component. The white light illumination component emits white light to illuminate the area to be photographed, and the camera component is used to take a picture. The display module can be a display screen. (See reference...) Figure 1 , Figure 1 This is a schematic flowchart illustrating the steps of an endoscopic image processing method according to an embodiment of this application. The endoscopic image processing method according to an embodiment of this application may include, but is not limited to, steps S110 to S190.

[0074] Step S110: Control the white light illumination component to illuminate the image and control the camera component to capture the image to obtain the current frame image;

[0075] Step S120: Based on the current frame image, obtain the first L channel image, the first a channel image, and the first b channel image of the CIELab color system;

[0076] It should be noted that the CIE Lab color system (CIE LAB) is a uniform color space defined by the International Commission on Illumination (CIE). Based on physiological characteristics, it aims to achieve equidistant color quantification through mathematical methods, ensuring that color differences perceived by the human eye correspond to geometric distances in space. This space consists of three components: L* (lightness), a* (red-green axis), and b* (yellow-blue axis), forming a three-dimensional Cartesian coordinate system to accurately describe color. Specifically, the pixel values ​​in the first L channel image represent the lightness of the corresponding pixel, the pixel values ​​in the first a channel image represent the position of the corresponding pixel's color in the red-green spectrum, and the pixel values ​​in the first b channel image represent the position of the corresponding pixel's color in the yellow-blue spectrum.

[0077] Step S130: Based on a preset mapping table, perform a mapping operation on each pixel of the first a-channel image to obtain the second a-channel image;

[0078] In some embodiments, a preset mapping table records the correspondence between multiple range intervals and mapping values. In step S130, each pixel of the first a-channel image is traversed. For example, the range interval to which the pixel value of the target pixel belongs is taken as the target interval. Based on the target interval, the corresponding target mapping value is determined from multiple mapping values. The pixel value of the target pixel is modified to the target mapping value. The target pixel is any pixel of the first a-channel image. In this way, the second a-channel image can be obtained. Those skilled in the art can set the various range intervals, various mapping values, and the correspondence between the various range intervals and various mapping values ​​in the mapping table according to the actual situation, so as to realize the mapping of red areas to green in the first a-channel image. Generally, red areas represent the areas where blood vessels are located, so the areas where blood vessels are located can be mapped to green. For example, the range intervals include [a, b], [b+1, c], and [c+1, d], where the mapping value corresponding to [a, b] is A, the mapping value corresponding to [b+1, c] is B, and the mapping value corresponding to [c+1, d] is C. Those skilled in the art can set the values ​​of a, b, c, d, A, B, and C according to the actual situation, and this application does not make specific limitations in this regard.

[0079] Step S140: Perform color shift processing on each pixel of the first b-channel image to reduce the value of each pixel of the first b-channel image to obtain the second b-channel image;

[0080] In some embodiments, under typical white light illumination, the background region in the current frame image is usually yellow. The pixel values ​​of the first b-channel image represent the position of the corresponding pixel's color in the yellow-blue spectrum. Therefore, color shifting is performed on each pixel of the first b-channel image to reduce the value of each pixel, resulting in the second b-channel image. For example, color shifting is performed on each pixel of the first b-channel image using an offset formula. The offset formula is:

[0081] Y=xz;

[0082] Where Y is the offset value, x is the pixel value of the first b-channel image, and z is the preset offset constant.

[0083] Step S150: Based on the first L-channel image, the second a-channel image, and the second b-channel image, a second CIELab image is obtained, and the second CIELab image is converted to HSV space to obtain the first HSV image;

[0084] It's important to note that HSV (Hue, Saturation, Value) is a color model based on human visual perception, widely used in image processing and computer vision. Unlike the RGB (Red, Green, Blue) color space, HSV decomposes color information into three intuitive components: Hue, Saturation, and Value. Hue represents the basic color type, such as red, green, and blue, and is represented by angles (0°~360°) on the color wheel. Saturation describes the purity or vividness of a color, typically ranging from 0% to 100%. Higher saturation results in a more vibrant color, while lower saturation makes the color closer to gray. Value (also called lightness) reflects the brightness of a color, ranging from 0 (black) to the maximum value (brightest), controlling overall brightness without altering hue or saturation. HSV is insensitive to lighting conditions, and its color thresholds accurately separate the front and back images, avoiding misjudgments of color mixing during image stacking. Converting the first image to a transformed image improves the accuracy of subsequent image processing.

[0085] Step S160: If no image adjustment operation command for the current frame image is detected, the previous frame image captured by the camera component is acquired, and the screen area of ​​the current frame image and the previous frame image are detected to be the same, and the detection result is obtained.

[0086] Step S170: If the detection result indicates that the current frame image and the previous frame image have the same picture area, obtain the historical HSV adjustment parameters corresponding to the previous frame image.

[0087] Step S180: Adjust the first HSV image based on historical HSV adjustment parameters to obtain the second HSV image;

[0088] For example, historical HSV adjustment parameters include historical hue adjustment values, historical saturation adjustment values, and historical brightness adjustment values. These values ​​can be positive or negative. Adjustments are made using the following methods:

[0089] H2 = H0 + H1;

[0090] S2 = S0 + S1;

[0091] V2 = V0 + V1;

[0092] Wherein, H0 is the hue value of a pixel in the first HSV image, H1 is the historical hue adjustment value, and H2 is the adjusted hue value. S0 is the saturation value of a pixel in the first HSV image, S1 is the historical saturation adjustment value, and S2 is the adjusted saturation value. V0 is the brightness value of a pixel in the first HSV image, V1 is the historical brightness adjustment value, and V2 is the adjusted brightness value.

[0093] Step S190: Convert the second HSV image to RGB space to obtain the image to be displayed in the current frame, and display the image to be displayed in the current frame through the display module.

[0094] It should be noted that during the use of an electronic endoscope, the display module typically displays images in video format. The video consists of multiple consecutive frames captured by the camera module, requiring the execution of steps S110 to S1190 for each frame. Furthermore, during the execution of steps S110 to S190, after obtaining the corresponding first HSV image, the hue, brightness, and saturation of the first HSV image need to be adjusted. In steps S170 to S180 of this application, if the image area of ​​the current frame is the same as that of the previous frame, it means that the current shooting area of ​​the camera component has not changed from the shooting area of ​​the previous frame. That is, the position of the camera component has not changed during the shooting process from the previous frame to the current frame. Therefore, the historical HSV adjustment parameters corresponding to the previous frame can be used to adjust the first HSV image so that the adjustment process of the current frame is the same as that of the previous frame. In this way, when the camera component is stationary, the adjustment process of the continuous multiple frames captured by the camera component is the same, thereby improving the smoothness of the display module when displaying video.

[0095] The endoscopic image processing method of this application embodiment, through steps S110 to S190, involves illuminating the image with a white light illumination component and then capturing it with a camera component to obtain a current frame image based on white light. The current frame image is converted to the CIELab color system to obtain a first L-channel image, a first a-channel image, and a first b-channel image. The first a-channel image represents the position of a color in the red-green spectrum. Therefore, based on a preset mapping table, the pixels of the first a-channel image are mapped to obtain a second a-channel image, which maps red areas in the image to preset colors, such as mapping blood vessel areas in the image to other preset colors. Since the first b-channel image represents the position of a color in the yellow-blue spectrum, color shifting is performed on each pixel of the first b-channel image to reduce the value of each pixel, thereby reducing the background color in the current frame image to obtain the second b-channel. The first L-channel image, the second a-channel image, and the second b-channel image are then merged to obtain a second CIELab image, which is then converted to the HSV space to obtain a first HSV image. If no image adjustment operation command is detected, the previous frame image captured by the camera component based on white light is acquired. If the image area of ​​the current frame image is the same as that of the previous frame image, it indicates that the current shooting area of ​​the camera component has not changed compared to the shooting area of ​​the previous frame image. That is, the position of the camera component has not changed during the shooting process from the previous frame image to the current frame image. Therefore, the historical HSV adjustment parameters corresponding to the previous frame image can be used to adjust the first HSV image so that the adjustment process of the current frame image is the same as that of the previous frame image. In this way, when displaying a video composed of multiple consecutive frames of images through the display module, the smoothness of the video can be improved, making it easier to observe. Furthermore, this application realizes electronic staining of endoscope images illuminated by white light, which facilitates the observation of endoscope images by medical practitioners, reduces equipment costs, reduces the volume occupied by the endoscope, and improves the smoothness of the video when displaying a video composed of multiple consecutive frames of images through the display module.

[0096] In some embodiments, the electronic endoscope is equipped with a host computer, which is configured with virtual reality processing software. The virtual reality processing software is used to execute the endoscope image processing method of the embodiments of this application, such as executing steps S110 to S190 described above.

[0097] Understandably, referring to Figure 2 , Figure 2 This is a schematic diagram of a sub-process before step S190 in an embodiment of this application. Figure 2 The illustrated process includes, but is not limited to, steps S210 to S220.

[0098] Step S210: If an image adjustment operation command for the current frame image is detected, the target HSV adjustment parameters are determined based on the image adjustment operation command.

[0099] Step S220: Adjust the first HSV image based on the target HSV adjustment parameters to obtain the second HSV image.

[0100] It should be noted that when an image adjustment operation command for the current frame image is detected, steps S160 to S180 do not need to be executed, but the second HSV image is obtained through step S220.

[0101] Understandably, the display module is also used to display adjustment screens, which include hue adjustment areas, saturation adjustment areas, and brightness adjustment areas; see reference. Figure 3 , Figure 3 This is a simplified schematic diagram of the adjustment screen according to an embodiment of this application.

[0102] Image adjustment operation commands are obtained through the following steps:

[0103] In response to the detection of a first operation command for the hue adjustment region, the target hue parameters are obtained based on the first operation command;

[0104] In response to the detection of a second operation command for the saturation adjustment region, the target saturation parameter is obtained based on the second operation command;

[0105] In response to the detection of a third operation command for the brightness adjustment area, the target brightness parameter is obtained based on the third operation command;

[0106] Image adjustment operation instructions are constructed based on the target hue parameter, target saturation parameter, and target brightness parameter.

[0107] Specifically, the tone adjustment area includes a tone slider, which users can slide using a keyboard or touchscreen. Upon stopping the slide, a first operation command is triggered, setting the current value of the tone slider as the target tone parameter. Similarly, the saturation adjustment area includes a saturation slider, which users can slide using a keyboard or touchscreen. Upon stopping the slide, a second operation command is triggered, setting the current value of the saturation slider as the target saturation parameter. Likewise, the brightness adjustment area includes a brightness slider, which users can slide using a keyboard or touchscreen. Upon stopping the slide, a third operation command is triggered, setting the current value of the brightness slider as the target brightness parameter.

[0108] For example, the process of adjusting the first HSV image based on the target hue parameter, target saturation parameter, and target brightness parameter can be represented as follows:

[0109] H2 = H0 + H3;

[0110] S2 = S0 + S3;

[0111] V2 = V0 + V3;

[0112] Wherein, H0 is the hue value of a pixel in the first HSV image, H3 is the target hue parameter, and H2 is the adjusted hue value. S0 is the saturation value of a pixel in the first HSV image, S3 is the target saturation parameter, and S2 is the adjusted saturation value. V0 is the brightness value of a pixel in the first HSV image, V3 is the target brightness parameter, and V2 is the adjusted brightness value. The target hue parameter, target saturation parameter, and target brightness parameter can be positive or negative values.

[0113] In some embodiments, refer to Figure 4 , Figure 4 This is a schematic diagram of a sub-process before step S190 in an embodiment of this application. Figure 4 The illustrated steps include steps S410 and S420.

[0114] Step S410: If the detection result indicates that the image area of ​​the current frame image is not the same as that of the previous frame image, obtain the default HSV adjustment parameters.

[0115] Step S420: Adjust the first HSV image based on the default HSV adjustment parameters to obtain the second HSV image.

[0116] It is worth noting that if the detection result indicates that the current frame image and the previous frame image are not the same, it means that the current shooting area of ​​the camera component has changed from the shooting area of ​​the previous frame image. That is, the position of the camera component has changed during the shooting process from the previous frame image to the current frame image. If the historical HSV adjustment parameters of the previous frame image are directly used for adjustment, the second HSV image may have problems due to the changed scene, such as color distortion. Therefore, the default HSV adjustment parameters are obtained, and the first HSV image is adjusted based on the default HSV adjustment parameters to obtain the second HSV image.

[0117] For example, the default HSV adjustment parameters include the default hue parameter, the default saturation parameter, and the default brightness parameter.

[0118] The process of adjusting the first HSV image based on the default HSV adjustment parameters can be represented as:

[0119] H2 = H0 + H4;

[0120] S2 = S0 + S4;

[0121] V2 = V0 + V4;

[0122] Where H0 is the hue value of a pixel in the first HSV image, H4 is the default hue parameter, and H2 is the adjusted hue value. S0 is the saturation value of a pixel in the first HSV image, S4 is the default saturation parameter, and S2 is the adjusted saturation value. V0 is the brightness value of a pixel in the first HSV image, V4 is the default brightness parameter, and V2 is the adjusted brightness value. The default hue parameter, default saturation parameter, and default brightness parameter can be positive or negative values.

[0123] Understandably, referring to Figure 5 , Figure 5 This is a schematic diagram of a specific process of step S160 in an embodiment of this application. In step S160, detecting whether the image area of ​​the current frame image is the same as that of the previous frame image and obtaining the detection result may include, but is not limited to, steps S510 to S540.

[0124] Step S510: Convert the current frame image to grayscale to obtain a first grayscale image, and convert the previous frame image to grayscale to obtain a second grayscale image;

[0125] Step S520: Calculate the structural similarity index between the first grayscale image and the second grayscale image;

[0126] Step S530: If the structural similarity index is greater than the preset threshold, the detection result indicates that the picture area of ​​the current frame image is the same as that of the previous frame image.

[0127] Step S540: If the structural similarity index is less than the preset threshold, the detection result indicates that the image regions of the current frame image and the previous frame image are not the same.

[0128] It should be noted that the Structural Similarity Index (SSIM) is an indicator used to measure the similarity between two images. It is based on the perceptual characteristics of the human visual system in detecting image structure information, comprehensively evaluating image quality from three dimensions: brightness, contrast, and structure. The SSIM value ranges from -1 to 1; a value closer to 1 indicates greater structural similarity between the two images. When two images are completely identical, the SSIM value is 1. Those skilled in the art can set preset index thresholds according to actual circumstances; this application does not impose any limitations on this.

[0129] In this embodiment of the application, through steps S510 to S540, a detection result can be obtained. If the structural similarity index is greater than a preset threshold, the detection result indicates that the current frame image and the previous frame image have the same picture area. If the structural similarity index is less than the preset threshold, the detection result indicates that the current frame image and the previous frame image have different picture areas.

[0130] Understandably, the sensing module also includes a gyroscope, which is used to acquire the attitude information of the camera component; refer to Figure 6 , Figure 6 This is a schematic diagram of the specific process of step S160 in another embodiment of this application. In step S160, it is detected whether the screen area of ​​the current frame image is the same as that of the previous frame image, and the detection result is obtained. This may include, but is not limited to, steps S610 to S630.

[0131] Step S610: Obtain the first attitude information of the camera component when capturing the current frame image using the gyroscope, and obtain the second attitude information of the camera component when capturing the previous frame image using the gyroscope.

[0132] Step S620: If the first pose information is the same as the second pose information, then the detection result indicates that the picture area of ​​the current frame image is the same as that of the previous frame image.

[0133] Step S630: If the first pose information and the second pose information are not the same, then the detection result indicates that the picture area of ​​the current frame image and the previous frame image are not the same.

[0134] It is worth noting that the first pose information includes the tilt angle and coordinates of the camera component, and the second pose information includes the tilt angle and coordinates of the camera component. Through steps S610 to S630, the detection result can be determined. If the first pose information and the second pose information are the same, the detection result indicates that the image area of ​​the current frame is the same as that of the previous frame; if the first pose information and the second pose information are different, the detection result indicates that the image area of ​​the current frame is different from that of the previous frame.

[0135] In some embodiments, step S120 may include, but is not limited to, steps S121 to S123.

[0136] Step S121: Enhance the green and blue channels of the current frame image to obtain an enhanced image;

[0137] Step S122: Convert the enhanced image to the CIELab color system to obtain the first CIELab image;

[0138] Step S123: Based on the first CIELab image, perform channel splitting to obtain the first L channel image, the first a channel image, and the first b channel image.

[0139] Specifically, the current frame image is first split into RGB channels to obtain a first red channel, a first green channel, and a first blue channel. Then, the first blue channel undergoes color enhancement processing to obtain a second blue channel; the first green channel also undergoes color enhancement processing to obtain a second green channel. The first red channel, second blue channel, and second green channel are then merged to obtain an enhanced image. This enhanced image is then converted to the CIELab color system to obtain a first CIELab image, thus yielding the first L channel image, the first a channel image, and the first b channel image. It should be noted that color enhancement processing on the first blue channel highlights superficial mucosal blood vessel information in the image. Color enhancement processing on the first green channel highlights deep submucosal blood vessel information in the image.

[0140] A second aspect of this application provides an endoscope image processing apparatus applied to the aforementioned electronic endoscope. (Refer to...) Figure 7 , Figure 7 This is a schematic diagram of the structure of an endoscopic image processing apparatus according to an embodiment of this application. The endoscopic image processing apparatus includes:

[0141] The imaging unit 710 is used to control the white light illumination component to illuminate the image and to control the camera component to capture the image of the current frame.

[0142] The first image processing unit 720 is used to obtain the first L-channel image, the first a-channel image, and the first b-channel image of the CIELab color system based on the current frame image;

[0143] The mapping unit 730 is used to perform a mapping operation on each pixel of the first a-channel image based on a preset mapping table to obtain the second a-channel image;

[0144] The offset unit 740 is used to perform color offset processing on each pixel of the first b-channel image to reduce the value of each pixel of the first b-channel image and obtain the second b-channel image.

[0145] The second image processing unit 750 is used to obtain a second CIELab image based on the first L-channel image, the second a-channel image, and the second b-channel image, and to convert the second CIELab image to HSV space to obtain a first HSV image;

[0146] The detection unit 760 is used to acquire the previous frame image captured by the camera component when no image adjustment operation command for the current frame image is detected, detect whether the screen area of ​​the current frame image and the previous frame image are the same, and obtain the detection result.

[0147] The acquisition unit 770 is used to acquire the historical HSV adjustment parameters corresponding to the previous frame image when the detection result indicates that the current frame image and the previous frame image have the same picture area.

[0148] The adjustment unit 780 is used to adjust the first HSV image based on historical HSV adjustment parameters to obtain the second HSV image;

[0149] Display unit 790 is used to convert the second HSV image to RGB space to obtain the image to be displayed in the current frame, and to display the image to be displayed in the current frame through the display module.

[0150] The endoscopic image processing apparatus of the second aspect of this application is used to execute the endoscopic image processing method of the first aspect of this application. When executing the method, after illumination by a white light illumination component, a current frame image based on white light is obtained by capturing an image using a camera component. The current frame image is converted to the CIELab color system to obtain a first L-channel image, a first a-channel image, and a first b-channel image. The first a-channel image can represent the position of a color in the red-green spectrum. Therefore, based on a preset mapping table, the pixels of the first a-channel image are mapped to obtain a second a-channel image, which can map red areas in the image to preset colors, such as mapping blood vessel areas in the image to other preset colors. Since the first b-channel image can represent the position of a color in the yellow-blue spectrum, color shifting is performed on each pixel of the first b-channel image to reduce the value of each pixel, thereby reducing the background color in the current frame image to obtain the second b-channel. The first L-channel image, the second a-channel image, and the second b-channel image are then merged to obtain a second CIELab image, which is then converted to the HSV space to obtain a first HSV image. If no image adjustment operation command is detected, the previous frame image captured by the camera component based on white light is acquired. If the image area of ​​the current frame image is the same as that of the previous frame image, it indicates that the current shooting area of ​​the camera component has not changed compared to the shooting area of ​​the previous frame image. That is, the position of the camera component has not changed during the shooting process from the previous frame image to the current frame image. Therefore, the historical HSV adjustment parameters corresponding to the previous frame image can be used to adjust the first HSV image so that the adjustment process of the current frame image is the same as that of the previous frame image. In this way, when displaying a video composed of multiple consecutive frames of images through the display module, the smoothness of the video can be improved, making it easier to observe. Furthermore, this application realizes electronic staining of endoscope images illuminated by white light, which facilitates the observation of endoscope images by medical practitioners, reduces equipment costs, reduces the volume occupied by the endoscope, and improves the smoothness of the video when displaying a video composed of multiple consecutive frames of images through the display module.

[0151] It should be noted that the specific implementation of the endoscopic image processing device is basically the same as the specific embodiment of the endoscopic image processing method described above, and will not be repeated here. Under the premise of meeting the requirements of the embodiments of this application, the endoscopic image processing device may also be provided with other functional units to implement the endoscopic image processing method in the above embodiments.

[0152] A third aspect of this application provides an electronic device including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the endoscopic image processing method of any one of the first aspects of the embodiment. This electronic device can be any smart terminal, including tablet computers, desktop computers, etc.

[0153] Reference Figure 8 , Figure 8 This is a schematic diagram of the structure of an electronic device according to one embodiment. The electronic device includes:

[0154] The processor 801 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0155] The memory 802 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 802 and is called and executed by the processor 801 to execute the endoscopic image processing method of the embodiments of this application.

[0156] The 803 input / output interface is used to implement information input and output.

[0157] The communication interface 804 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0158] Bus 805 transmits information between various components of the device (e.g., processor 801, memory 802, input / output interface 803, and communication interface 804);

[0159] The processor 801, memory 802, input / output interface 803, and communication interface 804 are connected to each other within the device via bus 805.

[0160] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the endoscopic image processing method of any one of the first aspects of this application.

[0161] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0162] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0163] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0164] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0165] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0166] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0167] It should be understood that in this application, "at least one (item)" means one or more, and "more than one" means two or more. "And / or" is used to describe the mapping relationship between the mapped objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following mapped objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0168] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0169] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0170] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0171] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0172] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method of processing an endoscopic image, characterized by, The application is applied to an electronic endoscope, the electronic endoscope comprising a display module and a sensing module, the sensing module comprising a white light irradiation assembly and a camera assembly; The method comprises: controlling the white light irradiation assembly to irradiate and controlling the camera assembly to capture a current frame image; based on the current frame image, obtaining a first L channel image, a first a channel image and a first b channel image of a CIELab color system; based on a preset mapping table, performing a mapping operation on each pixel of the first a channel image to obtain a second a channel image; performing color offset processing on each pixel of the first b channel image to reduce the value of each pixel of the first b channel image to obtain a second b channel image; based on the first L channel image, the second a channel image and the second b channel image, obtaining a second CIELab image and converting the second CIELab image to an HSV space to obtain a first HSV image; if no image adjustment operation instruction for the current frame image is detected, obtaining a previous frame image captured by the camera assembly, detecting whether the picture area of the current frame image is the same as that of the previous frame image to obtain a detection result; if the detection result indicates that the picture area of the current frame image is the same as that of the previous frame image, obtaining a historical HSV adjustment parameter corresponding to the previous frame image; based on the historical HSV adjustment parameter, adjusting the first HSV image to obtain a second HSV image; converting the second HSV image to an RGB space to obtain a current frame to-be-displayed image and displaying the current frame to-be-displayed image through the display module; the detection of whether the picture area of the current frame image is the same as that of the previous frame image to obtain a detection result comprises: performing gray scale conversion on the current frame image to obtain a first gray scale image and performing gray scale conversion on the previous frame image to obtain a second gray scale image; calculating a structural similarity index between the first gray scale image and the second gray scale image; if the structural similarity index is greater than a preset threshold, the detection result indicates that the picture area of the current frame image is the same as that of the previous frame image; if the structural similarity index is less than the preset threshold, the detection result indicates that the picture area of the current frame image is different from that of the previous frame image.

2. The method of processing an endoscopic image according to claim 1, characterized by, before the conversion of the second HSV image to an RGB space to obtain a current frame to-be-displayed image and the display of the current frame to-be-displayed image through the display module, the method further comprises: if the image adjustment operation instruction for the current frame image is detected, determining a target HSV adjustment parameter based on the image adjustment operation instruction; based on the target HSV adjustment parameter, adjusting the first HSV image to obtain the second HSV image.

3. The method of processing an endoscopic image according to claim 1, characterized by, before the conversion of the second HSV image to an RGB space to obtain a current frame to-be-displayed image and the display of the current frame to-be-displayed image through the display module, the method further comprises: In a case where the detection result represents that the picture area of the current frame image and the previous frame image is not the same, a default HSV adjustment parameter is acquired; The first HSV image is adjusted based on the default HSV adjustment parameter to obtain the second HSV image.

4. The method of processing an endoscopic image according to claim 1, characterized by, The first L channel image, the first a channel image and the first b channel image of the CIELab color system are obtained based on the current frame image, including: The green channel and the blue channel of the current frame image are enhanced to obtain an enhanced image; The enhanced image is converted to the CIELab color system to obtain a first CIELab image; The first L channel image, the first a channel image and the first b channel image are obtained based on the first CIELab image.

5. The method of processing an endoscopic image according to claim 1, characterized by, The sensing module is further provided with a gyroscope, and the gyroscope is used to acquire attitude information of the camera assembly; The detection result is obtained by detecting whether the picture area of the current frame image and the previous frame image is the same, including: The first attitude information of the camera assembly when shooting the current frame image is acquired through the gyroscope, and the second attitude information of the camera assembly when shooting the previous frame image is acquired through the gyroscope; If the first attitude information and the second attitude information are the same, the detection result represents that the picture area of the current frame image and the previous frame image is the same; If the first attitude information and the second attitude information are not the same, the detection result represents that the picture area of the current frame image and the previous frame image is not the same.

6. The method of processing an endoscopic image according to claim 1, characterized by, The display module is further used to display an adjustment picture, and the adjustment picture is provided with a hue adjustment area, a saturation adjustment area and a brightness adjustment area; The image adjustment operation instruction is obtained through the following steps: In response to detecting a first operation instruction for the hue adjustment area, a target hue parameter is obtained based on the first operation instruction; In response to detecting a second operation instruction for the saturation adjustment area, a target saturation parameter is obtained based on the second operation instruction; In response to detecting a third operation instruction for the brightness adjustment area, a target brightness parameter is obtained based on the third operation instruction; The image adjustment operation instruction is constructed based on the target hue parameter, the target saturation parameter and the target brightness parameter.

7. An apparatus for processing an endoscope image, characterized by comprising: The electronic endoscope includes a display module and a sensing module, and the sensing module includes a white light irradiation assembly and a camera assembly; The device includes: A shooting unit is configured to control the white light irradiation assembly to irradiate and control the camera assembly to shoot to obtain a current frame image; A first image processing unit is configured to obtain a first L channel image, a first a channel image and a first b channel image of a CIELab color system based on the current frame image; A mapping unit is configured to perform a mapping operation on each pixel of the first a channel image based on a preset mapping table to obtain a second a channel image. a shift unit, configured to perform color shift processing on each pixel of the first b channel image to reduce the value of each pixel of the first b channel image, to obtain a second b channel image; a second image processing unit, configured to obtain a second CIELab image based on the first L channel image, the second a channel image and the second b channel image, and convert the second CIELab image to HSV space to obtain a first HSV image; a detection unit, configured to, in a case where no image adjustment operation instruction for the current frame image is detected, acquire a previous frame image captured by the camera assembly, and detect whether a picture area of the current frame image is same as that of the previous frame image, to obtain a detection result; an acquisition unit, configured to, in a case where the detection result indicates that the picture area of the current frame image is same as that of the previous frame image, acquire a historical HSV adjustment parameter corresponding to the previous frame image; an adjustment unit, configured to adjust the first HSV image based on the historical HSV adjustment parameter to obtain a second HSV image; a display unit, configured to convert the second HSV image to RGB space to obtain a current frame to-be-displayed image, and display the current frame to-be-displayed image through the display module; the detection unit is specifically configured to: perform gray scale conversion on the current frame image to obtain a first gray scale image, and perform gray scale conversion on the previous frame image to obtain a second gray scale image; calculate a structural similarity index between the first gray scale image and the second gray scale image; if the structural similarity index is greater than a preset threshold, the detection result indicates that the picture area of the current frame image is same as that of the previous frame image; if the structural similarity index is less than the preset threshold, the detection result indicates that the picture area of the current frame image is not same as that of the previous frame image.

8. An electronic device, comprising: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the processing method of the endoscope image in any one of claims 1 to 6 when executing the computer program.

9. A computer readable storage medium, the storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the processing method of the endoscope image in any one of claims 1 to 6.

Images