Image processing method and endoscope system

Abstract

Embodiments of the present application provide an image processing method and an endoscope system. The method is applied to a processing device in an endoscope system, the endoscope system further comprising an endoscope, a light source module of the endoscope being configured to output white light and special light in time division, the spectrum of the white light being different from the spectrum of the special light, the method comprising: obtaining a special light image, the special light image being obtained by photographing an observed object irradiated by the special light; identifying a first attention region according to pixel information of each pixel point in the special light image; obtaining a white light image, the white light image being obtained by photographing the observed object irradiated by the white light; performing image enhancement processing on the white light image to enhance display of a second attention region in the white light image, and / or marking the second attention region in the white light image to obtain a target image, the second attention region corresponding to the first attention region. The present application can highlight target tissue in tissue in the white light image.

Description

Technical Field

[0001] This application relates to the field of endoscopic image processing technology, and in particular to an image processing method and an endoscopic system. Background Technology

[0002] An endoscope is an instrument used for medical examinations and surgeries, designed to enter the human body through natural openings or small incisions to directly observe internal organs and structures.

[0003] An endoscope typically consists of a long, thin tube with a light source and camera at one end and a monitor at the other, allowing doctors to observe real-time images of the patient's body. In endoscopic imaging diagnosis, clinicians often need to assess and judge the patient's condition by observing the shape and color of blood vessels in the images.

[0004] In practical applications, endoscopes primarily offer two observation modes: white light and special light. White light mode uses a normal white light source for illumination, producing images that more closely match the true color of the observed object, better reproducing the tissue color. However, some detailed features (such as superficial blood vessels, lesions, and bleeding points) are not highlighted in white light images, making it difficult to identify lesions. Special light mode uses narrowband light of a specific wavelength for illumination, producing images that highlight some detailed features of the observed object, facilitating the identification of lesions. However, the colors in special light images are severely distorted, differing significantly from the true color of the observed object. Summary of the Invention

[0005] This application provides an image processing method and an endoscope system that can better restore the tissue color of the observed object while highlighting the target tissue in a white light image.

[0006] In a first aspect, embodiments of this application provide an image processing method applied to a processing device in an endoscope system. The endoscope system further includes an endoscope, and the light source module of the endoscope is configured to output white light and special light in a time-division manner. The spectrum of the white light is different from the spectrum of the special light. The method includes: acquiring a special light image, which is obtained by photographing an object being observed illuminated by special light; identifying a first region of interest based on pixel information of each pixel in the special light image; acquiring a white light image, which is obtained by photographing an object being observed illuminated by white light; performing image enhancement processing on the white light image to enhance the display of a second region of interest in the white light image; and / or marking the second region of interest in the white light image to obtain a target image, wherein the second region of interest corresponds to the first region of interest.

[0007] Secondly, embodiments of this application provide an image processing apparatus configured in an endoscope system. The endoscope system further includes an endoscope, and the endoscope's light source module is configured to output white light and special light in a time-division manner. The spectrum of the white light is different from the spectrum of the special light. The apparatus includes:

[0008] The image acquisition module is used to acquire special light images, which are obtained by photographing the observed object illuminated by special light.

[0009] The recognition module is used to identify the first region of interest based on the pixel information of each pixel in the special light image;

[0010] The image acquisition module is also used to acquire white light images, which are obtained by photographing the observed object illuminated by white light.

[0011] The image processing module is used to perform image enhancement processing on the white light image to enhance the display of the second region of interest in the white light image, and / or to mark the second region of interest in the white light image to obtain a target image, wherein the second region of interest corresponds to the first region of interest;

[0012] The display module is used to display the target image.

[0013] Thirdly, embodiments of this application provide an endoscope system, including: an endoscope, a memory, and a processor;

[0014] The endoscope includes a light source module, which is configured to output white light and special light in a time-division manner, with the spectrum of the white light being different from that of the special light.

[0015] The memory stores instructions that the computer executes;

[0016] The processor executes computer execution instructions stored in memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0018] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0019] In this embodiment, the endoscope's light source module can output special light and white light in a time-division manner. A special light image is acquired by illuminating the observed object with the special light. By analyzing the special light image, the target tissue within the observed object's tissue is extracted, and possible lesion areas (i.e., the first region of interest) are identified based on the target tissue. Then, a white light image is acquired by illuminating the observed object with white light. Based on the information extracted from the special light image, image enhancement processing is performed on the corresponding region in the white light image (i.e., the second region of interest) to enhance the display of the target tissue in the white light image. In this way, while better restoring the tissue color of the observed object, the target tissue within the tissue is highlighted in the white light image. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0021] Figure 1 A schematic diagram comparing a white light image and a NEI image provided in an embodiment of this application;

[0022] Figure 2 This is a schematic diagram illustrating an application scenario of an image processing method according to an embodiment of this application;

[0023] Figure 3 This is a schematic diagram of the structure of an endoscope system provided in an embodiment of this application;

[0024] Figure 4 This is one of the flowcharts illustrating the image processing method provided in the embodiments of this application;

[0025] Figure 5 A second schematic flowchart illustrating the image processing method provided in this application embodiment;

[0026] Figure 6 The third schematic flowchart of the image processing method provided in the embodiments of this application;

[0027] Figure 7 The fourth schematic flowchart of the image processing method provided in the embodiments of this application;

[0028] Figure 8 A schematic diagram of a target image in white light and NEI modes provided for an embodiment of this application;

[0029] Figure 9 Fifth schematic flowchart of the image processing method provided in the embodiments of this application;

[0030] Figure 10 A schematic flowchart of the image processing method provided in this application embodiment is shown in Figure 6.

[0031] Figure 11 Seventh schematic flowchart of the image processing method provided in the embodiments of this application;

[0032] Figure 12 Eighth schematic flowchart of the image processing method provided in the embodiments of this application;

[0033] Figure 13 A flowchart illustrating the image processing method provided in this application embodiment is shown in Figure 9.

[0034] Figure 14 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application;

[0035] Figure 15 This is a schematic diagram of another endoscope system provided in an embodiment of this application. Detailed Implementation

[0036] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.

[0037] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0038] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0039] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0040] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0041] Currently, endoscopes primarily offer two observation modes: white light and special light. White light mode uses a normal white light source for illumination, producing images that more closely match the true color of the observed object, better reproducing the tissue color. However, some detailed features (such as superficial blood vessels, lesions, and bleeding points) are not highlighted in white light images, making it difficult to identify lesions. Special light mode uses narrowband light of a specific wavelength for illumination, producing images that highlight some detailed features of the observed object, facilitating the identification of lesions. However, the colors in special light images are severely distorted, differing significantly from the true color of the observed object.

[0042] For example, taking white light images and NEI images as examples, such as Figure 1 As shown in (a), white light images more closely match the true color of the observed object and can better reproduce the tissue color of the observed object. Figure 1 As shown in (b) in the figure, the superficial blood vessels in the NEI image are clearer compared to the white light image.

[0043] In view of this, embodiments of this application provide an image processing method in which the light source module of an endoscope can output special light and white light in a time-division manner. A special light image obtained by illuminating the observed object with special light is acquired. By analyzing the special light image, target tissue within the observed object's tissue is extracted, and possible lesion areas (i.e., the first region of interest) are identified based on the target tissue. Then, a white light image obtained by illuminating the observed object with white light is acquired. Based on the information extracted from the special light image, image enhancement processing is performed on the corresponding region (i.e., the second region of interest) in the white light image to enhance the display of the target tissue in the white light image. In this way, while better restoring the tissue color of the observed object, the target tissue within the tissue is highlighted in the white light image.

[0044] Before introducing the image processing method provided in the embodiments of this application, the application scenarios of the image processing method will be explained first.

[0045] Figure 2 This is a schematic diagram illustrating an application scenario of an image processing method provided in an embodiment of this application. Figure 3 This is a schematic diagram of the structure of an endoscope system provided in an embodiment of this application.

[0046] The image processing method provided in this application embodiment can be executed by a processing device in an endoscope system. For example... Figure 2 and Figure 3As shown, the endoscope system may include an endoscope (also referred to as an endoscope insertion end) 10 and a processing device 20. The user (i.e., medical personnel) can use the endoscope 10 to acquire images of the tissues inside the observed object (i.e., the patient) and transmit them to the display screen of the processing device 20 for display. The endoscope 10 is used to acquire images of the tissues inside the observed object (i.e., the patient) and to perform operations within the observed object's body. The processing device 20 may include a processing unit 21 and a display screen 22. The processing unit 21 is connected to the endoscope 10 and is used to control the endoscope 10 and to process and transmit the data acquired by the endoscope 10 (such as endoscopic images). The display screen 22 is connected to the processing unit 21 and is used to display the real-time images (i.e., endoscopic images) acquired by the endoscope 10.

[0047] For example, endoscope 10 may include a light source module, a camera module, and an operating component. The light source module is configured to output light with a set spectral characteristics. The camera module is configured to acquire images of the internal tissues of the observed object (i.e., the patient) when the light source module illuminates the observed object with light having the set spectral characteristics, thereby obtaining endoscopic images.

[0048] For example, the processing unit 21 may be a unit or integrated chip capable of executing a series of instructions to process data or control other devices (such as an endoscope). Specifically, the processing unit 21 is used to perform functions such as processing instructions, executing operations, controlling timing, processing data, data conversion, and data transmission. The display screen 22 is used to display images and interactive interfaces in response to the control instructions of the processing unit 21.

[0049] It should be noted that the processing unit 21 and the display screen 22 described above can be integrated into a single processing device. This processing device can be, for example, a personal computer, a laptop computer, a tablet computer, etc. The display screen 22 can also be a standalone device or integrated into other electronic devices, but it is controlled by the processing unit 21.

[0050] In this embodiment, the endoscope system has multiple observation modes, such as a fusion observation mode. In fusion observation mode, the endoscope's light source module outputs white light and at least one special light simultaneously. White light refers to light in the white wavelength band or light in multiple wavelength bands. When the observed object is illuminated with white light, the acquired image can be called a white light image. Special light refers to narrowband light with a spectrum different from that of white light. When the observed object is illuminated with special light, the acquired image can be called a special light image.

[0051] In some examples, the endoscope system has a first fusion mode, namely white light and NEI enhancement mode. In the first fusion mode, the endoscope's light source module outputs white light and a first special light simultaneously, the first special light comprising a narrow green band light with a center wavelength of 540 nm and a narrow blue-violet band light with a center wavelength of 415 nm.

[0052] In practical applications, in the first fusion mode, the endoscope's light source module can alternately output a first special light and white light. A NEI image obtained by illuminating the observed object with the first special light is acquired. By analyzing the NEI image, vascular details within the tissue of the observed object are extracted, and potential lesion areas (i.e., regions of interest) are identified based on these vascular details. Then, a white light image obtained by illuminating the observed object with white light is acquired. Based on the information extracted from the NEI image, image enhancement processing is performed on the corresponding areas in the white light image to enhance the display of vascular details. In this way, while better restoring the tissue color of the observed object, the vascular details within the tissue are highlighted in the white light image.

[0053] In some examples, the endoscopic system has a second fusion mode, namely a white light and REI enhancement mode. In the second fusion mode, the endoscope's light source module outputs white light and a second special light simultaneously. The second special light includes a narrow green light with a center wavelength of 540 nm and narrow red light with center wavelengths of 600 nm and 630 nm, respectively.

[0054] In practical applications, in the second fusion mode, the endoscope's light source module can alternately output a second special light and white light. A REI image obtained by illuminating the observed object with the second special light is acquired. By analyzing the REI image, bleeding points within the tissue of the observed object are identified. Then, a white light image obtained by illuminating the observed object with white light is acquired. Based on the information extracted from the REI image, image enhancement processing is performed on the bleeding points in the white light image to enhance their display. In this way, while better restoring the tissue color of the observed object, the bleeding points within the tissue are highlighted in the white light image.

[0055] In some examples, the endoscopic system features a third fusion mode: a white light, NEI, and REI enhancement mode. In this mode, the endoscope's light source module outputs white light, a first special light, and a second special light simultaneously. By analyzing the NEI image, vascular details within the observed tissue are extracted; by analyzing the REI image, bleeding points within the observed tissue are identified. Based on the information extracted from the NEI and REI images, image enhancement processing is applied to the corresponding areas in the white light image to enhance the display of vascular details and bleeding points within the tissue.

[0056] Of course, in some examples, the endoscopic system can also support white light mode and various special light modes, such as NEI mode and REI mode. In NEI mode, the endoscope's light source module outputs a first special light. Using the acquired NEI images, vascular details of the observed object's internal tissues can be extracted for lesion identification. In REI mode, the endoscope's light source module outputs a second special light. Using the acquired REI images, bleeding points in the observed object's internal tissues can be extracted for lesion identification.

[0057] In some examples, special light modes may also include other observation modes such as narrowband imaging mode, blue laser imaging mode, and fluorescence navigation mode.

[0058] The above is a schematic description of the application scenarios and endoscopic systems of the image processing method provided in the embodiments of this application. The image processing method provided in the embodiments of this application will be specifically described below in conjunction with application scenarios.

[0059] Figure 4 This is a schematic flowchart illustrating an image processing method provided in an embodiment of this application. This image processing method can be executed by the processing device of the endoscope system described in the above embodiments. Figure 4 As shown, the image processing method includes the following steps:

[0060] S401, acquire a special light image.

[0061] In this embodiment, the light source module of the endoscope system can output white light and at least one special light in a time-division multiplexing manner. White light refers to light in the white wavelength band or light in multiple wavelength bands. Special light refers to narrowband light whose spectrum differs from that of white light. A white light image is obtained by capturing an image of the object being observed illuminated by white light. A special light image is obtained by capturing an image of the object being observed illuminated by special light.

[0062] In practical applications, endoscopic systems have multiple operating modes. In different operating modes, the endoscope's light source module outputs specialized light with different spectral characteristics. When the observed object is illuminated with specialized light of different spectral characteristics, different tissue details can be highlighted in the corresponding specialized light images.

[0063] For example, in the first fusion mode, the endoscope's light source module outputs a first special light and white light simultaneously. The first special light includes a green narrowband light with a center wavelength of 540 nm and a blue-violet narrowband light with a center wavelength of 415 nm. In this mode, the acquired special light image is an NEI image.

[0064] For example, in the second fusion mode, the endoscope's light source module outputs a second special light and white light simultaneously. The second special light includes a green narrowband light with a center wavelength of 540 nm and red narrowband light with center wavelengths of 600 nm and 630 nm, respectively. In this mode, the acquired special light image is a REI image.

[0065] For example, in the third fusion mode, the endoscope's light source module outputs a first special light, a second special light, and white light simultaneously. In this mode, the acquired special light images include NEI images and REI images.

[0066] S402 identifies the first region of interest based on the pixel information of each pixel in the special light image.

[0067] The pixel information of a special light image can be used to characterize different tissues of the observed object. The pixel information can include the pixel values ​​of the red channel (R channel), green channel (G channel), and blue channel (B channel) of the pixel, as well as the brightness information (Y value) and chromaticity information (U value and V value) of the pixel.

[0068] The primary region of interest refers to the area that the user (such as a doctor) is concerned about and where lesions may occur. By using pixel information from special light images, different tissues of the observed object can be identified, and thus the primary region of interest can be identified.

[0069] For example, in the first fusion mode, the first region of interest is the region where the blood vessel density meets the criteria. Specifically, blood vessels can be identified through the pixel information of each pixel in the NEI image, and the region where the blood vessel density meets the criteria is then selected as the first region of interest.

[0070] For example, in the second fusion mode, the first region of interest is the area where the bleed point is located. Specifically, the bleed point can be identified by the pixel information of each pixel in the REI image, and the area where the bleed point is located is taken as the first region of interest.

[0071] For example, in the third fusion mode, the identified first region of interest includes the region where the blood vessel density meets the condition and the region where the bleeding point is located. Specifically, blood vessels are identified by the pixel information of each pixel in the NEI image, and bleeding points are identified by the pixel information of each pixel in the REI image. The region where the blood vessel density meets the condition and the region where the bleeding point is located are taken as the first region of interest.

[0072] In some embodiments, the process of identifying the first region of interest may include the following steps: determining the feature information of each pixel based on the pixel information of each pixel in the special light image; identifying the target pixel corresponding to the target tissue based on the feature information of each pixel; and determining the first region of interest based on the target pixel.

[0073] Specifically, by using the feature information of a pixel, it can be determined whether the pixel is a target pixel, that is, whether the tissue corresponding to the pixel belongs to the target tissue. Based on the target pixel, the first region of interest can be determined.

[0074] In practice, the feature information to be acquired can be determined based on the difference between the image features (such as color or brightness) of the target tissue to be identified and other tissues in the special light image. The feature information can then be used as a basis to determine whether the pixel is the target pixel.

[0075] For example, in the first fusion mode, the target tissue to be identified is a blood vessel, and the target pixels to be identified are blood vessel pixels. Based on the characteristics of special light images (such as NEI images), mucosal tissue is typically brighter, while blood vessel tissue is typically darker. Therefore, the feature information to be determined can be a high-frequency brightness component characterizing brightness changes; through the high-frequency brightness components of each pixel, blood vessel pixels can be identified.

[0076] For example, in the second fusion mode, the target tissue to be identified is a bleeding point, and the target pixel to be identified is a bleeding point pixel. Typically, in special lighting images (such as REI images), the display color of fresh blood (i.e., the preset blood) is known. Therefore, by comparing the display color of each pixel with the display color of fresh blood, bleeding point pixels can be identified from each pixel. That is, the color deviation information representing the difference between the display color of a pixel and the display color of fresh blood can be used as the feature information to be determined to identify bleeding point pixels.

[0077] For example, in the third fusion mode, the target tissue to be identified includes blood vessels and bleeding points, the target pixels to be identified include blood vessel pixels and bleeding point pixels, and the feature information to be determined may include high-frequency luminance components and color deviation information.

[0078] S403, acquire white light image.

[0079] Among them, white light images are obtained by taking pictures of the observed object illuminated by white light.

[0080] It is understandable that white light images and special light images are acquired in a time-division manner, with the acquisition time of white light images potentially later than that of special light images. Both target the same anatomical sites and fields of view; the difference lies in the spectral information they contain for display.

[0081] S404, perform image enhancement processing on the white light image to enhance the display of the second region of interest in the white light image, and / or, mark the second region of interest in the white light image to obtain the target image.

[0082] The second region of interest refers to the region in the white light image that corresponds to the first region of interest. In other words, the second region of interest is the region in the white light image that the user (such as a doctor) is concerned about and that may have lesions.

[0083] For example, in the first fusion mode, the second region of interest is the region in the white light image where the blood vessel density meets the condition. Similarly, in the second fusion mode, the second region of interest is the region in the white light image where the hemorrhage point is located.

[0084] In this embodiment, after identifying the first region of interest using a special light image, the corresponding second region of interest in the white light image is processed based on the information extracted from the first region of interest, so that the second region of interest stands out in the white light image.

[0085] In some examples, after identifying the first region of interest using a special light image, image enhancement processing can be performed on the corresponding second region of interest in the white light image to enhance its display. In some examples, after identifying the first region of interest, the corresponding second region of interest in the white light image can also be marked. Alternatively, image enhancement processing can be performed on the corresponding second region of interest in the white light image, and the second region of interest can be marked.

[0086] In some embodiments, when performing image enhancement processing on a white light image, image enhancement processing can be performed on a first pixel in the white light image to enhance the display of a second region of interest in the white light image. Here, the first pixel refers to the pixel in the white light image that corresponds to the target pixel in the special light image, and the first pixel belongs to the second region of interest.

[0087] For example, in the first fusion mode, based on information from the first region of interest extracted from the NEI image, the blood vessel pixels (i.e., the first pixels) in the second region of interest of the white light image are enhanced.

[0088] For example, in the second fusion mode, based on the information of the first region of interest extracted from the REI image, the bleed pixel (i.e. the first pixel) in the second region of interest of the white light image is enhanced.

[0089] For example, in the third fusion mode, based on the information of the first region of interest extracted from the NEI image, the blood vessel pixels (i.e., the first pixel) in the second region of interest of the white light image are enhanced, and based on the information of the first region of interest extracted from the REI image, the bleeding point pixels (i.e., the first pixel) in the second region of interest of the white light image are enhanced.

[0090] In some examples, the first pixel can be enhanced by adjusting the target image parameters of the first pixel.

[0091] For example, image processing can be performed on the target image parameters of the first pixel based on empirical values.

[0092] For example, image processing can also be performed on the target image parameters of the first pixel based on the target probability of the target pixel. Here, the target probability is used to characterize the confidence level of determining that the tissue corresponding to the pixel belongs to the target tissue. It can be understood that the first pixel is the pixel in the white light image corresponding to the target pixel; therefore, the target probability of the target pixel can also represent the target probability of the first pixel, that is, the probability that the tissue corresponding to the first pixel belongs to the target tissue.

[0093] Optionally, determining the target probability of a target pixel may include: after identifying the target pixel, determining the current reference probability of the target pixel based on the feature information of the target pixel; acquiring a historical special light image, which includes a reference pixel corresponding to the target pixel; determining the historical reference probability of the reference pixel based on the historical special light image; and performing a weighted fusion process on the current reference probability and the historical reference probability to obtain the target probability of the target pixel.

[0094] The historical special light images were acquired earlier than the currently acquired special light images. It is understood that the anatomical sites and fields of view targeted by the historical and currently acquired special light images may be the same or different; furthermore, the target pixels in the currently acquired special light images and the reference pixels in the historical special light images correspond to the same location of the observed object.

[0095] In practical implementation, after acquiring historical special light images, reference pixels belonging to the target organization can be identified within these images. Based on the feature information of these reference pixels, their historical reference probabilities are determined. After acquiring special light images, target pixels belonging to the target organization are identified. Based on the feature information of these target pixels, their current reference probabilities are determined. The current and historical reference probabilities are then weighted and fused to obtain the target probability of the target pixel. This process prevents missed detections caused by jumps during image acquisition and improves the accuracy of target probability identification.

[0096] Specifically, taking the first fusion mode as an example, the color saturation of the first pixel can be increased by adjusting the target image parameters of the first pixel. These target image parameters can include the component values ​​of the blue and green channels of the first pixel. Therefore, the color saturation of the first pixel is increased by decreasing the component values ​​of the blue and green channels.

[0097] For example, the target image parameters can also be the component values ​​of the red channel of the first pixel. In this case, the color saturation of the first pixel is increased by increasing the component values ​​of the red channel of the first pixel.

[0098] Optionally, in the first fusion mode, the target pixel is a blood vessel pixel, and the target probability can be a blood vessel probability, used to characterize the confidence that the pixel belongs to a blood vessel. Therefore, the target image parameters of the first pixel can be adjusted based on the blood vessel probability of the first pixel.

[0099] In determining the probability of blood vessels, the probability of blood vessels in a target pixel can be determined based on the high-frequency brightness component of the target pixel. The probability of blood vessels in a target pixel is directly proportional to (i.e., positively correlated with) the absolute value of the high-frequency brightness component of the target pixel.

[0100] Specifically, the process of image enhancement processing of white light images in the first fusion mode can be found in Method Example 1 below, and will not be repeated here.

[0101] Taking the second fusion mode as an example, the color saturation of the first pixel can be increased by adjusting the target image parameters of the first pixel. The target image parameters can include the saturation value (S value) of the first pixel. Therefore, by increasing the saturation value (S value) of the first pixel, the color saturation of the first pixel is increased.

[0102] The color brightness of the first pixel can also be reduced by adjusting the target image parameters of the first pixel. These target image parameters can include the luminance component (Y value) of the first pixel. Therefore, by reducing the luminance component (Y value) of the first pixel, the color brightness of the first pixel is reduced.

[0103] Optionally, in the second fusion mode, the target pixel is a bleed pixel, and the target probability can be a bleed probability, used to characterize the confidence that the tissue to which the pixel belongs is a bleed. Therefore, the target image parameters of the first pixel can be adjusted based on the bleed probability of the first pixel.

[0104] In determining the probability of bleed points, the probability of bleed points of target pixels can be determined based on the color deviation information of the target pixels. The probability of bleed points of target pixels is negatively correlated with the color deviation information of the target pixels.

[0105] Specifically, the process of image enhancement processing of white light images in the second fusion mode can be found in Method Example 2 below, and will not be repeated here.

[0106] In other embodiments, when performing image enhancement processing on a white light image, image processing can be performed on the second pixel in the white light image to enhance the display of the second region of interest in the white light image. Here, the second pixel refers to any pixel in the white light image other than the first pixel, and the second pixel does not belong to the second region of interest.

[0107] In other words, after acquiring a white light image, the first pixel in the white light image that belongs to the second region of interest can be directly enhanced to improve the display of the second region of interest. Alternatively, the second pixel in the white light image that does not belong to the second region of interest can also be processed to enhance the display of the second region of interest.

[0108] In some examples, the first pixel can be enhanced by adjusting the target image parameters of the second pixel.

[0109] For example, taking the first fusion mode as an example, the color saturation (i.e., red saturation) of the second pixel can be reduced by adjusting the target image parameters of the second pixel, thereby increasing the color saturation of the first pixel. The target image parameters may include the component values ​​of the blue and green channels of the second pixel. Therefore, by increasing the component values ​​of the blue and green channels of the second pixel, the red saturation of the second pixel is reduced, thereby increasing the red saturation of the first pixel.

[0110] Specifically, image processing can be performed on the target image parameters of the second pixel based on empirical values.

[0111] Specifically, the process of image enhancement processing of white light images in the first fusion mode can be found in Method Example 1 below, and will not be repeated here.

[0112] For example, taking the second fusion mode as an example, the color saturation of the second pixel can be reduced by adjusting the target image parameters of the second pixel, thereby increasing the color saturation of the first pixel. The target image parameters may include the saturation value (i.e., the S value) of the second pixel. Therefore, by reducing the saturation value (i.e., the S value) of the second pixel, the color saturation of the second pixel is reduced, thereby increasing the color saturation of the first pixel.

[0113] Alternatively, the color brightness of the second pixel can be increased by adjusting the target image parameters of the second pixel, thereby increasing the brightness difference between the first pixel and its surrounding second pixels. The target image parameters can include the brightness component (Y value) of the second pixel. Therefore, by increasing the brightness component (Y value) of the second pixel, the color brightness of the second pixel is increased, thus increasing the brightness difference between the first pixel and its surrounding second pixels.

[0114] Specifically, the process of image enhancement processing of white light images in the second fusion mode can be found in Method Example 2 below, and will not be repeated here.

[0115] In some embodiments, when marking a second region of interest in a white light image, a bounding box can be added to the white light image based on the position of the first region of interest in a special light image to mark the corresponding second region of interest in the white light image. This allows for faster and more intuitive marking of the location of the second region of interest, making it easier for users to view it.

[0116] In other embodiments, when marking a second region of interest in a white light image, the contour pixels of the second region of interest can be identified and enhanced to mark the second region of interest in the white light image.

[0117] For example, such as Figure 5 As shown, the process of marking the second region of interest in a white light image may include the following steps:

[0118] S501, for each target pixel, determine multiple adjacent pixels located in a preset neighborhood of the target pixel.

[0119] For example, the preset neighborhood can be an 8-neighborhood or a 4-neighborhood. The type of preset neighborhood can be set according to the actual situation, and this application embodiment does not limit this.

[0120] S502, if the target probability of the target pixel is greater than or equal to the preset probability threshold, and there are adjacent pixels in the neighborhood of the target pixel whose target probability is less than the preset probability threshold, then the target pixel is determined as a contour pixel.

[0121] If the target probability of a target pixel is greater than or equal to a preset probability threshold, it means that the target pixel belongs to the target organization; if the target probability of a target pixel is less than the preset probability threshold, it means that the target pixel does not belong to the target organization. Based on this, the contour pixels of the first region of interest corresponding to the target organization can be determined.

[0122] S503, based on the position of the contour pixels in the special light image, performs image enhancement processing on the corresponding pixels to be processed in the white light image to mark the second region of interest in the white light image.

[0123] Among them, the pixels to be processed in the white light image are the contour pixels of the second region of interest in the white light image.

[0124] In this embodiment, pixel-level precise marking is achieved through contour recognition and enhancement, preserving the morphological features of the target region, which is suitable for refined diagnosis and treatment planning. The specific implementation methods for marking the second region of interest under different working modes are described in the embodiments below and will not be repeated here.

[0125] The above is a general description of the image processing method provided in the embodiments of this application. The following describes the image processing method illustratively using different working modes as examples.

[0126] Method Example 1

[0127] The following section uses the working mode of the endoscope system as the first fusion mode (i.e., white light and NEI enhancement mode) as an example to elaborate on this image processing method.

[0128] In the first fusion mode, the endoscope's light source module outputs a first special light and white light simultaneously. When the observed object is illuminated with the first special light, the acquired special light image is a NEI image, the target tissue to be identified is blood vessels, and the first region of interest to be identified is the region where the blood vessel density meets the condition.

[0129] Figure 6 This is a schematic flowchart illustrating another image processing method provided in an embodiment of this application. Figure 6 As shown, the image processing method may include the following steps:

[0130] S601, acquire the NEI image (i.e., special light image).

[0131] S602, based on the pixel information of each pixel in the NEI image, identify the first region of interest, wherein the first region of interest is the region where the blood vessel density meets the condition.

[0132] The pixel information of a pixel can be used to characterize different tissues of the observed object. For example, the pixel information may include the pixel values ​​of the red channel (R channel), green channel (G channel), and blue channel (B channel) of the pixel.

[0133] For example, in the first fusion mode, the first region of interest is the region where the blood vessel density meets the condition. Specifically, blood vessels can be identified through the pixel information of each pixel in the NEI image, and the region where the blood vessel density meets the condition is then taken as the first region of interest.

[0134] In this embodiment, the first fusion mode is used to identify regions where the blood vessel density meets the criteria. Based on the characteristics of NEI images, mucosal tissue is typically brighter, while blood vessel tissue is typically darker. Therefore, after acquiring the NEI image, high-frequency brightness components characterizing brightness changes are extracted from the NEI image. Based on these high-frequency brightness components, blood vessel pixels (i.e., target pixels) are identified, and then regions where the blood vessel density meets the criteria are identified based on these blood vessel pixels.

[0135] In some examples, such as Figure 7 As shown, the process of identifying the first region of interest may include the following steps:

[0136] S701, based on the pixel information of each pixel in the NEI image, determine the brightness information of each pixel, and remove the low-frequency brightness component from the brightness information of each pixel to obtain the high-frequency brightness component of each pixel.

[0137] The low-frequency luminance component can characterize smooth, uniformly colored background areas in the NEI image. The high-frequency luminance component can characterize detailed areas in the NEI image with rapid or drastic changes in brightness. In this embodiment, the smooth, uniformly colored areas in the NEI image represent mucosal tissue, while the areas with rapid or drastic changes in brightness represent vascular tissue. Based on this, vascular pixels (i.e., target pixels) can be identified through the high-frequency luminance components of each pixel.

[0138] In practice, the NEI image is first converted into a grayscale image (i.e., a brightness image), which is used to represent the brightness information of each pixel.

[0139] Specifically, by calculating the brightness information of each pixel in the NEI image, the corresponding grayscale image of the NEI image is obtained.

[0140] For example, the brightness information of each pixel in the NEI image can be calculated according to the following formula (1).

[0141] (1)

[0142] Where Y represents the grayscale value (i.e., brightness value) of a pixel in a grayscale image; R, G, and B represent the pixel values ​​of the red, green, and blue channels in the NEI image, respectively.

[0143] Then, the grayscale image is low-pass filtered to obtain the low-frequency luminance components of each pixel. Next, the low-frequency luminance components are removed from the luminance information of each pixel to obtain the high-frequency luminance components of each pixel.

[0144] Specifically, a low-pass filter is used to process the grayscale image to obtain the low-frequency luminance component of each pixel. For each pixel, the corresponding low-frequency luminance component is subtracted from the pixel's grayscale value to obtain the pixel's high-frequency luminance component.

[0145] For example, the low-pass filter can be a Gaussian filter or a bilateral filter. The specific type of low-pass filter is not limited in the embodiments of this application.

[0146] S702, the pixels in the NEI image whose high-frequency brightness components meet the first preset condition are identified as blood vessel pixels.

[0147] The first preset condition is used to determine whether a pixel is a blood vessel pixel. For example, the first preset condition is that the absolute value of the high-frequency luminance component of the pixel is greater than a preset luminance threshold. The preset luminance threshold can be a positive value. When the absolute value of the high-frequency luminance component of a pixel is greater than the preset luminance threshold, the pixel is determined to be a blood vessel pixel.

[0148] Understandably, in NEI images, blood vessels are darker than mucosal tissue; therefore, the high-frequency brightness components of blood vessels (i.e., blood vessel pixels) are typically negative. Based on this, after determining the high-frequency brightness components of each pixel, the absolute value of the negative high-frequency brightness components is compared with a preset brightness threshold. Pixels with high-frequency brightness components greater than the preset brightness threshold are identified as blood vessel pixels (i.e., target pixels).

[0149] S703, determine the percentage of blood vessel pixels in each image region of the NEI image.

[0150] S704, Image regions whose quantity percentage is greater than or equal to a preset percentage threshold are identified as the first region of interest.

[0151] Specifically, by using a preset sliding window, the proportion of blood vessel pixels in each sliding window is calculated sequentially. If the proportion of blood vessel pixels in the sliding window is greater than or equal to a preset proportion threshold, the image region corresponding to that sliding window is determined as the first region of interest.

[0152] In some embodiments, a pre-trained recognition model can be used to perform recognition analysis on the NEI image to identify the first region of interest. The pre-trained recognition model can be YOLO, Unet, Deeplab, etc.

[0153] S603, acquire white light image.

[0154] S604, perform image enhancement processing on the white light image to enhance the display of the second region of interest in the white light image, and / or, mark the second region of interest in the white light image to obtain a target image, wherein the second region of interest corresponds to the first region of interest.

[0155] In this embodiment, after identifying the first region of interest using the NEI image, the corresponding second region of interest in the white light image is processed based on the information extracted from the first region of interest, so that the second region of interest stands out in the white light image. In this embodiment, the second region of interest is the area in the white light image where the blood vessel density meets the specified condition.

[0156] The following is a schematic illustration of the image enhancement process for white light images.

[0157] In some examples, when performing image enhancement processing on a white light image, the color saturation of a first pixel in the white light image can be adjusted to enhance the display of the second region of interest in the white light image, thereby obtaining the target image. Here, the first pixel refers to the pixel in the white light image that corresponds to the blood vessel pixel (i.e., the target pixel) in the NEI image, and the first pixel belongs to the second region of interest.

[0158] The color saturation of the first pixel is related to the component values ​​of the red channel (R channel), blue channel (B channel), and green channel (G channel) of the first pixel.

[0159] In one possible implementation, for each first pixel in the white light image, the color saturation (i.e., red saturation) of the first pixel can be enhanced by reducing the component values ​​of the blue and green channels of the first pixel, thereby enhancing the color saturation of the second region of interest.

[0160] For example, when performing image enhancement processing on the first pixel in a white light image, based on experience, the blue channel component value of the first pixel can be reduced to a first preset value, and the green channel component value of the first pixel can be reduced to a second preset value. In this way, after acquiring the white light image, by reducing the blue and green channel components of the blood vessel pixel, the display color of the blood vessel pixel is mainly determined by the red channel component. This effectively improves the red saturation of the blood vessel pixel, making the second region of interest, where the blood vessel tissue is located, stand out in the white light image.

[0161] For example, when performing image enhancement processing on the first pixel in a white light image, the component value of the blue channel of the first pixel can be reduced to the first blue component value, and the component value of the green channel of the first pixel can be reduced to the first green component value. The first blue component value and the first green component value are determined based on the blood vessel probability of the first pixel (i.e., the blood vessel pixel).

[0162] Specifically, the value of the first blue component is negatively correlated with the probability of blood vessels in the first pixel. A higher probability of blood vessels in the first pixel results in a lower first blue component value, meaning a greater decrease in the blue channel component value of the first pixel. Conversely, a lower probability of blood vessels in the first pixel results in a higher first blue component value, meaning a smaller decrease in the blue channel component value of the first pixel.

[0163] Similarly, the value of the first green component is negatively correlated with the probability of blood vessels in the first pixel. A higher probability of blood vessels in the first pixel results in a lower first green component value, meaning a greater decrease in the green channel component value of the first pixel. Conversely, a lower probability of blood vessels in the first pixel results in a higher first green component value, meaning a smaller decrease in the green channel component value of the first pixel.

[0164] In other words, when performing image enhancement processing on the first pixel in a white light image, for the first pixel with a higher probability of blood vessels, the component values ​​of the blue and green channels of the first pixel are reduced by a larger amount; while for the first pixel with a lower probability of blood vessels, the component values ​​of the blue and green channels of the first pixel are reduced by a smaller amount. This makes the first pixel with a higher probability of blood vessels in the second area of ​​interest have higher red saturation, thereby highlighting the first pixel with a higher probability of blood vessels and helping users to identify lesion areas.

[0165] More specifically, a linear approach can be adopted, reducing the component value of the blue channel of the first pixel to the first blue component value, and reducing the component value of the green channel of the first pixel to the first green component value.

[0166] For example, in linear form, the value of the first blue component can be determined according to the following formula (2).

[0167] (2)

[0168] in, This is the first blue component value, which is the adjusted blue channel component value; The probability of a blood vessel in the first pixel; This is the blue channel component value of the first pixel, i.e., the blue channel component value before adjustment; , These are preset parameters. .

[0169] For example, in linear form, the first green component value can be determined according to the following formula (3).

[0170] (3)

[0171] in, This is the first green component value, which is the adjusted green channel component value; This is the component value of the green channel of the first pixel, that is, the component value of the green channel before adjustment; , These are preset parameters. .

[0172] More specifically, an exponential approach can be used to reduce the blue channel component value of the first pixel to the first blue component value, and reduce the green channel component value of the first pixel to the first green component value.

[0173] For example, in exponential form, the value of the first blue component can be determined according to the following formula (4).

[0174] (4)

[0175] For example, in linear form, the first green component value can be determined according to the following formula (5).

[0176] (5)

[0177] When adjusting exponentially, if the probability of blood vessels in the first pixel is low, the adjustment coefficient is close to 1. This means the adjusted first blue component value is close to the original blue channel component value. However, when the probability of blood vessels in the first pixel exceeds a certain threshold, the adjustment coefficient drops sharply. The adjusted first blue component value is significantly lower than the original blue channel component value, resulting in higher red saturation for the first pixel with a higher probability of blood vessels. This highlights the first pixel with a higher probability of blood vessels, helping users identify lesion areas. Furthermore, adjusting the color saturation of the first pixel exponentially better aligns with the non-linear perception of color changes in the human eye, making the enhanced white light image more closely resemble the actual colors of human tissue while highlighting vascular tissue.

[0178] In some examples, when determining the blood vessel probability of the first pixel (i.e., the blood vessel pixel), it can be determined based on the high-frequency brightness component of the target pixel, wherein the blood vessel probability of the blood vessel pixel is proportional to the absolute value of the high-frequency brightness component of the target pixel.

[0179] For example, the probability of a blood vessel in the first pixel (i.e., the blood vessel pixel) can be determined according to the following formula (6).

[0180] (6)

[0181] in, This represents the probability of a blood vessel in the pixel located in the i-th row and j-th column of the NEI image (i.e., any blood vessel pixel). This represents the high-frequency brightness component of the blood vessel pixel (i.e., any blood vessel pixel) located in the i-th row and j-th column of the NEI image. This indicates the preset brightness threshold.

[0182] In one possible implementation, for each first pixel in the white light image, the color saturation (i.e., red saturation) of the first pixel can be enhanced by increasing the component value of the red channel of the first pixel, thereby enhancing the color saturation of the second region of interest.

[0183] For example, when performing image enhancement processing on the first pixel in a white light image, the red channel component value of the first pixel can be increased to a third preset value based on experience. In this way, after acquiring the white light image, by increasing the red channel component value of the blood vessel pixel, the display color of the blood vessel pixel is mainly determined by the red channel component. This can effectively improve the red saturation of the blood vessel pixel, making the second area of ​​interest where the blood vessel tissue is located stand out in the white light image.

[0184] For example, when performing image enhancement processing on the first pixel in a white light image, the component value of the red channel of the first pixel can be increased to a first red component value, wherein the first red component value is determined based on the blood vessel probability of the first pixel (i.e., the blood vessel pixel).

[0185] The value of the first red component is positively correlated with the probability of blood vessels in the first pixel. A higher probability of blood vessels in the first pixel results in a higher value of the first red component, meaning a greater increase in the red channel component value of the first pixel. Conversely, a lower probability of blood vessels in the first pixel results in a lower value of the first red component, meaning a smaller increase in the red channel component value of the first pixel.

[0186] More specifically, a linear approach can be used to increase the red channel component value of the first pixel to the first red component value. Alternatively, an exponential approach can be used to increase the red channel component value of the first pixel to the first red component value.

[0187] In this way, when performing image enhancement processing on the first pixel in the white light image, the red channel component value of the first pixel with a higher probability of blood vessels is increased by a larger amount; while the red channel component value of the first pixel with a lower probability of blood vessels is increased by a smaller amount. This makes the first pixel with a higher probability of blood vessels in the second area of ​​interest have higher red saturation, thereby highlighting the first pixel with a higher probability of blood vessels and helping users to identify lesion areas.

[0188] In other examples, when performing image enhancement processing on a white light image, image processing can be applied to the second pixel in the white light image to enhance the display of the second region of interest in the white light image, thereby obtaining the target image. Here, the second pixel refers to any pixel in the white light image other than the first pixel, and the second pixel does not belong to the second region of interest.

[0189] In one possible implementation, for each second pixel in the white light image, the color saturation (i.e., red saturation) of the second pixel can be reduced by increasing the component values ​​of the blue and green channels of the second pixel, thereby increasing the difference in red saturation between the second pixel and the first pixel, thus enhancing the color saturation of the first pixel and consequently enhancing the color saturation of the second region of interest.

[0190] For example, when performing image processing on the second pixel in a white light image, the component value of the blue channel of the second pixel can be increased to a fourth preset value and the component value of the green channel of the second pixel can be increased to a fifth preset value based on experience.

[0191] In this way, after acquiring the white light image, for the second pixel outside the second region of interest, the color saturation (i.e., red saturation) of the second pixel is reduced by increasing the blue and green channel components of the second pixel, thereby increasing the difference in red saturation between the second pixel and the first pixel, which enhances the color saturation of the first pixel. This can effectively improve the red saturation of the blood vessel pixel, making the second region of interest where the blood vessel tissue is located stand out in the white light image.

[0192] In one possible implementation, for each second pixel in the white light image, the color saturation (i.e., red saturation) of the second pixel can be reduced by decreasing the component value of the red channel of the second pixel, thereby increasing the difference in red saturation between the second pixel and the first pixel, thus enhancing the color saturation of the first pixel, and further enhancing the color saturation of the second region of interest.

[0193] For example, when performing image enhancement processing on the second pixel in a white light image, the component value of the red channel of the second pixel can be reduced to a sixth preset value based on experience.

[0194] In some embodiments, after image enhancement processing is performed on the white light image, a second region of interest in the white light image can be marked to obtain the target image.

[0195] In some examples, the location of the second region of interest in the white light image is determined based on the location of the first region of interest in the NEI image, and then the second region of interest is marked with a rectangle based on the location of the second region of interest in the white light image.

[0196] For example, Figure 8 The image shows white light images processed in white light and NEI enhancement modes, such as... Figure 8 As shown, in the white light image, the area marked by the rectangle is the second region of interest, and the second region of interest has a higher color saturation than the surrounding area, with the blood vessels appearing redder and more vibrant.

[0197] In other examples, for each blood vessel pixel, multiple neighboring pixels within a preset neighborhood of the blood vessel pixel are identified. If the blood vessel probability of the blood vessel pixel is greater than or equal to a preset blood vessel probability threshold, and among the multiple neighboring pixels within the neighborhood of the blood vessel pixel, there are neighboring pixels with a blood vessel probability less than the preset blood vessel probability threshold, then the blood vessel pixel is identified as a contour pixel. Based on the position of the contour pixel in the NEI image, image enhancement processing is performed on the corresponding pixels to be processed in the white light image to mark the second region of interest in the white light image.

[0198] Specifically, the color value of the pixel to be processed is set to the preset outline appearance.

[0199] S605, Display the target image.

[0200] In this embodiment, under both white light and NEI enhancement modes, after acquiring the NEI image, vascular pixels are identified based on the NEI image, and a first region of interest whose vascular density meets the condition is identified. Based on the information of the vascular pixels in the first region of interest, image enhancement processing is performed on the white light image to enhance the display of the second region of interest whose vascular density meets the condition in the white light image. This increases the red saturation of the vascular pixels in the second region of interest in the white light image, making the vascular pixels stand out in the white light image and helping users to distinguish lesion areas.

[0201] Method Example 2

[0202] The following section uses the second fusion mode (i.e., white light and REI enhancement mode) of the endoscope system as an example to elaborate on this image processing method.

[0203] In the second fusion mode, the endoscope's light source module outputs a second special light and white light simultaneously. When the observed object is illuminated with the second special light, the acquired special light image is a REI image, the target tissue to be identified is the bleeding point, and the first region of interest to be identified is the area where the bleeding point is located.

[0204] Figure 9 This is a schematic flowchart illustrating another image processing method provided in an embodiment of this application. Figure 9 As shown, the image processing method may include the following steps:

[0205] S901, acquire REI image (i.e., special light image).

[0206] S902, based on the pixel information of each pixel in the REI image, identify the first region of interest, where the first region of interest is the area where the bleeding point is located.

[0207] The pixel information can be used to characterize different tissues of the observed object. For example, the pixel information may include the pixel's luminance information (i.e., Y value) and chromaticity information (i.e., U value and V value).

[0208] For example, in the second fusion mode, the first region of interest is the region where the bleed point is located. Specifically, the bleed point can be identified using the pixel information of each pixel in the REI image, and the region where the bleed point is located is taken as the first region of interest.

[0209] In this embodiment, the second fusion mode is used to identify the region where the bleeding point is located. Typically, the color of fresh blood in a REI image is known. Therefore, by comparing the color of each pixel with the color of fresh blood, the bleeding point pixel can be identified from each pixel. That is, the color deviation information representing the difference between the color of a pixel and the color of fresh blood can be used as the feature information to be determined to identify the bleeding point pixel.

[0210] In some examples, such as Figure 10 As shown, the process of identifying the first region of interest may include the following steps:

[0211] S1001, Determine the YUV domain information of each pixel based on the pixel information of each pixel in the REI image.

[0212] Specifically, after acquiring the REI image, the REI image is converted from the RGB space to the YUV domain to obtain the YUV image corresponding to the REI image (denoted as image I). The YUV image corresponding to the REI image contains the YUV domain information of each pixel, namely the Y value, U value and V value of the pixel.

[0213] S1002, determine the color deviation information of each pixel based on the YUV domain information of each pixel.

[0214] The color deviation information is used to characterize the difference between the displayed color of a pixel and the displayed color of fresh blood. The smaller the color deviation information, the closer the displayed color of the pixel is to the displayed color of fresh blood, meaning that the pixel is more likely to be a bleeding point pixel.

[0215] For example, for each pixel, the deviation of the pixel's YUV domain information from the preset blood YUV domain information can be used as the pixel's color deviation information.

[0216] Specifically, the color deviation information of the pixel can be determined according to the following formula (7).

[0217] (7)

[0218] in, This represents the distance between the YUV domain information of a pixel and the preset blood YUV domain information, i.e., color deviation information; , , Represents the YUV field information of a pixel; , , This indicates the preset blood YUV domain information.

[0219] S1003, pixels in the REI image whose color deviation information is less than or equal to a preset color deviation threshold are identified as bleed pixels.

[0220] Understandably, if the color deviation of a pixel is less than or equal to a preset color deviation threshold, it means that the displayed color of that pixel is relatively close to the displayed color of fresh blood, and thus the pixel is determined to be a bleed point pixel. Otherwise, it means that the displayed color of that pixel differs significantly from the displayed color of fresh blood, and the pixel is not a bleed point pixel.

[0221] After identifying the bleed pixels in the REI image, the region where the bleed pixels are located is determined as the first region of interest.

[0222] S903, acquire white light image.

[0223] S904, perform image enhancement processing on the white light image to enhance the display of the second region of interest in the white light image, and / or, mark the second region of interest in the white light image to obtain a target image, wherein the second region of interest corresponds to the first region of interest.

[0224] In this embodiment, after identifying the first region of interest using the REI image, the corresponding second region of interest in the white light image is processed based on the extracted information of the first region of interest, so that the second region of interest stands out in the white light image. In this embodiment, the second region of interest is the area where the bleed point is located in the white light image.

[0225] The following is a schematic illustration of the image enhancement process for white light images.

[0226] In some embodiments, when performing image enhancement processing on a white light image, image enhancement processing can be performed on a first pixel in the white light image to enhance the display of a second region of interest in the white light image, thereby obtaining a target image. Here, the first pixel refers to the pixel in the white light image that corresponds to the bleed pixel (i.e., the target pixel) in the REI image.

[0227] In some examples, image enhancement processing of the first pixel in a white light image to obtain the target image may include increasing the color saturation of the first pixel.

[0228] For example, such as Figure 11 As shown, the process of increasing the color saturation of the first pixel can include the following steps:

[0229] S1101, convert the white light image from RGB space to HSV space to obtain the HSV image corresponding to the white light image.

[0230] Specifically, after acquiring the white light image, the white light image is converted from the RGB space to the HSV space to obtain the corresponding HSV image. The HSV image contains the H value (hue or saturation), S value (saturation), and V value (brightness) of each pixel. The H value represents the color type, the S value represents the color purity, and the V value represents the color brightness.

[0231] S1102, increase the saturation value (i.e. S value) of the corresponding first pixel in the HSV image to the first saturation value to obtain the processed HSV image.

[0232] In the processed HSV image, the saturation of the first pixel is greater than that of the first pixel in the original HSV image.

[0233] It is understandable that the S value (i.e., saturation value) of each pixel represents the color purity or vividness of that pixel. By increasing the S value of the first pixel, the saturation of the first pixel is improved, making the bleed point redder and more vivid, thus making it stand out from other surrounding tissues.

[0234] For example, the first saturation value can be preset, that is, when processing the HSV image, the S value of each first pixel is increased to the preset first saturation value to increase the color saturation of the first pixel.

[0235] For example, a first saturation value can be determined based on the bleed probability of the target pixel (bleed pixel), and the S value of the corresponding first pixel in the HSV image can be increased to the first saturation value to obtain the processed HSV image.

[0236] The magnitude of the first saturation value is positively correlated with the bleed probability of the target pixel (i.e., the bleed probability of the first pixel). Understandably, in the HSV color space, the saturation (S value) typically ranges from 0 to 1. Correspondingly, the higher the bleed probability of the first pixel, the higher the first saturation value, and the closer it is to 1; that is, the greater the increase in the S value of the first pixel. Conversely, the lower the bleed probability of the first pixel, the lower the first saturation value; that is, the smaller the increase in the S value of the first pixel.

[0237] In other words, when increasing the color saturation of the first pixel in the white light image, the S value of the first pixel with a higher probability of bleeding is increased by a larger amount; while the S value of the first pixel with a lower probability of bleeding is increased by a smaller amount. This makes the first pixel with a higher probability of bleeding in the second area of ​​interest have higher red saturation, thereby highlighting the first pixel with a higher probability of bleeding and helping the user to identify the lesion area.

[0238] More specifically, the first saturation value can be determined according to the following formula (8).

[0239] (8)

[0240] in, This represents the first saturation value of any first pixel (i.e., the first pixel in the i-th row and j-th column of the HSV image); This represents the saturation of the first pixel in the i-th row and j-th column of the HSV image (i.e., the saturation before image processing). This represents the probability of bleed at the target pixel in the i-th row and j-th column of the REI image (the corresponding first pixel in the white light image).

[0241] Understandably, referring to formula (8), when When, that is, for pixels that are not bleed pixels, the first saturation value is equal to the actual S value of that pixel; that is, for pixels that are not bleed pixels, no saturation adjustment is performed. At that time, for the first pixel that is 100% bleed point, the first saturation value is 1, and the color of the pixel is redder and more vivid.

[0242] S1103, convert the processed HSV image from HSV space to RGB space to obtain the target image.

[0243] After increasing the saturation value (i.e., S value) of the corresponding first pixel in the HSV image to the first saturation value, the processed HSV image is converted from HSV space to RGB space to obtain the target image.

[0244] In some examples, image enhancement processing is performed on the first pixel in a white light image to obtain a target image, which may include: reducing the brightness of the first pixel to obtain the target image.

[0245] For example, such as Figure 12 As shown, the process of reducing the brightness of the first pixel may include the following steps:

[0246] S1201 converts the white light image from RGB space to YUV space to obtain the YUV image corresponding to the white light image.

[0247] Specifically, after acquiring the white light image, the white light image is converted from the RGB space to the YUV domain to obtain the YUV image corresponding to the white light image. The YUV image corresponding to the white light image contains the YUV domain information of each pixel, namely the Y value, U value and V value of the pixel.

[0248] S1202, reduce the luminance component (i.e. Y value) of the corresponding first pixel in the YUV image corresponding to the white light image to the first luminance component to obtain the processed YUV image.

[0249] In the processed YUV image, the brightness of the first pixel is less than the brightness of the first pixel in the YUV image corresponding to the white light image.

[0250] It's understandable that the Y value (i.e., luminance information) of each pixel represents the color brightness of that pixel. By reducing the Y value of the first pixel, the contrast between the first pixel and other pixels can be increased, making the first pixel corresponding to the bleed point appear redder and more vibrant.

[0251] For example, the first luminance component can be preset, that is, when processing the YUV image corresponding to the white light image, the S value of each first pixel is reduced to the preset first luminance component to reduce the color luminance of the first pixel.

[0252] For example, the first luminance component can be determined based on the bleed probability of the target pixel (bleed pixel), and the Y value of the corresponding first pixel in the YUV image corresponding to the white light image can be reduced to the first luminance component to obtain the processed YUV image.

[0253] Specifically, the value of the first luminance component is negatively correlated with the bleed probability of the target pixel (i.e., the bleed probability of the first pixel). When the bleed probability of the first pixel is higher, the first luminance component is lower, meaning the decrease in the Y value of the first pixel is greater. Conversely, when the bleed probability of the first pixel is lower, the first luminance component is higher, meaning the decrease in the Y value of the first pixel is smaller.

[0254] In other words, when reducing the color brightness of the first pixel in the white light image, the Y value of the first pixel with a higher probability of bleeding is reduced by a larger amount; while the Y value of the first pixel with a lower probability of bleeding is reduced by a smaller amount. This makes the first pixel with a higher probability of bleeding in the second area of ​​interest have a greater contrast with other pixels around it, thereby highlighting the first pixel with a higher probability of bleeding and helping the user to identify the lesion area.

[0255] More specifically, the first luminance component can be determined according to the following formula (9).

[0256] (9)

[0257] in, The first luminance component represents any first pixel (i.e., the first pixel in the i-th row and j-th column of the YUV image corresponding to the white light image); This represents the Y value of the first pixel in the i-th row and j-th column of the YUV image corresponding to the white light image (i.e., the luminance component before image processing). This represents the probability of bleed at the target pixel in the i-th row and j-th column of the REI image (the corresponding first pixel in the white light image).

[0258] Understandably, referring to formula (9), when When, that is, for pixels that are not bleed points, the first luminance component is equal to the actual Y value of that pixel; that is, for pixels that are not bleed points, no color or luminance adjustment is performed. When the first pixel is 100% a bleed point, the first luminance component is 0, and the contrast between the first pixel and the surrounding pixels is stronger.

[0259] S1203 converts the processed YUV image from YUV space to RGB space to obtain the target image.

[0260] After reducing the luminance component (i.e., Y value) of the corresponding first pixel in the YUV image corresponding to the white light image to the first luminance component, the processed YUV image is converted from YUV space to RGB space to obtain the target image.

[0261] In some examples, image enhancement processing is performed on the first pixel in a white light image to obtain the target image. This may include increasing the color saturation of the first pixel and decreasing the brightness of the first pixel.

[0262] Specifically, the implementation method for increasing the color saturation of the first pixel can be found in S1101 to S1103 above, and the implementation method for decreasing the brightness of the first pixel can be found in S1201 to S1203 above, which will not be repeated here.

[0263] In some examples, the bleed probability of a target pixel (i.e., the bleed pixel, or the first pixel) can be determined based on the color deviation information of the target pixel, where the bleed probability of the bleed pixel is negatively correlated with the color deviation information of the target pixel.

[0264] For example, the bleed probability of the first pixel (i.e. the bleed pixel) can be determined according to the following formula (10).

[0265] (10)

[0266] in, This represents the probability of bleed in the pixel located in the i-th row and j-th column of the REI image (i.e., any bleed pixel); This represents the color deviation information of the pixel located in the i-th row and j-th column of the REI image (i.e., any bleed pixel); This indicates the preset color deviation threshold.

[0267] In other embodiments, when performing image enhancement processing on a white light image, image processing can be performed on the second pixel in the white light image to enhance the display of the second region of interest in the white light image, thereby obtaining the target image. Here, the second pixel refers to any pixel in the white light image other than the first pixel in the REI image.

[0268] In some examples, image enhancement processing of the second pixel in a white light image to obtain a target image may include: reducing the color saturation of the second pixel to obtain the target image.

[0269] Specifically, the white light image is converted from RGB space to HSV space to obtain the corresponding HSV image; the saturation value (i.e., S value) of the corresponding second pixel in the HSV image is reduced to the second saturation value to obtain the processed HSV image; the processed HSV image is converted from HSV space to RGB space to obtain the target image.

[0270] For example, the second saturation value can be preset.

[0271] It is understandable that the S value (i.e., saturation value) of each pixel represents the color purity or vividness of that pixel. By reducing the S value of the second pixel other than the bleed pixel, the saturation of the second pixel is reduced, making the saturation difference between the bleed pixel and the surrounding second pixels more obvious, thus making the bleed pixel stand out from the surrounding tissue.

[0272] In some examples, image enhancement processing of the second pixel in a white light image to obtain a target image may include: increasing the color brightness of the second pixel to obtain the target image.

[0273] Specifically, the white light image is converted from RGB space to YUV space to obtain the YUV image corresponding to the white light image; the luminance component (i.e., Y value) of the corresponding second pixel in the YUV image corresponding to the white light image is increased to the second luminance component to obtain the processed YUV image; the processed YUV image is converted from YUV space to RGB space to obtain the target image.

[0274] For example, the second luminance component can be preset.

[0275] It is understandable that the Y value (i.e., the luminance component) of each pixel represents the color luminance of that pixel. By increasing the Y value of the second pixel (excluding the bleed pixel), the color luminance of the second pixel is increased, making the contrast between the bleed pixel and the surrounding second pixels more obvious, thus making the bleed pixel stand out from the surrounding tissue.

[0276] In some examples, image enhancement processing of the second pixel in a white light image to obtain a target image may include: reducing the color saturation of the second pixel and increasing the brightness of the second pixel to obtain the target image.

[0277] In some examples, the location of the second region of interest in the white light image is determined based on the location of the first region of interest in the REI image, and then the second region of interest is marked with a rectangle based on the location of the second region of interest in the white light image.

[0278] In other examples, for each bleed pixel, multiple neighboring pixels within a preset neighborhood of the bleed pixel are determined. If the bleed probability of the bleed pixel is greater than or equal to a preset bleed probability threshold (i.e., P...), then... t If a pixel is identified as a bleed point, and among its neighboring pixels, there exists a neighboring pixel with a bleed probability less than a preset bleed probability threshold, then the bleed point pixel is determined as a contour pixel. Based on the position of the contour pixel in the REI image, image enhancement processing is performed on the corresponding pixel to be processed in the white light image to mark the second region of interest in the white light image.

[0279] Specifically, the color value of the pixel to be processed is set to the preset outline appearance.

[0280] S905, display the target image.

[0281] In this embodiment, under white light and REI enhancement modes, after acquiring the REI image, the bleeding point pixels are identified based on the REI image, and the first region of interest where the bleeding point is located is identified. Based on the information of the bleeding point pixels in the first region of interest, the white light image is enhanced to enhance the display of the second region of interest where the bleeding point is located in the white light image. This can improve the red saturation of the bleeding point in the second region of interest in the white light image and increase the brightness difference between the bleeding point and other surrounding tissues, making the bleeding point stand out in the white light image and helping users to distinguish the lesion area.

[0282] Method Example 3

[0283] The following section uses the third fusion mode (i.e., white light, NEI and REI enhancement mode) of the endoscope system as an example to elaborate on this image processing method.

[0284] In the third fusion mode, the endoscope's light source module outputs a first special light, a second special light, and a white light simultaneously. When the observed object is illuminated with the first special light, the acquired special light image is a NEI image; when the observed object is illuminated with the second special light, the acquired special light image is a REI image. The target tissues to be identified include blood vessels and bleeding points, and the first region of interest to be identified includes areas where the blood vessel density meets the criteria and areas where bleeding points are located.

[0285] Figure 13 This is a schematic flowchart illustrating another image processing method provided in an embodiment of this application. Figure 13 As shown, the image processing method may include the following steps:

[0286] S1301, acquire the NEI image (i.e., special light image).

[0287] S1302, Based on the pixel information of each pixel in the NEI image, identify the region where the blood vessel density meets the condition.

[0288] Specifically, the specific implementation methods of S1301 and S1302 can be found in the specific implementation methods of S601 and S602 in the above method embodiment 1, which will not be repeated here.

[0289] S1303, acquire REI image (i.e., special light image).

[0290] S1304, based on the pixel information of each pixel in the REI image, identify the region where the bleeding point is located.

[0291] Specifically, the specific implementation methods of S1303 and S1304 can be found in the specific implementation methods of S901 and S902 in the above method embodiment 2, which will not be repeated here.

[0292] S1305, acquire white light image.

[0293] S1306, perform image enhancement processing on the white light image to enhance the display of the second region of interest in the white light image, and / or, mark the second region of interest in the white light image to obtain a target image, wherein the second region of interest includes the region where the blood vessel density meets the condition and the region where the bleeding point is located.

[0294] The specific implementation method for image enhancement processing of regions in the white light image where the blood vessel density meets the conditions can be found in the specific implementation method of S604 in the above-described method embodiment 1, and will not be repeated here. The specific implementation method for image enhancement processing of regions where hemorrhage points are located in the white light image can be found in the specific implementation method of S904 in the above-described method embodiment 2, and will not be repeated here.

[0295] S1307, Display the target image.

[0296] In this embodiment, under white light, NEI, and REI enhancement modes, blood vessels and bleeding points can be highlighted in white light images, which helps users identify lesion areas.

[0297] This application also provides an image processing apparatus. For example... Figure 14 As shown, the image processing device 1400 includes an image acquisition module 1401, a recognition module 1402, an image processing module 1403, and a display module 1404. The image acquisition module 1401 acquires a special light image, which is obtained by photographing an object illuminated by special light. The recognition module 1402 identifies a first region of interest based on the pixel information of each pixel in the special light image. The image acquisition module 1401 also acquires a white light image, which is obtained by photographing an object illuminated by white light. The image processing module 1403 performs image enhancement processing on the white light image to enhance the display of a second region of interest in the white light image, and / or marks the second region of interest in the white light image to obtain a target image, wherein the second region of interest corresponds to the first region of interest. The display module 1404 displays the target image.

[0298] In some embodiments, the identification module 1402 is specifically used to: determine the feature information of each pixel based on the pixel information of each pixel in the special light image, wherein the feature information is used to determine whether the tissue corresponding to the pixel belongs to the target tissue; identify the target pixel corresponding to the target tissue based on the feature information of each pixel; and determine the first region of interest based on the target pixel.

[0299] In some embodiments, the special light is a first special light, the feature information is a high-frequency brightness component, the target tissue is a blood vessel, and both the first and second regions of interest are regions where the blood vessel density meets the condition; or, the special light is a second special light, the feature information is color deviation information, the target tissue is a bleeding point, and both the first and second regions of interest are regions where the bleeding point is located.

[0300] In some embodiments, the special light is a first special light, the feature information is a high-frequency brightness component, the target tissue is a blood vessel, and the first region of interest is a region where the blood vessel density meets the condition; the recognition module 1402 is specifically used to: determine the brightness information of each pixel based on the pixel information of each pixel in the special light image, and remove the low-frequency brightness component from the brightness information of each pixel to obtain the high-frequency brightness component of each pixel; determine the pixels in the special light image whose high-frequency brightness component meets the first preset condition as target pixels, wherein the first preset condition is that the absolute value of the high-frequency brightness component of the pixel is greater than a preset brightness threshold; determine the proportion of the number of target pixels in each image region in the special light image; and determine the image region whose proportion of the number of target pixels is greater than or equal to a preset proportion threshold as the first region of interest.

[0301] In some embodiments, the recognition module 1402 is specifically used to: convert a special light image into a grayscale image, the grayscale image being used to characterize the brightness information of each pixel; perform low-pass filtering on the grayscale image to obtain the low-frequency brightness component of each pixel; and remove the low-frequency brightness component from the brightness information of each pixel to obtain the high-frequency brightness component of each pixel.

[0302] In some embodiments, the special light is a second special light, the feature information is color deviation information, the target tissue is a bleeding point, and the first region of interest is the region where the bleeding point is located; the recognition module 1402 is specifically used to: determine the YUV domain information of each pixel based on the pixel information of each pixel in the special light image; determine the color deviation information of each pixel based on the YUV domain information of each pixel, wherein the color deviation information is used to indicate the deviation of the YUV domain information of the pixel relative to the preset blood YUV domain information; and determine the pixels in the special light image whose color deviation information is less than or equal to the preset color deviation threshold as target pixels.

[0303] In some embodiments, the image processing module 1403 is specifically configured to: determine a first pixel and a second pixel in a white light image, wherein the first pixel is a pixel in the white light image corresponding to the position of the target pixel, the first pixel belongs to a second region of interest, and the second pixel is any other pixel in the white light image besides the first pixel, the second pixel does not belong to the second region of interest; perform image enhancement processing on the first pixel, and / or perform image processing on the second pixel, so as to enhance the display of the second region of interest in the white light image, thereby obtaining a target image.

[0304] In some embodiments, the image processing module 1403 is specifically used to: perform image processing on the target image parameters of the first pixel; or, perform image processing on the target image parameters of the first pixel according to the target probability corresponding to the target pixel; wherein the target probability is used to characterize the confidence level of determining that the tissue corresponding to the pixel belongs to the target tissue.

[0305] In some embodiments, the target tissue is a blood vessel, the target probability is a blood vessel probability, and the blood vessel probability is proportional to the absolute value of the high-frequency brightness component of the pixel; the target image parameters include component values ​​of the blue channel and the green channel; the image processing module 1403 is specifically used to: reduce the component value of the blue channel of the first pixel to a first blue component value, and reduce the component value of the green channel of the first pixel to a first green component value; wherein, the first blue component value is a first preset value, or the first blue component value is determined according to the blood vessel probability of the first pixel; the first green component value is a second preset value, or the first green component value is determined according to the blood vessel probability of the first pixel.

[0306] In some embodiments, the image processing module 1403 is specifically configured to: reduce the component value of the blue channel of the first pixel to a first blue component value in an exponential or linear manner, and reduce the component value of the green channel of the first pixel to a first green component value.

[0307] In some embodiments, the image processing module 1403 is specifically configured to: increase the component value of the blue channel of the second pixel to a second blue component value, and increase the component value of the green channel of the second pixel to a second green component value.

[0308] In some embodiments, the target tissue is a bleed point, the target probability is a bleed point probability, and the bleed point probability is determined based on the color deviation information of the target pixel and a preset color deviation threshold; the target image parameters include saturation and / or brightness; the image processing module 1403 is specifically used to: increase the saturation of the first pixel in the white light image based on the bleed point probability of the target pixel to obtain the target image; and / or, decrease the brightness of the first pixel based on the bleed point probability of the target pixel to obtain the target image.

[0309] In some embodiments, the image processing module 1403 is specifically used to: reduce the saturation of the second pixel; and / or increase the brightness of the second pixel.

[0310] In some embodiments, the image processing module 1403 is specifically configured to: convert a white light image from RGB space to HSV space to obtain an HSV image corresponding to the white light image; based on the bleed probability of the target pixel, increase the saturation value of the corresponding first pixel in the HSV image corresponding to the white light image to a first saturation value to obtain a processed HSV image, wherein the saturation of the first pixel in the processed HSV image is greater than the saturation of the first pixel in the HSV image; and convert the processed HSV image from HSV space to RGB space to obtain a target image.

[0311] In some embodiments, the image processing module 1403 is specifically configured to: convert a white light image from RGB space to YUV space to obtain a YUV image corresponding to the white light image; based on the bleed probability of the target pixel, reduce the luminance component of the corresponding first pixel in the YUV image corresponding to the white light image to the first luminance component to obtain a processed YUV image, wherein the luminance of the first pixel in the processed YUV image is less than the luminance of the first pixel in the YUV image; and convert the processed YUV image from YUV space to RGB space to obtain a target image.

[0312] In some embodiments, the image processing apparatus 1400 further includes a probability determination module, configured to: after identifying a target pixel, determine the current reference probability of the target pixel based on the feature information of the target pixel; acquire a historical special light image, which includes a reference pixel corresponding to the target pixel; determine the historical reference probability of the reference pixel based on the historical special light image; and perform weighted fusion processing on the current reference probability and the historical reference probability to obtain the target probability of the target pixel.

[0313] In some embodiments, the image processing module 1403 is specifically configured to: for each target pixel, determine a plurality of neighboring pixels located in a preset neighborhood of the target pixel; if the target probability of the target pixel is greater than or equal to a preset probability threshold, and there are neighboring pixels in the plurality of neighboring pixels located in the neighborhood of the target pixel whose target probability is less than the preset probability threshold, then determine the target pixel as a contour pixel; based on the position of the contour pixel in the special light image, perform image enhancement processing on the corresponding pixel to be processed in the white light image to mark the second region of interest in the white light image.

[0314] The image processing apparatus provided in this embodiment can execute the image processing method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0315] Figure 15 This is a schematic diagram of the endoscope system provided in this application. Figure 15 As shown, the endoscope system 150 provided in this embodiment includes at least one processor 1501 and a memory 1502. Optionally, the endoscope system 150 further includes a communication component 1503. The processor 1501, memory 1502, and communication component 1503 are connected via a bus.

[0316] In a specific implementation, at least one processor 1501 executes computer execution instructions stored in memory 1502, causing at least one processor 1501 to perform the above-described method.

[0317] The specific implementation process of processor 1501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0318] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0319] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0320] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0321] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described image processing method.

[0322] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described image processing method.

[0323] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0324] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0325] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0326] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0327] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0328] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0329] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0330] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. An image processing method, characterized in that, A processing device for use in an endoscope system, the endoscope system further including an endoscope, wherein the light source module of the endoscope is configured to output white light and special light in a time-division manner, the spectrum of the white light being different from the spectrum of the special light, the method comprising: Acquire a special light image, which is obtained by photographing the object being observed illuminated by the special light; Based on the pixel information of each pixel in the special light image, the first region of interest is identified; Acquire a white light image, which is obtained by photographing the object being observed illuminated by the white light; Image enhancement processing is performed on the white light image to enhance the display of the second region of interest in the white light image, and / or the second region of interest in the white light image is marked to obtain a target image, wherein the second region of interest corresponds to the first region of interest; Display the target image.

2. The method according to claim 1, characterized in that, The step of identifying the first region of interest based on the pixel information of each pixel in the special light image includes: Based on the pixel information of each pixel in the special light image, the feature information of each pixel is determined, and the feature information is used to determine whether the tissue corresponding to the pixel belongs to the target tissue. Based on the feature information of each pixel, identify the target pixel corresponding to the target tissue; The first region of interest is determined based on the target pixel.

3. The method according to claim 2, characterized in that, The special light is a first special light, the feature information is a high-frequency brightness component, the target tissue is a blood vessel, and the first region of interest and the second region of interest are both regions where the blood vessel density meets the conditions. or, The special light is a second special light, the feature information is color deviation information, the target tissue is a bleeding point, and the first region of interest and the second region of interest are both regions where the bleeding point is located.

4. The method according to claim 2, characterized in that, The special light is a first special light, the feature information is a high-frequency brightness component, the target tissue is a blood vessel, and the first region of interest is a region where the blood vessel density meets the condition; determining the feature information of each pixel based on the pixel information of each pixel in the special light image includes: Based on the pixel information of each pixel in the special light image, the brightness information of each pixel is determined, and the low-frequency brightness component in the brightness information of each pixel is removed to obtain the high-frequency brightness component of each pixel. The step of identifying the target pixel corresponding to the target tissue based on the feature information of each pixel includes: The pixel in the special light image whose high-frequency brightness component satisfies the first preset condition is determined as the target pixel, wherein the first preset condition is that the absolute value of the high-frequency brightness component of the pixel is greater than a preset brightness threshold. Determining the first region of interest based on the target pixel includes: Determine the percentage of the number of target pixels in each image region of the special light image; The image regions whose quantity percentage is greater than or equal to a preset percentage threshold are determined as the first region of interest.

5. The method according to claim 4, characterized in that, The step of determining the brightness information of each pixel based on the pixel information of each pixel in the special light image, and removing the low-frequency brightness components from the brightness information of each pixel to obtain the high-frequency brightness components of each pixel, includes: The special light image is converted into a grayscale image, and the grayscale image is used to represent the brightness information of each pixel. The grayscale image is subjected to low-pass filtering to obtain the low-frequency brightness components of each pixel; Remove the low-frequency luminance components from the luminance information of each pixel to obtain the high-frequency luminance components of each pixel.

6. The method according to claim 2, characterized in that, The special light is a second special light, the feature information is color deviation information, the target tissue is a bleed point, and the first region of interest is the region where the bleed point is located; determining the feature information of each pixel based on the pixel information of each pixel in the special light image includes: Based on the pixel information of each pixel in the special light image, determine the YUV domain information of each pixel; Based on the YUV domain information of each pixel, the color deviation information of each pixel is determined, and the color deviation information is used to indicate the deviation of the YUV domain information of the pixel relative to the preset blood YUV domain information. The step of identifying the target pixel corresponding to the target tissue based on the feature information of each pixel includes: The pixels in the special light image whose color deviation information is less than or equal to a preset color deviation threshold are identified as the target pixels.

7. The method according to claim 2, characterized in that, The step of performing image enhancement processing on the white light image to enhance the display of the second region of interest in the white light image and obtain the target image includes: Determine a first pixel and a second pixel in the white light image, wherein the first pixel is the pixel in the white light image that corresponds to the position of the target pixel, the first pixel belongs to the second region of interest, and the second pixel is any other pixel in the white light image other than the first pixel, the second pixel does not belong to the second region of interest; The first pixel is subjected to image enhancement processing, and / or the second pixel is subjected to image processing, so as to enhance the display of the second region of interest in the white light image, thereby obtaining the target image.

8. The method according to claim 7, characterized in that, The image enhancement processing of the first pixel includes: Image processing is performed on the target image parameters of the first pixel. or, Based on the target probability corresponding to the target pixel, image processing is performed on the target image parameters of the first pixel; wherein, the target probability is used to characterize the confidence level of determining that the tissue corresponding to the pixel belongs to the target tissue.

9. The method according to claim 8, characterized in that, The target tissue is a blood vessel, the target probability is a blood vessel probability, and the blood vessel probability is proportional to the absolute value of the high-frequency brightness component of the pixel; the target image parameters include the component values ​​of the blue channel and the green channel; the image enhancement processing of the first pixel includes: Reduce the component value of the blue channel of the first pixel to the first blue component value, and reduce the component value of the green channel of the first pixel to the first green component value; Wherein, the first blue component value is a first preset value, or the first blue component value is determined based on the blood vessel probability of the first pixel; the first green component value is a second preset value, or the first green component value is determined based on the blood vessel probability of the first pixel.

10. The method according to claim 9, characterized in that, The step of reducing the blue channel component value of the first pixel to a first blue component value and reducing the green channel component value of the first pixel to a first green component value includes: The blue channel component value of the first pixel is reduced to the first blue component value in an exponential or linear manner, and the green channel component value of the first pixel is reduced to the first green component value.

11. The method according to claim 9, characterized in that, The image processing of the second pixel includes: Increase the blue channel component value of the second pixel to the second blue component value, and increase the green channel component value of the second pixel to the second green component value.

12. The method according to claim 8, characterized in that, The target tissue is a bleed point, the target probability is a bleed point probability, and the bleed point probability is determined based on the color deviation information of the target pixel and a preset color deviation threshold; The target image parameters include saturation and / or brightness; the image enhancement processing of the first pixel includes: Based on the bleed probability of the target pixel, the saturation of the first pixel in the white light image is increased to obtain the target image; and / or, based on the bleed probability of the target pixel, the brightness of the first pixel is decreased to obtain the target image; The image processing of the second pixel includes: Decrease the saturation of the second pixel and / or increase the brightness of the second pixel.

13. The method according to claim 12, characterized in that, The step of increasing the saturation of the first pixel based on the bleed probability of the target pixel to obtain the target image includes: The white light image is converted from RGB space to HSV space to obtain the HSV image corresponding to the white light image; Based on the bleed probability of the target pixel, the saturation value of the corresponding first pixel in the HSV image corresponding to the white light image is increased to a first saturation value to obtain a processed HSV image. The saturation of the first pixel in the processed HSV image is greater than the saturation of the first pixel in the HSV image. The processed HSV image is converted from HSV space to RGB space to obtain the target image; The step of reducing the brightness of the first pixel based on the bleed probability of the target pixel to obtain the target image includes: The white light image is converted from RGB space to YUV space to obtain the YUV image corresponding to the white light image; Based on the bleed probability of the target pixel, the luminance component of the corresponding first pixel in the YUV image corresponding to the white light image is reduced to the first luminance component to obtain a processed YUV image. The luminance of the first pixel in the processed YUV image is less than the luminance of the first pixel in the YUV image. The processed YUV image is converted from YUV space to RGB space to obtain the target image.

14. The method according to claim 8, characterized in that, The method further includes: After identifying the target pixel, the current reference probability of the target pixel is determined based on the feature information of the target pixel. Acquire historical special light images, wherein the historical special light images include reference pixels corresponding to the target pixel; Based on the historical special light image, determine the historical reference probability of the reference pixel; The current reference probability and the historical reference probability are weighted and fused to obtain the target probability of the target pixel.

15. The method according to claim 2, characterized in that, The step of marking the second region of interest in the white light image includes: For each target pixel, determine a plurality of adjacent pixels located within a preset neighborhood of the target pixel; If the target probability of the target pixel is greater than or equal to a preset probability threshold, and among the multiple neighboring pixels in the neighborhood of the target pixel, there is a neighboring pixel with a target probability less than the preset probability threshold, then the target pixel is determined as a contour pixel. Based on the position of the contour pixels in the special light image, image enhancement processing is performed on the corresponding pixels to be processed in the white light image to mark the second region of interest in the white light image.

16. An endoscope system, characterized in that, include: Memory, processor, and endoscope; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the image processing method as described in any one of claims 1-15; The endoscope includes a light source module configured to output white light and special light in a time-division manner, wherein the spectrum of the white light is different from the spectrum of the special light.

Images