Endoscope image enhancement method, device and system

Abstract

The invention provides an endoscope image enhancement method, device and system, and relates to the field of endoscope image processing.The method comprises the steps that after an image collected by an endoscope is obtained, the image is displayed through a display screen, and the display screen comprises a first operation area used for adjusting image enhancement parameters; in response to a control operation for the first operation area, acquiring an enhancement parameter of the image; adjusting the current saturation of the image according to a saturation segmentation threshold value of the image and the enhancement parameter so as to enhance and display blood or vascular tissues in the image according to the adjusted saturation; wherein the saturation segmentation threshold value is determined according to saturation statistical information of at least multiple pixel points of a previous frame image collected by an endoscope. Through the method, the enhancement effect of blood or vascular tissue in the image is improved, the preference and requirements of different users for image observation can be met, and the diagnosis efficiency and accuracy are further improved.

Description

Technical Field

[0001] The present application relates to the technical field of endoscopic image processing, and in particular to an endoscopic image enhancement method, device and system. Background Art

[0002] An endoscope is an instrument used in medical examinations and surgery that is designed to be passed into the body through natural openings or small incisions to allow direct visualization of internal organs and structures.

[0003] An endoscope usually consists of a long, thin tube with a light source and a camera at one end and a monitor at the other end, through which doctors can observe real-time images of the patient's body. In imaging diagnosis based on endoscopic images, clinicians often need to evaluate and judge the patient's condition by observing the shape and color of the blood vessels in the image.

[0004] However, in the actual image observation process, doctors face a major challenge, that is, tissues other than blood vessels and blood sometimes appear in colors similar to blood and blood vessels. This phenomenon of similar colors may cause visual fatigue to doctors during long-term observation, thereby affecting the accuracy and efficiency of diagnosis. Summary of the invention

[0005] The present application provides an endoscopic image enhancement method, device and system to solve at least one of the above problems.

[0006] According to a first aspect of the present application, there is provided an endoscopic image enhancement method, comprising:

[0007] After acquiring an image collected by the endoscope, displaying the image through a display screen, the display screen includes a first operation area for adjusting image enhancement parameters;

[0008] In response to a control operation on the first operation area, acquiring enhancement parameters of the image;

[0009] According to the saturation segmentation threshold of the image and the enhancement parameter, the current saturation of the image is adjusted to enhance the display of blood or vascular tissue in the image according to the adjusted saturation;

[0010] The saturation segmentation threshold is determined based on saturation statistical information of at least a plurality of pixel points of a previous frame of image acquired by the endoscope.

[0011] In an optional implementation manner, the method for determining the saturation segmentation threshold includes:

[0012] According to the saturation statistical information about each pixel point in the previous frame image, the saturation of each pixel point is sorted to obtain a saturation sorting result; wherein the pixel point is a sampled pixel point after downsampling the previous frame image;

[0013] According to the saturation sorting result, the saturation of the corresponding pixel point is selected from each pixel point as the saturation segmentation threshold of the image.

[0014] In an optional implementation, selecting the saturation of a corresponding pixel from each pixel according to the saturation sorting result as the saturation segmentation threshold of the image includes:

[0015] According to the saturation sorting result and a preset threshold correction coefficient, a sorting index for querying the saturation of a target pixel among the pixels is determined; wherein the threshold correction coefficient is used to indicate data that reaches a preset percentage in the saturation sorting result as a saturation segmentation threshold;

[0016] According to the sorting index, the saturation of the target pixel is queried from the saturation sorting result, and the saturation of the pixel is determined as the saturation segmentation threshold of the image.

[0017] In an optional embodiment, the enhancement parameter includes blood enhancement intensity;

[0018] The adjusting the current saturation of the image according to the saturation segmentation threshold of the image and the enhancement parameter comprises:

[0019] For each pixel in the image, correcting the current saturation of the pixel according to the saturation segmentation threshold and the blood enhancement strength to obtain a first corrected saturation of each pixel;

[0020] The adjusted saturation of the image is obtained according to the first corrected saturation of each pixel.

[0021] In an optional embodiment, the enhancement parameter further includes a blood enhancement intensity threshold; and the method further includes:

[0022] For each pixel in the image, determining a blood enhancement amplitude according to a difference between the blood enhancement intensity and the blood enhancement intensity threshold;

[0023] According to the blood enhancement amplitude, the current saturation of the pixel point is corrected to obtain a second corrected saturation;

[0024] The step of obtaining the adjusted saturation of the image according to the first corrected saturation of each pixel point includes:

[0025] The adjusted saturation of the image is obtained according to the sum of the first corrected saturation and the second corrected saturation of each pixel.

[0026] In an optional implementation, the enhancement parameter includes a grayscale protection threshold;

[0027] The correcting the current saturation of the pixel point according to the saturation segmentation threshold and the blood enhancement strength includes:

[0028] When the grayscale level of the pixel reaches the grayscale protection threshold, the current saturation of the pixel is corrected according to the saturation segmentation threshold and the blood enhancement intensity.

[0029] In an optional implementation, the method further includes:

[0030] Determine a low grayscale protection coefficient according to a grayscale protection threshold and a maximum color component of the image with respect to red, green and blue channels;

[0031] The correcting the current saturation of the pixel point according to the saturation segmentation threshold and the blood enhancement strength includes:

[0032] When the grayscale level of the pixel reaches the grayscale protection threshold, the current saturation of the pixel is corrected according to the saturation segmentation threshold and the blood enhancement strength, and the correction result is not lower than the low grayscale protection coefficient.

[0033] In an optional implementation, the display screen includes a second operation area for setting a division multiplication table; and the method further includes:

[0034] In response to a control operation for the second operation area, acquiring the division multiplication table, wherein the division multiplication table includes a multiplication coefficient corresponding to each divisor used for calculation;

[0035] The adjusting the current saturation of the image according to the saturation segmentation threshold of the image and the enhancement parameter comprises:

[0036] According to the division multiplication table, the saturation segmentation threshold of the image and the divisor involved in the enhancement parameter calculation process are converted into corresponding multiplication coefficients to adjust the current saturation of the image.

[0037] In an optional implementation, the displaying of blood or vascular tissue in the image according to the adjusted saturation enhancement includes:

[0038] According to the adjusted saturation, the corresponding color components in the red, green and blue channels of the image are adjusted to obtain a color component adjustment result;

[0039] The result is adjusted according to the color component to enhance the display of blood or blood vessel tissue in the image.

[0040] In an optional implementation, adjusting corresponding color components in red, green and blue channels in the image according to the adjusted saturation includes:

[0041] Adjusting the minimum color component in the red, green and blue channels of the image according to the adjusted saturation and the maximum color component in the red, green and blue channels of the image;

[0042] The intermediate color components in the red, green and blue channels of the image are adjusted according to the maximum color component and the adjusted minimum color component by using a preset interpolation algorithm.

[0043] According to a second aspect of the present application, an endoscope image enhancement device is provided, comprising:

[0044] A display module, which is configured to display the image acquired by the endoscope through a display screen after acquiring the image, wherein the display screen includes a first operation area for adjusting image enhancement parameters;

[0045] a parameter acquisition module, configured to acquire enhancement parameters of the image in response to a control operation on the first operation area;

[0046] an enhancement module, configured to adjust the current saturation of the image according to the saturation segmentation threshold of the image and the enhancement parameter, so as to enhance and display the blood or vascular tissue in the image according to the adjusted saturation;

[0047] The saturation segmentation threshold is determined based on saturation statistical information of at least a plurality of pixel points of a previous frame of image acquired by the endoscope.

[0048] According to a third aspect of the present application, there is provided an image processing device, comprising: a memory and a processor;

[0049] The memory stores computer-executable instructions;

[0050] The processor executes the computer-executable instructions stored in the memory, so that the image processing device performs the endoscopic image enhancement method provided in any one of the first aspects above.

[0051] According to a fourth aspect of the present application, a computer-readable storage medium is provided, which includes computer instructions. When the computer instructions are executed on a computer, the computer executes the endoscopic image enhancement method provided in any one of the first aspects above.

[0052] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when the at least one processor executes the computer program, the endoscopic image enhancement method provided in any one of the above first aspects can be implemented.

[0053] According to a sixth aspect of the present application, an image enhancement system is provided, including an image processing device provided in the above third aspect, and an endoscope electrically connected to the image processing device. The endoscope is used to collect images of organs in a patient's body and transmit the collected images to the image processing device for image enhancement.

[0054] For the endoscopic image enhancement method, device and system provided by the present application, after obtaining the image collected by the endoscope, the image is displayed through a display screen. The display screen includes a first operation area for adjusting image enhancement parameters. In response to a control operation on the first operation area, the enhancement parameters of the image are obtained, and according to the saturation segmentation threshold and enhancement parameters of the image, the current saturation of the image is adjusted to enhance the blood or blood vessel tissue in the displayed image according to the adjusted saturation. The saturation segmentation threshold is determined based on the saturation statistical information of at least multiple pixel points of the previous frame of the image collected by the endoscope. In this process, instead of the method of using a fixed saturation parameter for image enhancement in the related art, the saturation segmentation threshold is determined by the saturation statistical information of the pixel points in the previous frame of the image in this embodiment. The saturation segmentation threshold can be adaptively adjusted based on the actual situation, so that the enhanced blood or blood vessel tissue image can better reflect the actual needs under specific endoscopes or specific patient conditions, thereby improving the image enhancement effect; moreover, through the interaction between the display screen and the user, the user can realize personalized setting of enhancement parameters (such as blood enhancement intensity, gray protection threshold, etc.) in the corresponding operation area. Combining the enhancement parameters with the saturation segmentation threshold to enhance the image can effectively meet the preferences and requirements of different users for image observation, further facilitating the medical staff to observe the image, and thus improving the diagnosis efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The drawings herein are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present application and used together with the specification to explain the principles of the present application.

[0056] Figure 1 It is a schematic diagram of a possible scenario provided by an embodiment of the present application;

[0057] Figure 2 It is a flowchart of an endoscopic image enhancement method provided by an embodiment of the present application;

[0058] Figure 3 A schematic diagram of a flow chart of another endoscopic image enhancement method provided in an embodiment of the present application;

[0059] Figure 4 This is a schematic diagram of the interface of the first operating area and the second operating area in the embodiment of the present application;

[0060] Figure 5 A schematic diagram of a flow chart of another endoscopic image enhancement method provided in an embodiment of the present application;

[0061] Figure 6a This is one of the sample images collected by the endoscope;

[0062] Figure 6b In order to adopt the technical solution of the embodiment of the present application Figure 6a An example of an image after image enhancement is performed on the image in the figure;

[0063] Figure 7a This is one of the sample images collected by the endoscope;

[0064] Figure 7b In order to adopt the technical solution of the embodiment of the present application Figure 7a An example of an image after image enhancement is performed on the image in the figure;

[0065] Figure 8 A schematic diagram of the structure of an endoscope image enhancement device provided in an embodiment of the present application;

[0066] Fig. 9 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0067] Fig.10 A schematic diagram of the structure of an image enhancement system provided in an embodiment of the present application.

[0068] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0069] In the course of their work, clinicians often need to observe the shape and color of blood vessels in human organs in images collected by endoscopes, so as to make specific judgments on the patient's condition. For example, when collecting images of the digestive tract through endoscopes, inflammation usually causes vascular dilation and congestion, which is manifested as the color of the blood vessels becoming brighter red or purple, and the vascular morphology may become blurred. Doctors observe the shape and color of blood vessels to identify inflammatory diseases such as ulcerative colitis. Alternatively, tumor tissues usually cause new blood vessels to form, which may appear as irregular, twisted or abnormally dense vascular networks. Doctors observe irregular vascular shapes and abnormal vascular networks to determine whether cancer or tumors exist. However, in the process of observing images, the colors of tissues other than blood vessels and blood in the image may be similar to those of blood and blood vessels, which will cause visual fatigue to doctors, thereby affecting the accuracy and efficiency of diagnosis. Therefore, image processing is required to facilitate medical staff to observe and diagnose endoscopic images.

[0070] It has been found that the enhancement of blood or vascular tissue in the image can be achieved by correcting the saturation of the image. However, the current method of image enhancement is usually adjusted by using a set saturation parameter (e.g., using a fixed saturation segmentation threshold to correct the saturation of the endoscopic image). On the one hand, the use of a fixed threshold for image enhancement will produce significant differences for different endoscopes or different patients, resulting in the enhancement effect of blood or vascular tissue in some scenarios being difficult to meet the actual user needs; on the other hand, different users have different needs and expectations for images. Since the above method does not support personalized setting and adjustment of enhancement parameters (such as blood enhancement intensity, grayscale protection threshold, etc.), it is difficult to solve the visual differences between users for the same image (e.g., different doctors or technicians have different preferences and requirements for the color and contrast of the image based on their professional background, experience or specific diagnostic tasks. For example, some users may want to see higher contrast images to facilitate the identification of tiny vascular abnormalities, while some users may pay more attention to the overall color balance to assess the health of the tissue). This limitation will cause inconvenience to some users during the observation process and may even affect the accuracy and efficiency of the diagnosis.

[0071] In view of this, an embodiment of the present application provides an endoscopic image enhancement method, device and system. After acquiring an image captured by an endoscope, the image is displayed through a display screen, which includes a first operating area for adjusting image enhancement parameters. In response to a control operation on the first operating area, the enhancement parameters of the image are acquired, and the current saturation of the image is adjusted according to the saturation segmentation threshold and the enhancement parameters of the image, so as to enhance the display of blood or vascular tissue in the image according to the adjusted saturation. The saturation segmentation threshold is determined based on the saturation statistical information of at least multiple pixel points of the previous frame of image captured by the endoscope. In this process, instead of using fixed saturation parameters for image enhancement in related technologies, this embodiment uses the saturation statistical information of the pixels in the previous frame of the image to determine the saturation segmentation threshold. The saturation segmentation threshold can be adaptively adjusted based on the actual situation, so that the enhanced blood or vascular tissue image can better reflect the actual needs under the conditions of a specific endoscope or a specific patient, thereby improving the blood enhancement effect; and, through the interaction between the display screen and the user, the user can realize personalized setting of enhancement parameters (such as blood enhancement intensity, grayscale protection threshold, etc.) in the corresponding operation area, and enhance the image by combining the enhancement parameters with the saturation segmentation threshold, which can effectively meet the preferences and requirements of different users for image observation, further facilitate medical staff to observe the image, and thus improve diagnostic efficiency and accuracy.

[0072] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be described in more detail below in conjunction with the drawings in the embodiments of the present application. In the drawings, the same or similar reference numerals throughout represent the same or similar parts or parts with the same or similar functions. The described embodiments are part of the embodiments of the present application, not all of the embodiments. The embodiments described below with reference to the drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limitations on the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0073] Figure 1 A possible scenario diagram provided for an embodiment of the present application is as follows: Figure 1As shown in the figure, it includes an image processing terminal 110 and an endoscope 120. The image processing terminal 110 and the endoscope 120 are electrically connected. Medical staff can use the endoscope 120 to collect images of tissues inside the patient's body, such as collecting images inside the digestive tract. The images collected by the endoscope 120 are transmitted to the display screen of the image processing terminal 110 for display. The display screen includes a first operation area 111. Medical staff can adjust or set the image enhancement parameters they need in the first operation area 111. The image processing terminal 110 responds to the control operation in the first operation area 111 (for example, the user can set the corresponding enhancement parameters in the first operation area 111 through an input operation or a selection operation), obtains the enhancement parameters, and enhances the saturation of the image by combining the enhancement parameters and the saturation segmentation threshold. Optionally, the image processing terminal 110 may include, but is not limited to, a computer, a smart phone, a tablet computer, an e-book reader, a Moving Picture experts group audio layerIII (MP3) player, a Moving Picture experts group audio layer IV (MP4) player, a portable computer, a vehicle-mounted computer, a wearable device, a desktop computer, a set-top box, a smart TV, and so on.

[0074] Optionally, when the medical staff performs a control operation on the image processing terminal 110, it can be sent from the user terminal to the image processing terminal 110, or input through a control (such as a touch screen) configured on the image processing terminal 110. This application does not make specific limitations on the above specific implementation methods.

[0075] It should be noted that the electrical connection between the image processing terminal 110 and the endoscope 120 can be a wired connection, connecting devices through physical cables such as data cables and network cables, such as Universal Serial Bus (USB), High-Definition Multimedia Interface (HDMI), Ethernet and other interfaces to connect devices for data transmission. It can also be a wireless connection, communicating through radio wave signals, such as wireless fidelity (WiFi), Bluetooth, infrared and other wireless technologies to achieve communication between devices. Or it can be a remote connection, connecting devices through network technologies such as the Internet to achieve remote control and data transmission, such as cloud services, remote desktops, etc. It can also be a near field communication (NFC) connection, which realizes the connection between devices through near field communication technology, which is usually used for applications such as file transfer; low-power Bluetooth connection, which is used for short-distance communication connection between devices, such as the connection of smart bracelets, smart watches and other devices; wireless communication technology connection, which is used for low-speed, short-distance communication connection between devices, and is suitable for low-power Internet of Things devices. This application does not specifically limit the specific method of communication connection between physical devices.

[0076] The above is a brief description of the scenario diagram of this application. Figure 1 The image processing terminal 110 in the figure is taken as an example to explain in detail the endoscopic image enhancement method, device and system provided in the embodiments of the present application.

[0077] First of all, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0078] Figure 2 A schematic diagram of a process flow of an endoscopic image enhancement method provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the method may include steps S201-S203:

[0079] Step S201: After acquiring an image collected by an endoscope, the image is displayed on a display screen, and the display screen includes a first operation area for adjusting image enhancement parameters.

[0080] It is understood that the camera or image sensor of the endoscope can capture real-time images of tissues in the patient's body during the inspection process, and these images can be transmitted to the image sensor through the optical system of the endoscope. Exemplarily, the collected image signals can be transmitted to the image processing terminal via a cable or wirelessly for display. For example, for a wired endoscope, the image signal can be transmitted via a plug-in cable, and for a wireless endoscope, the signal may be transmitted via radio waves.

[0081] Different from the related art in which only a display screen is used to display the image collected by the endoscope, the display screen in this embodiment includes a first operating area for adjusting image enhancement parameters, and the image enhancement parameters may include blood enhancement intensity, grayscale protection threshold, and the like.

[0082] Exemplarily, a special operation area can be designed in the user interface (UI) of the display screen for adjusting image enhancement parameters. This area can be set at the edge or bottom of the display screen so as not to block the main image display. Optionally, the first operation area can include a variety of controls, such as sliders, buttons, drop-down menus or knobs or input boxes, etc. The user can interact with the operation area through a touch screen, mouse, touchpad or other input device, so that the user can flexibly adjust different image enhancement parameters according to actual needs. When the user adjusts the parameters in the first operation area, the image processing terminal can respond to the enhancement parameters and enter the subsequent process to combine the saturation segmentation threshold for image enhancement.

[0083] By designing the first operating area in the display screen, it is possible to effectively help users adjust image enhancement parameters according to specific needs, thereby improving the accuracy and efficiency of diagnosis and treatment.

[0084] Step S202: In response to a control operation on the first operation area, acquiring enhancement parameters of the image.

[0085] In this embodiment, the image processing terminal can continuously monitor the first operation area to detect the user's control operation, such as control operation performed through a touch screen, mouse, touchpad or other input device. When the user's input is detected, the image processing terminal can identify the specific control operation type. For example, operations such as sliding, clicking, rotating, etc. are used to adjust different image enhancement parameters. For example, the user adjusts the blood enhancement intensity or grayscale protection threshold through a slider, and the image processing terminal obtains a new parameter value according to the user's control operation. For example, increase or decrease the value of the blood enhancement intensity, or adjust the grayscale protection threshold, and the latest enhancement parameter can be cached for application to the image processing algorithm in the subsequent process, thereby achieving the purpose of image enhancement.

[0086] It can be understood that in response to the conditions or states on which the executed operations depend, when the dependent conditions or states are met, one or more operations executed can be in real time or have a set delay; unless otherwise specified, there is no restriction on the order of execution of the multiple operations executed.

[0087] Step S203, adjusting the current saturation of the image according to the saturation segmentation threshold of the image and the enhancement parameter; wherein the saturation segmentation threshold is determined based on the saturation statistical information of at least multiple pixel points of the previous frame image collected by the endoscope.

[0088] Exemplarily, the enhancement parameter in this embodiment can be the blood enhancement intensity. It can be understood that the blood enhancement intensity is used to enhance the visibility of blood or vascular tissue in the image. Specifically, by adjusting certain features of the image (such as color, contrast or saturation), the blood or blood vessels are made more prominent and easy to identify in the image, so that the blood or blood vessels can be quickly identified in the complex image background. The saturation segmentation threshold is used to distinguish between areas with different saturation levels in the image. Based on the threshold, the pixels that need to adjust the saturation can be quickly identified to highlight specific image features, such as blood or vascular tissue. In other words, the saturation segmentation threshold represents the area that needs to be corrected for saturation, and the blood enhancement intensity represents the saturation amount that needs to be corrected in the corresponding area. In some embodiments, the enhancement parameter can also be any other parameter that can be customized by the user, such as a texture enhancement parameter (enhancing the texture details in the image to make a specific structure or pattern more obvious), a tone mapping parameter (adjusting the tone mapping of the image to adapt to different display devices or visual effect requirements), and the like.

[0089] Among them, by selecting an appropriate saturation segmentation threshold using the saturation statistical information of at least a plurality of pixels in the previous frame image (for example, all pixels, or some pixels determined by a person skilled in the art in combination with practical applications and empirical values), the distribution characteristics of the saturation in the image can be efficiently determined, and the picture is smoother and more stable. For example, the average value or median can be selected as the threshold value based on the saturation statistical information of the pixels in the previous frame image, or the threshold value can be dynamically adjusted according to a specific algorithm to adapt to different imaging conditions. The specific determination method of the saturation segmentation threshold value is not particularly limited in this embodiment. In some embodiments, in addition to determining the saturation segmentation threshold value using the saturation statistical information of the pixels in the previous frame image, the saturation statistical information of the previous frames of images can also be used to jointly determine the saturation segmentation threshold value, or the saturation statistical information of the pixels in the current frame image can be directly used to determine the saturation segmentation threshold value, and so on.

[0090] Step S204: displaying the blood or blood vessel tissue in the image in an enhanced manner according to the adjusted saturation.

[0091] For example, the adjusted saturation can be used to adjust the corresponding color components in the red, green, and blue (RGB) channels in the image, and the blood or vascular tissue in the image can be enhanced according to the color component adjustment result. Alternatively, the adjusted saturation can be used to enhance the display in other ways, such as recombining the adjusted saturation channel with the original hue and brightness channels, converting back to the red, green, and blue (RGB) color space to obtain an enhanced image, etc.

[0092] Compared with the related art, which uses a fixed saturation segmentation threshold to correct the saturation of an image, it is difficult to meet the image enhancement effect in specific scenarios and the personalized needs of users. In this embodiment, the saturation of the pixel points of the previous frame of the image collected by the endoscope is used to determine the saturation segmentation threshold, and the image enhancement is performed in combination with the blood enhancement intensity adaptively set by the user, thereby realizing dynamic adjustment of the image saturation, effectively improving the blood enhancement effect, and meeting the preferences and requirements of different users for image observation, further facilitating medical staff to observe the image, and thus improving diagnostic efficiency and accuracy.

[0093] Figure 3 This is a flow chart of another endoscopic image enhancement method provided by an embodiment of the present application. On the basis of the above embodiment, considering that the image is composed of a large number of pixels, especially in high-resolution images or video streams, the number of pixels is usually very large, which will lead to a large amount of calculation and thus affect the image processing efficiency. In order to improve the calculation efficiency in the saturation adjustment process, this embodiment obtains the division multiplication table set by the user by interacting with the user, and uses the division multiplication table to convert into a multiplication operation when a division operation is required, which can effectively improve the calculation efficiency and thus improve the image processing efficiency. Specifically, the display screen includes a second operation area for setting the division multiplication table. In addition to the above steps S201-S204, the method provided in this embodiment can also include the following step S301, and further divide the above step S203 into step S2031.

[0094] Step S301: In response to a control operation on a second operation area, the division multiplication table is obtained, wherein the division multiplication table includes a multiplication coefficient corresponding to each divisor used for calculation.

[0095] It is understandable that the implementation of division operations in a field-programmable gate array (FPGA) can usually be optimized by converting it into a multiplication operation, where the multiplication operation is more efficient than the division operation in hardware. Specifically, the concept of the reciprocal can be used to convert the division operation into a multiplication operation. For example, calculating A divided by B can be converted to calculating the reciprocal of A multiplied by B (1 / B). In FPGA, the approximate calculation of the reciprocal is achieved by looking up the division multiplication table (div2mulTab), thereby achieving efficient division operations.

[0096] Considering that the resolution and number of pixels of the image may be different during the image processing operation of different endoscopes, if a fixed division multiplication table (i.e., a lookup table) is used for operation, since the lookup table is usually designed for a specific range and resolution, its size and accuracy are limited. If the number of pixels or resolution of the image changes, the accuracy of the original lookup table may not be sufficient to meet the new requirements, resulting in inaccurate calculation results, thereby affecting the accuracy of the operation. This embodiment designs a second operation area in the display screen to facilitate interaction with the user and setting or updating the division multiplication table, so that when processing images with different resolutions or different numbers of pixels in different scenarios (such as different endoscopes), the adaptability of the division multiplication table is improved, thereby further improving the accuracy of the operation.

[0097] Optionally, the second operating area can be designed at the edge of the first operating area, such as Figure 4 As shown, the second operation area 112 is arranged vertically and side by side on the right side of the image. In some embodiments, it can also be designed within the first operation area, for example, the second operation area is a part of the first operation area, such as a sub-area or a pop-up window, or the first operation area and the second operation area are designed in a stacked manner, that is, one area visually covers another area. This embodiment does not specifically limit the design method of the first operation area and the second operation area on the UI interface.

[0098] Step S2031: According to the division multiplication table, the saturation segmentation threshold of the image and the divisor involved in the enhancement parameter calculation process are converted into corresponding multiplication coefficients to adjust the current saturation of the image.

[0099] Exemplarily, the image processing terminal uses a division multiplication table in the FPGA to convert division into multiplication, which can be done in the following manner:

[0100] For each possible divisor i (such as pixel value) from 0 to 255, calculate a multiplication table div2mulTab(i). Optionally, the calculation formula can be: div2mulTab(i) = round(16384 / ii), where 16384 is the scaling factor (the user can adjust it according to specific needs. For example, if higher accuracy is required, a larger scaling factor can be selected, or if faster calculation speed is required, a smaller scaling factor can be selected); round is the rounding function; ii = max(1,i), that is, when i = 0, it is regarded as 1 for calculation, because the numerator must also be 0 at this time to avoid the situation where the divisor is zero. At this time, the operation of dividing any variable y by a variable (where x takes a value range of [0,255]) can be achieved by table lookup and multiplication to obtain the division multiplication table div2mulTab, then the operation of dividing variable y by any variable x (∈[0,255]) can be converted to:

[0101]

[0102] Through the above technical solution, the image processing efficiency can be effectively improved, and the adaptability of the division multiplication table can be improved when processing images with different resolutions or different numbers of pixels in different scenarios (such as different endoscopes), thereby further improving the accuracy of operations.

[0103] In some embodiments, in order to further improve the efficiency and accuracy of determining the saturation segmentation threshold, the saturation segmentation threshold can be selected from the sorting results by saturation sorting. Specifically, the saturation segmentation threshold can be determined in the following manner:

[0104] According to the saturation statistical information about each pixel point in the previous frame image, the saturation of each pixel point is sorted to obtain a saturation sorting result; wherein the pixel point is a sampled pixel point after downsampling the previous frame image;

[0105] According to the saturation sorting result, the saturation of the corresponding pixel point is selected from each pixel point as the saturation segmentation threshold of the image.

[0106] For example, for the saturation statistics of each pixel point in the previous frame image, a sampling statistics method can be used to improve the statistical efficiency. The sampling statistics method can be performed by downsampling rate dspRat (for example, the sampling rate can be 2 n , that is, every 2 n Pixels select a pixel for processing, n is a non-negative integer, which determines the density of downsampling) to downsample the previous frame image, then the number of image sampling points based on the sampling rate dspRat is H_stat_len*W_stat_len:

[0107]

[0108] In the above formula, H_stat_len and W_stat_len are the height and width of the downsampled image, respectively. It can be understood that the downsampled image is a two-dimensional grid with a size of H_stat_len rows and W_stat_len columns. Therefore, H_stat_len*W_stat_len is the number of sampling points in the image, and H and W are the height and width of the original image, respectively. floor is the floor function. The sampled pixel points (hereinafter referred to as sampling points) are stored in the array S_stat. The saturation is solved by traversing the sampled data points. The sampling process can be as follows:

[0109] 1) Calculate the sequence number pnt of the sampling point in the S_stat array: pnt = (i-1)*W_stat_len+j, where i and j are the row and column indices of the sampling point in the downsampled image respectively;

[0110] 2) Calculate the vertical coordinate pnt_y of the sampling point in the original image img_in_pre: pnt_y = (i-1)*dspRat+1;

[0111] 3) Calculate the horizontal coordinate pnt_x of the sampling point in the original image img_in_pre: pnt_x = (j-1)*dspRat+1.

[0112] In this way, sampling points can be effectively extracted from the image and saturation calculations can be performed on these sampling points, thereby obtaining saturation statistical information for subsequent image processing steps and providing data support for selecting saturation segmentation thresholds.

[0113] Next, the saturation is calculated for each pixel that has been downsampled to obtain the saturation statistics of each pixel. Optionally, i and j traverse from 0 to H_stat_len-1 and from 0 to W_stat_len-1 respectively to cover all sampling points. The saturation calculation process of each sampling point can be to obtain the RGB value r / g / b of the current sampling point from the original image img_in_pre, and use r / g / b to calculate the saturation, where the r / g / b extraction process is:

[0114] r=img_in_pre(pnt_y,pnt_x,1);

[0115] g=img_in_pre(pnt_y,pnt_x,2);

[0116] b=img_in_pre(pnt_y,pnt_x,3);

[0117] Where r represents the value of the red channel, extracted from the position (pnt_y, pnt_x, 1); g represents the value of the green channel, extracted from the position (pnt_y, pnt_x, 2); b represents the value of the blue channel, extracted from the position (pnt_y, pnt_x, 3).

[0118] Calculate saturation using r / g / b The process of calculating saturation is:

[0119] MAX=max([r,g,b]); MIN=min([r,g,b]);

[0120] S_stat(pnt)=(MAX-MIN)*div2mulTab(MAX);

[0121] In the formula, i∈[0,H_stat_len-1],j∈[0,W_stat_len-1],max,min are functions for finding the maximum and minimum values ​​of the vector to calculate MAX-MIN. MAX-MIN represents the color range of the current sampling point, that is, the intensity difference of the color. The larger the difference, the higher the saturation. div2mulTab(MAX) is the division multiplication table, which is used to convert the division operation into the multiplication operation to improve the calculation efficiency. The calculated saturation value is stored in the array S_stat, which is the saturation statistics table, and the position (that is, the index of the sampling point) is pnt. In this way, the saturation of each sampling point can be effectively calculated and stored to obtain saturation statistics.

[0122] Next, according to the saturation of each sampling point in the saturation statistical information, the saturation of each pixel point is sorted. Assuming that the number of statistical pixel points is about 1000, the time complexity of the algorithm is allowed to be higher, thereby lowering the space complexity. The bubble sort method can be sampled (bubble sort is a simple sorting algorithm that gradually moves larger elements to the end of the array by traversing the array multiple times. In some embodiments, other sorting methods can also be used. This embodiment does not specifically limit the specific sorting method) to sort the statistical table S_stat to obtain the sorting result sSort. By way of example, the saturation statistical table S_stat can be first copied to a new array sSort for sorting operations, and then the length function is used to obtain the length LEN of the array sSort, which represents the total number of elements to be sorted. Based on the bubble sort method, the outer loop variable i starts from 0 and gradually increases to LEN-1 to control the number of sorting rounds, and the inner loop variable j starts from 0 and gradually increases to LEN-i-1 to compare and exchange adjacent elements. In each inner loop, the two adjacent elements sSort(j) and sSort(j+1) are compared. If sSort(j) is greater than sSort(j+1), the positions of the two elements are exchanged. This can be achieved by using a temporary variable tmp: by storing the value of sSort(j) in tmp, the value of sSort(j+1) is assigned to sSort(j). Assign the value of tmp to sSort(j+1). In this way, larger elements gradually "bubble" to the end of the array, and finally obtain a saturation array sSort arranged in ascending order. After completing the saturation sorting, the saturation of the corresponding pixel point can be selected from each pixel point according to the median or average of the saturation sorting.

[0123] In an optional implementation of this embodiment, in order to further improve the accuracy of the saturation segmentation threshold, this embodiment combines the threshold correction coefficient to correct the saturation segmentation threshold. Specifically, in the above steps, according to the saturation sorting result, the saturation of the corresponding pixel point is selected from each pixel point as the saturation segmentation threshold of the image, which can be adopted in the following manner:

[0124] According to the saturation sorting result and a preset threshold correction coefficient, a sorting index for querying the saturation of a target pixel among the pixels is determined; wherein the threshold correction coefficient is used to indicate data that reaches a preset percentage in the saturation sorting result as a saturation segmentation threshold;

[0125] According to the sorting index, the saturation of the target pixel is queried from the saturation sorting result, and the saturation of the pixel is determined as the saturation segmentation threshold of the image.

[0126] In this embodiment, the target pixel point is the pixel point used to determine the saturation segmentation threshold. By using the saturation sorting result and the threshold correction coefficient, the position of the target pixel point used to determine the saturation segmentation threshold is found, wherein a sorting index can be assigned to each sort in the sorting result (sSort), and the position of each pixel point can be quickly determined by the sorting index. Among them, the threshold correction coefficient is used to determine which position's saturation value is selected as the segmentation threshold, and can be a floating point number between 0 and 1, indicating the selected percentage position. Those skilled in the art can adaptively set the preset percentage ratio in combination with actual applications, such as 50% or other proportions, and this embodiment does not specifically limit this.

[0127] Exemplarily, the calculation formula of the saturation segmentation threshold S_thr may be as follows:

[0128] S_thr=sSort(round(satThrInx*H_stat_len*W_stat_len / 16))

[0129] In the formula, S_thr is the saturation segmentation threshold, satThrInx is the threshold correction coefficient, sSort(round(satThrInx*H_stat_len*W_stat_len / 16)) is to select the saturation of an element at a specific position (i.e., the element corresponding to the sorting index) in the sorting array sSort as the saturation segmentation threshold, where round(satThrInx*H_stat_len*W_stat_len / 16) is the sorting index, and H_stat_len*W_stat_len is the number of sampling points. It can be understood that 16 in the formula is a scaling factor used to scale the calculated index, and those skilled in the art can adjust it based on empirical values.

[0130] Through the above technical solution, the saturation segmentation threshold is dynamically adjusted using the threshold correction coefficient, which can adapt to the characteristics of different endoscopic images to improve the accuracy of the saturation segmentation threshold, thereby obtaining a better saturation segmentation effect.

[0131] In some embodiments, the enhancement parameter may include blood enhancement strength, and the saturation is corrected in combination with the blood enhancement strength to further improve the image enhancement effect. The saturation adjustment process is further introduced below. In the above steps, the current saturation of the image is adjusted according to the saturation segmentation threshold of the image and the enhancement parameter, which may include the following steps:

[0132] For each pixel in the image, correcting the current saturation of the pixel according to the saturation segmentation threshold and the blood enhancement strength to obtain a first corrected saturation of each pixel;

[0133] The adjusted saturation of the image is obtained according to the first corrected saturation of each pixel.

[0134] Exemplarily, each pixel in the image can be processed by traversing each pixel in the image in combination with the saturation segmentation threshold and the blood enhancement strength. Specifically, for each pixel, the saturation segmentation threshold is used to determine whether the saturation of the pixel needs to be corrected (if it is lower than the saturation segmentation threshold, it means that the pixel features corresponding to the blood or vascular tissue are not obvious enough, and it is corrected and enhanced). Instead of using fixed parameters for image enhancement in the related art, this embodiment receives the blood enhancement strength adjusted by the user to meet the image observation needs of a specific user, and adjusts the saturation of the pixel that needs to be corrected according to the adjustable blood enhancement strength, such as increasing the color intensity of the low saturation area to make it more significant (in some scenarios, the excessive saturation can also be reduced to avoid excessive enhancement), so as to obtain the first corrected saturation of each pixel after correction. By integrating the first corrected saturation of all pixels into a new image data, the saturation of the image adjusted by the user can be obtained, so as to achieve the purpose of further improving the image enhancement effect, thereby improving the diagnostic value and usability of the image.

[0135] In an optional implementation of this embodiment, considering that the image enhancement algorithm is usually nonlinear, in order to avoid excessive image enhancement, the enhancement parameters of this embodiment may also include a blood enhancement intensity threshold, and the blood enhancement intensity threshold is used to perform saturation correction to further optimize the image enhancement effect. Specifically, the method may also include the following steps:

[0136] For each pixel in the image, determining a blood enhancement amplitude according to a difference between the blood enhancement intensity and the blood enhancement intensity threshold;

[0137] According to the blood enhancement amplitude, the current saturation of the pixel point is corrected to obtain a second corrected saturation.

[0138] Exemplarily, the user can input or select a blood enhancement strength threshold value strLelMax in the first operation area according to his or her own needs (for example, the user can select a threshold value in the blood enhancement strength threshold range as the blood enhancement strength threshold value, and the blood enhancement strength threshold range can be determined based on empirical values) and calculate the blood enhancement amplitude (strLelMax-strLel) based on the difference between the blood enhancement strength strLel set by the user. The blood enhancement amplitude is a parameter used to enhance the visual performance of a specific area (such as a blood area) in the image, and the current saturation s_ori of the pixel point is further corrected based on the blood enhancement amplitude. The correction formula is: (strLelMax-strLel)*s_ori.

[0139] Wherein, s_ori is the current saturation of the pixel. Combined with the saturation calculation method mentioned above, s_ori = (MAX-MIN)*div2mulTab(MAX).

[0140] Furthermore, in the above steps, the saturation of the image after adjustment is obtained according to the first corrected saturation of each pixel point, specifically: the saturation of the image after adjustment is obtained according to the sum of the first corrected saturation and the second corrected saturation of each pixel point.

[0141] Exemplarily, in combination with the first corrected saturation and the second corrected saturation, the saturation adjustment process of the pixel point can be performed using the following formula: s_tmp=round(((strLelMax-strLel)*s_ori+strLel*sInx*s_ori / 1024)*div2mulTab(strLelMax+1) / 16384)

[0142] Where s_tmp is the adjusted saturation of the pixel, sInx is the saturation threshold segmentation coefficient corresponding to the saturation segmentation threshold, which is determined based on the saturation segmentation threshold and can be used to adjust the saturation segmentation threshold. For example, sInx = round((32*s_ori / S_thr) 2 ), in some embodiments, sInx may be S_thr, which is not particularly limited in this embodiment. It can be understood that (strLelMax-strLel)*s_ori is the second corrected saturation, which may be a positive value or a negative value. When the first corrected saturation is too large, it indicates that the image is over-enhanced, and the second corrected saturation weakens the image based on the blood enhancement intensity threshold. When the first corrected saturation is too small, it indicates that the image is under-enhanced, and the image is further enhanced based on the blood enhancement intensity threshold, thereby achieving the purpose of further improving the image enhancement effect.

[0143] In some embodiments, considering that there may be shadows, dark details or low-brightness areas in the image, if low grayscale saturation is not performed during the saturation adjustment of the image, details may be lost, and different users have different requirements for low grayscale protection. In order to solve this problem and further improve the overall enhancement effect of the image adapted to the user, the enhancement parameters of this embodiment may also include a grayscale protection threshold; in the above steps, the current saturation of the pixel point is corrected according to the saturation segmentation threshold and the blood enhancement strength, which can be done in the following way:

[0144] When the grayscale level of the pixel reaches the grayscale protection threshold, the current saturation of the pixel is corrected according to the saturation segmentation threshold and the blood enhancement intensity.

[0145] In this embodiment, the grayscale protection threshold, also known as the low grayscale protection threshold, is used to protect the details of the low grayscale area in the image to avoid the loss of important information during the enhancement process. The user can make adaptive settings based on their needs for low grayscale protection of the image. Similar to the above-mentioned blood enhancement strength threshold, the terminal can also determine the blood enhancement strength threshold range through empirical values, and the user can select the grayscale protection threshold that meets their needs from the range through control operations.

[0146] Exemplarily, when traversing the pixels for processing, the grayscale level of each pixel can be checked. Specifically, the grayscale value of the current pixel is obtained to determine whether the grayscale value reaches the grayscale protection threshold. If the grayscale value is lower than the threshold, it means that the area needs to be protected to avoid loss of details. If the grayscale level reaches or exceeds the grayscale protection threshold, saturation correction is performed. The saturation correction process can be found above and will not be described in detail here.

[0147] Through the above technical solution, the low grayscale protection area in the image can be effectively prevented from being over-enhanced, and at the same time, it can adapt to the personalized needs of different users.

[0148] In an optional implementation of this embodiment, in order to further protect the saturation correction effect of the pixel points that reach the gray protection threshold, this embodiment uses a low gray protection coefficient to constrain the saturation correction process. Specifically, the above method may also include the following steps:

[0149] A low grayscale protection coefficient is determined according to a grayscale protection threshold and maximum color components of the image with respect to red, green and blue channels.

[0150] Exemplarily, for each pixel in the image, the color components of the three channels of red, green, and blue (RGB) can be analyzed separately to find the component with the largest value among the three channels, which represents the main color feature of the pixel. Then, based on the grayscale protection threshold determined previously and the maximum color component of each pixel (i.e., the maximum color component of the image with respect to the RGB channel), a low grayscale protection coefficient is calculated to adjust the intensity of the saturation correction to ensure that during the correction process, pixels that reach or approach the grayscale protection threshold are not over-adjusted. Optionally, the grayscale protection coefficient can be calculated using the following formula:

[0151] lowGrayFreeRat=256*MAX / lowGrayFreeThr;

[0152] lowGrayFreeRat=min(256,lowGrayFreeRat);

[0153] In the formula, lowGrayFreeRat represents the low gray protection coefficient, lowGrayFreeThr represents the gray protection threshold, and MAX is the value of the maximum color component in the red, green, and blue (RGB) channels of a pixel in the image. Among them, min(256,lowGrayFreeRat) is used to set the low gray protection coefficient lower than 256 (which can be adaptively adjusted based on empirical values) to prevent the low gray protection coefficient from being too large, thereby causing unnecessary impact on saturation correction.

[0154] Furthermore, in the above steps, the current saturation of the pixel is corrected according to the saturation segmentation threshold and the blood enhancement strength. Specifically, when the grayscale level of the pixel reaches the grayscale protection threshold, the current saturation of the pixel is corrected according to the saturation segmentation threshold and the blood enhancement strength, and the correction result is not lower than the low grayscale protection coefficient.

[0155] In this embodiment, when performing saturation correction, the calculated low-gray protection coefficient is used to constrain the pixels that reach the gray protection threshold. That is, when adjusting the saturation of these pixels, the adjustment amplitude is reduced according to the low-gray protection coefficient, so as to protect the details of the image and the natural expression of colors. Specifically, when the gray level of a pixel reaches the gray protection threshold, saturation correction is required. The correction may be to increase the saturation (adjust upward) to make the color more vivid; it can also be to reduce the saturation (adjust downward) to make the color softer or closer to gray. For the case of adjusting downward, when the saturation needs to be reduced, in order to prevent the saturation from being overly reduced so that the color becomes too dull or loses its due color characteristics, the low-gray protection coefficient is used for constraint, so that the adjustment result cannot be lower than the low-gray protection coefficient. While enhancing the overall color saturation of the image, overprocessing of low-gray pixels can be avoided, and the visual quality and color balance of the image can be maintained.

[0156] In some embodiments, the low-gray protection coefficient can also be used as the protection degree of low-gray pixels, that is, as a weight coefficient, to protect low-gray pixels and reduce the influence of noise, so that the saturation of the finally output pixel points is as follows:

[0157] s_out = (s_tmp * lowGrayFreeRat + (256 - lowGrayFreeRat) * s_ori) / 256;

[0158] s_out = min(16384, s_out);

[0159] In the formula, s_out is the saturation of the pixel points finally output after saturation adjustment in combination with the low-gray protection coefficient, and the finally output saturation is lower than 16384 (those skilled in the art can adaptively set this value in combination with empirical values).

[0160] Through the above technical solution, using the low-gray protection coefficient for constraint in the saturation correction process can protect low-gray pixels, and at the same time constrain the overprocessing when the saturation is adjusted downward, further optimizing the image enhancement effect.

[0161] Figure 5 It is a schematic flowchart of another endoscope image enhancement method provided by an embodiment of the present application. On the basis of the above embodiment, this embodiment realizes image enhancement by adjusting the color components of the RGB channels to further optimize the effect of enhancing blood characteristics. Specifically, as Figure 5 shown, the above step S204 enhances and displays the blood or blood vessel tissue in the image according to the adjusted saturation, and is further divided into the following step S501 and step S502:

[0162] Step S501: Adjust the corresponding color components in the red, green and blue channels of the image according to the adjusted saturation to obtain a color component adjustment result.

[0163] It can be understood that adjusting the saturation can highlight specific color features. In the scene of enhancing blood or vascular tissue in endoscopic images, since blood and vascular tissue usually appear in red or near-red tones, the adjusted saturation can be used to enhance the component of the red channel to highlight the color of the blood and blood vessels.

[0164] In an optional implementation, the corresponding color components in the red, green and blue channels of the image are adjusted using the adjusted saturation, which can be done in the following manner:

[0165] Adjusting the minimum color component in the red, green and blue channels of the image according to the adjusted saturation and the maximum color component in the red, green and blue channels of the image;

[0166] The intermediate color components in the red, green and blue channels of the image are adjusted according to the maximum color component and the adjusted minimum color component by using a preset interpolation algorithm.

[0167] Exemplarily, each pixel of the current frame image can be traversed, such as using two nested (i, j, where i = 0:1:H-1, j = 0:1:N-1, H is the height of the image, and N is the width of the image) loops to traverse each pixel of the image. For each pixel, the RGB components, i.e., the red r, green g, and blue b components, are extracted, and the maximum value MAX (i.e., the maximum color component) and the minimum value MIN (i.e., the minimum color component) are determined according to the size relationship of the RGB components. Different situations are processed according to the size relationship of the RGB channels. In combination with the corrected saturation, the RGB components are adjusted to make the red (or other target colors) more prominent, thereby enhancing the display of blood or vascular tissue. For each case, one color component is kept unchanged (i.e., the component corresponding to the maximum value MAX), and the other two components are adjusted according to the adjusted saturation value.

[0168] Next, we will further introduce the preset interpolation algorithm and the processing methods for different situations according to the size relationship of RGB channels as follows:

[0169] Case 1, if r>g>=b, the maximum color component MAX is the red component r, the minimum color component MIN is the blue component b, the red component ro remains unchanged, and the blue component bo is adjusted to MAX*(16384-s_tmp) / 16384. The intermediate color component, that is, the green component go, can be adjusted by the interpolation algorithm using the following formula: go=bo+round((ro-bo)*(gb)*div2mulTab(rb)) / 16384; where s_tmp is the adjusted saturation, which can be replaced with s_out in the above embodiment, and this embodiment does not limit this, and div2mulTab(rb) is a division multiplication table involving (r–b) calculation that can be adjusted by the user.

[0170] Case 2: r>=b>g, the maximum value MAX is r, the minimum value MIN is g, the red component ro remains unchanged, and the green component go is adjusted to MAX*(16384-s_tmp) / 16384. The intermediate color component, that is, the blue component bo, can be adjusted by the interpolation algorithm using the following formula: bo=ro-round((ro-go)*(rb)*div2mulTab(rg)) / 16384;

[0171] Case 3: g>=r>b, the maximum value MAX is g, the minimum value MIN is b, and the green component go remains unchanged. Adjust the blue component bo to MAX*(16384-s_tmp) / 16384, and adjust the intermediate color component through the interpolation algorithm, that is, the red component ro=go-round((go-bo)*(gr)*div2mulTab(gb)) / 16384;

[0172] Case 4: b>r>=g, the maximum value MAX is b, and the minimum value MIN is g. The blue component bo remains unchanged, the green component go is adjusted to MAX*(16384-s_tmp) / 16384, and the middle color component is adjusted by the interpolation algorithm, that is, the red component ro=go+round((bo-go)*(rg)*div2mulTab(bg)) / 16384;

[0173] Case 5: g>b>=r, the maximum value MAX is g, and the minimum value MIN is r. The green component go remains unchanged, and the red component ro is adjusted to MAX*(16384-s_tmp) / 16384. The interpolation algorithm is used to adjust the intermediate color component, that is, the blue component bo=ro+round((go-ro)*(br)*div2mulTab(gr)) / 16384;

[0174] Case 6: b>g>=r, the maximum value MAX is b, and the minimum value MIN is r. The blue component bo remains unchanged, and the red component ro is adjusted to MAX*(16384-s_tmp) / 16384. The interpolation algorithm is used to adjust the intermediate color component, that is, the green component go is bo-round((bo-ro)*(bg)*div2mulTab(br)) / 16384.

[0175] Step S502: adjusting the result according to the color component to enhance the display of blood or blood vessel tissue in the image.

[0176] Combined with the color component adjustment results ro, go, and bo obtained in the above-mentioned situations, the color components ro, go, and bo are respectively used as the red, green, and blue components of the current pixel and stored in the output image img_out to obtain an enhanced image and realize enhanced display of blood or vascular tissue in the image. It has been verified that the effect of image enhancement before and after using the technical solution of this embodiment is as follows: Figure 6a , Figure 6b , and Figure 7a and Figure 7b shown.

[0177] The above method directly adjusts the saturation in the RGB space by judging the size relationship of the RGB channels, thereby eliminating the process of solving the hue H and converting the HSV and RGB color spaces, saving computing resources and improving image enhancement efficiency.

[0178] In summary, this embodiment realizes the blood enhancement function through adaptive image saturation segmentation and image color adjustment, which can effectively avoid visual fatigue of doctors caused by the similar colors of mucosa, blood and blood vessels, and can flexibly adapt to the preferences and visual differences of different medical staff.

[0179] Figure 8 is a schematic diagram of the structure of an endoscope image enhancement device provided in an embodiment of the present application, such as Figure 8 As shown, the device 800 includes a display module 801, a parameter acquisition module 802 and an enhancement module 803, wherein:

[0180] A display module 801, which is configured to display the image acquired by the endoscope through a display screen after acquiring the image, and the display screen includes a first operation area for adjusting image enhancement parameters;

[0181] A parameter acquisition module 802, configured to acquire enhancement parameters of the image in response to a control operation on the first operation area;

[0182] an enhancement module 803, configured to adjust the current saturation of the image according to the saturation segmentation threshold of the image and the enhancement parameter, so as to enhance and display the blood or vascular tissue in the image according to the adjusted saturation;

[0183] The saturation segmentation threshold is determined based on saturation statistical information of at least a plurality of pixel points of a previous frame of image acquired by the endoscope.

[0184] In an optional implementation, a threshold determination module for determining a saturation segmentation threshold is further included, and the threshold determination module includes:

[0185] A sorting unit, which is configured to sort the saturation of each pixel point according to the saturation statistical information about each pixel point in the previous frame image to obtain a saturation sorting result; wherein the pixel point is a sampled pixel point after downsampling the previous frame image;

[0186] A selection unit is configured to select the saturation of a corresponding pixel point from each pixel point according to the saturation sorting result as a saturation segmentation threshold of the image.

[0187] In an optional implementation manner, the selection unit is specifically configured to:

[0188] According to the saturation sorting result and a preset threshold correction coefficient, a sorting index for querying the saturation of a target pixel among the pixels is determined; wherein the threshold correction coefficient is used to indicate data that reaches a preset percentage in the saturation sorting result as a saturation segmentation threshold;

[0189] According to the sorting index, the saturation of the target pixel is queried from the saturation sorting result, and the saturation of the pixel is determined as the saturation segmentation threshold of the image.

[0190] In an optional embodiment, the enhancement parameter includes blood enhancement intensity; the adjustment module includes:

[0191] a correction unit configured to correct, for each pixel in the image, a current saturation of the pixel according to the saturation segmentation threshold and the blood enhancement strength, to obtain a first corrected saturation of each pixel;

[0192] An acquisition unit is configured to obtain the adjusted saturation of the image according to the first corrected saturation of each pixel.

[0193] In an optional embodiment, the enhancement parameter further includes a blood enhancement intensity threshold; the device further includes:

[0194] an enhancement amplitude determination module, configured to determine the blood enhancement amplitude for each pixel point in the image according to the difference between the blood enhancement intensity and the blood enhancement intensity threshold;

[0195] The correction unit is further configured to correct the current saturation of the pixel point according to the blood enhancement amplitude to obtain a second corrected saturation;

[0196] The acquisition unit is specifically configured to obtain the adjusted saturation of the image according to the sum of the first corrected saturation and the second corrected saturation of each pixel.

[0197] In an optional embodiment, the enhancement parameters include a grayscale protection threshold; the correction unit is specifically configured to correct the current saturation of the pixel point according to the saturation segmentation threshold and the blood enhancement intensity when the grayscale level of the pixel point reaches the grayscale protection threshold.

[0198] In an optional embodiment, the device further comprises:

[0199] A protection coefficient determination module, configured to determine a low grayscale protection coefficient according to a grayscale protection threshold and a maximum color component of the image with respect to red, green and blue channels;

[0200] The correction unit is specifically configured to correct the current saturation of the pixel point according to the saturation segmentation threshold and the blood enhancement strength when the grayscale level of the pixel point reaches the grayscale protection threshold, and make the correction result not lower than the low grayscale protection coefficient.

[0201] In an optional implementation, the display screen includes a second operation area for setting a division multiplication table; the device further includes:

[0202] a multiplication table acquisition module, configured to acquire the division multiplication table in response to a control operation for the second operation area, wherein the division multiplication table includes a multiplication coefficient corresponding to each divisor used for calculation;

[0203] The enhancement module 803 includes: a conversion unit, which is configured to convert the saturation segmentation threshold of the image and the divisor involved in the enhancement parameter calculation process into a corresponding multiplication coefficient according to the division multiplication table to adjust the current saturation of the image.

[0204] In an optional implementation, the enhancement module 803 includes:

[0205] A color adjustment unit, which is configured to adjust corresponding color components in red, green and blue channels in the image according to the adjusted saturation to obtain a color component adjustment result;

[0206] A color enhancement unit is configured to adjust the result according to the color component to enhance the display of blood or blood vessel tissue in the image.

[0207] In an optional implementation, the color adjustment unit is specifically configured to:

[0208] Adjusting the minimum color component in the red, green and blue channels of the image according to the adjusted saturation and the maximum color component in the red, green and blue channels of the image;

[0209] The intermediate color components in the red, green and blue channels of the image are adjusted according to the maximum color component and the adjusted minimum color component by using a preset interpolation algorithm.

[0210] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the parameter acquisition module 802 can be a separately established processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The function of the above parameter acquisition module 802. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0211] The relevant instructions can be understood by referring to the relevant descriptions and effects corresponding to the steps in the method embodiments of the present application, and no further elaboration is made here.

[0212] Fig. 9 An image processing device provided in an embodiment of the present application, such as Fig. 9 As shown, the image processing device 900 may include: a memory 901 and a processor 902;

[0213] The memory 901 stores computer-executable instructions;

[0214] The processor 902 executes the computer-executable instructions stored in the memory, so that the image processing device executes the endoscopic image enhancement method provided by the above method embodiment. In addition, a transceiver 903 for data communication with an external device may also be included.

[0215] The relevant instructions can be understood by referring to the relevant descriptions and effects corresponding to the steps in the method embodiments of the present application, and no further elaboration is made here.

[0216] The embodiment of the present application also provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are executed on a computer, the computer executes the endoscopic image enhancement method provided by the above method embodiment.

[0217] The relevant instructions can be understood by referring to the relevant descriptions and effects corresponding to the steps in the method embodiments of the present application, and no further elaboration is made here.

[0218] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, it can implement the endoscopic image enhancement method provided in any one of the first aspects above.

[0219] The relevant instructions can be understood by referring to the relevant descriptions and effects corresponding to the steps in the method embodiments of the present application, and no further elaboration is made here.

[0220] Fig. 9 A schematic diagram of the structure of an image enhancement system provided in an embodiment of the present application is shown in FIG. Fig.10 The system 1000 includes an image processing device 900 as provided in the above embodiment, and an endoscope 1001 electrically connected to the image processing device 900, wherein the endoscope 1001 is used to collect images of organs in the patient's body and transmit the collected images to the image processing device 900 for image enhancement.

[0221] The relevant instructions can be understood by referring to the relevant descriptions and effects corresponding to the steps in the method embodiments of the present application, and no further elaboration is made here.

[0222] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium).

[0223] As is well known to those skilled in the art, the term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer.

[0224] Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0225] In the description of the embodiments of the present application, the term "and / or" merely represents an association relationship that describes associated objects, indicating that three relationships may exist. For example, A and / or B may represent three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" represents any combination of at least two of any one or more of a plurality of. For example, at least one of A, B, may represent any one or more elements selected from a set that communicates A, B, and C. In addition, the term "plurality" means two or more, unless otherwise precisely and specifically specified.

[0226] In the description of the embodiment of the present application, the terms "first", "second", "third", "fourth", etc. (if present) are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0227] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An endoscopic image enhancement method, characterized in that: include: After acquiring an image collected by the endoscope, displaying the image through a display screen, the display screen includes a first operation area for adjusting image enhancement parameters; In response to a control operation on the first operation area, acquiring enhancement parameters of the image; According to the saturation segmentation threshold of the image and the enhancement parameter, the current saturation of the image is adjusted to enhance the display of blood or vascular tissue in the image according to the adjusted saturation; The saturation segmentation threshold is determined based on saturation statistical information of at least a plurality of pixel points of a previous frame of image acquired by the endoscope.

2. The method according to claim 1, characterized in that The method for determining the saturation segmentation threshold includes: According to the saturation statistical information about each pixel point in the previous frame image, the saturation of each pixel point is sorted to obtain a saturation sorting result; wherein the pixel point is a sampled pixel point after downsampling the previous frame image; According to the saturation sorting result, the saturation of the corresponding pixel point is selected from each pixel point as the saturation segmentation threshold of the image.

3. The method according to claim 2, characterized in that The selecting, from each pixel according to the saturation sorting result, the saturation of the corresponding pixel as the saturation segmentation threshold of the image includes: According to the saturation sorting result and a preset threshold correction coefficient, a sorting index for querying the saturation of a target pixel among the pixels is determined; wherein the threshold correction coefficient is used to indicate data that reaches a preset percentage in the saturation sorting result as a saturation segmentation threshold; According to the sorting index, the saturation of the target pixel is queried from the saturation sorting result, and the saturation of the pixel is determined as the saturation segmentation threshold of the image.

4. The method according to claim 1, characterized in that: The enhancement parameters include blood enhancement intensity; The adjusting the current saturation of the image according to the saturation segmentation threshold of the image and the enhancement parameter comprises: For each pixel in the image, correcting the current saturation of the pixel according to the saturation segmentation threshold and the blood enhancement strength to obtain a first corrected saturation of each pixel; The adjusted saturation of the image is obtained according to the first corrected saturation of each pixel.

5. The method according to claim 4, characterized in that The enhancement parameter further includes a blood enhancement intensity threshold; the method further includes: For each pixel in the image, determining a blood enhancement amplitude according to a difference between the blood enhancement intensity and the blood enhancement intensity threshold; According to the blood enhancement amplitude, the current saturation of the pixel point is corrected to obtain a second corrected saturation; The step of obtaining the adjusted saturation of the image according to the first corrected saturation of each pixel point includes: The adjusted saturation of the image is obtained according to the sum of the first corrected saturation and the second corrected saturation of each pixel.

6. The method according to claim 4 or 5, characterized in that: The enhancement parameters include a grayscale protection threshold; The correcting the current saturation of the pixel point according to the saturation segmentation threshold and the blood enhancement strength includes: When the grayscale level of the pixel reaches the grayscale protection threshold, the current saturation of the pixel is corrected according to the saturation segmentation threshold and the blood enhancement intensity.

7. The method according to claim 4 or 5, characterized in that: Also includes: Determine a low grayscale protection coefficient according to a grayscale protection threshold and a maximum color component of the image with respect to red, green and blue channels; The correcting the current saturation of the pixel point according to the saturation segmentation threshold and the blood enhancement strength includes: When the grayscale level of the pixel reaches the grayscale protection threshold, the current saturation of the pixel is corrected according to the saturation segmentation threshold and the blood enhancement strength, and the correction result is not lower than the low grayscale protection coefficient.

8. The method according to any one of claims 1 to 5 and 7, characterized in that: The display screen includes a second operation area for setting a division multiplication table; the method further includes: In response to a control operation for the second operation area, acquiring the division multiplication table, wherein the division multiplication table includes a multiplication coefficient corresponding to each divisor used for calculation; The adjusting the current saturation of the image according to the saturation segmentation threshold of the image and the enhancement parameter comprises: According to the division multiplication table, the saturation segmentation threshold of the image and the divisor involved in the enhancement parameter calculation process are converted into corresponding multiplication coefficients to adjust the current saturation of the image.

9. The method according to any one of claims 1 to 5 and 7, characterized in that: The step of enhancing the display of the blood or blood vessel tissue in the image according to the adjusted saturation comprises: According to the adjusted saturation, the corresponding color components in the red, green and blue channels of the image are adjusted to obtain a color component adjustment result; The result is adjusted according to the color component to enhance the display of blood or blood vessel tissue in the image.

10. The method according to claim 9, characterized in that The step of adjusting corresponding color components in red, green and blue channels of the image according to the adjusted saturation comprises: Adjusting the minimum color component in the red, green and blue channels of the image according to the adjusted saturation and the maximum color component in the red, green and blue channels of the image; The intermediate color components in the red, green and blue channels of the image are adjusted according to the maximum color component and the adjusted minimum color component by using a preset interpolation algorithm.

11. An image processing device, characterized in that: include: Memory and processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the image processing device performs the endoscopic image enhancement method according to any one of claims 1 to 10.

12. An image enhancement system, characterized in that: It comprises the image processing device as described in claim 11, and an endoscope electrically connected to the image processing device, the endoscope is used to collect images of organs in the patient's body and transmit the collected images to the image processing device for image enhancement.

Images