Image processing method and device, endoscope, equipment and storage medium

Abstract

The embodiment of the invention provides an image processing method and device, an endoscope, equipment and a storage medium, and the method comprises the steps: obtaining a brightness image according to an HSV image of an original RGB image; respectively obtaining a binary image and a brightness enhanced image according to the brightness image; according to the binary image, obtaining boundary pixel points at the boundary of the bright area and the dark area in the brightness enhanced image; for each boundary pixel point, obtaining a to-be-corrected point set corresponding to the boundary pixel point from the brightness enhanced image; for each to-be-corrected point set, according to the brightness values of all the pixel points in the to-be-corrected point set, updating the brightness values of the pixel points in the to-be-corrected point set to obtain a target point set corresponding to the to-be-corrected point set; according to all the target point sets, updating the brightness of pixel points at corresponding positions in the brightness enhanced image to obtain a target brightness image; updating V channel data in the HSV image according to the target brightness image to obtain a target HSV image; and obtaining a target RGB image according to the target HSV image.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method, device, endoscope, equipment and storage medium. Background Art

[0002] In low-light environments, the images captured by image acquisition devices will have quality problems such as high noise, blurred texture information, low contrast, and incorrect color reproduction, which will affect the quality of subsequent image processing. For example, in clinical application scenarios, in order to observe the tissue morphology of human internal organs and the pathological conditions in the body, and to provide important basis for clinical diagnosis, disease tracking, and medical plan determination, medical personnel will use endoscopes to penetrate the human body to shoot videos of human internal organs and cavities. However, due to the low-light environment in the human cavity and organs, even if the front end of the endoscope can provide lighting, the captured images will still have low-light areas, which will lead to the above-mentioned quality problems in the image. As a result, not only does it lead to poor visual perception, but it also causes some errors in image transmission, which seriously affects the clinical use of the endoscope. Summary of the invention

[0003] In view of this, in order to at least solve the quality problem of images in low-light environments in the related art, an object of the present invention is to provide an image processing method, apparatus, endoscope, device and storage medium.

[0004] In order to achieve the above purpose, the technical solution adopted by the embodiment of the present invention is as follows:

[0005] A first aspect of an embodiment of the present invention provides an image processing method, comprising:

[0006] According to the HSV image of the original RGB image, the brightness image is obtained;

[0007] Obtaining a binary image according to the negative image of the brightness image;

[0008] Obtaining a brightness enhanced image according to the brightness image;

[0009] According to the binary image, a boundary pixel point at the boundary between a bright area and a dark area in the brightness enhanced image is obtained;

[0010] For each boundary pixel point, a set of points to be corrected corresponding to the boundary pixel point is obtained from the brightness enhanced image; the set of points to be corrected includes the boundary pixel point and pixel points located on the dark area side and the bright area side of the boundary pixel point respectively; all the pixel points in the set of points to be corrected are adjacent to each other and are on the same straight line;

[0011] For each point set to be corrected, the brightness values ​​of the pixels in the point set to be corrected are updated according to the brightness values ​​of all the pixels in the point set to be corrected, so as to obtain a target point set corresponding to the point set to be corrected; the brightness uniformity of all the pixels in the target point set is higher than the brightness uniformity of all the pixels in the point set to be corrected;

[0012] According to all target point sets, the brightness of the pixel points at the corresponding positions in the brightness enhanced image is updated to obtain a target brightness image;

[0013] According to the target brightness image, update the V channel data in the HSV image to obtain a target HSV image;

[0014] A target RGB image is obtained according to the target HSV image.

[0015] In an optional implementation, the step of obtaining a binary image based on the negative image of the brightness image comprises:

[0016] Performing a power operation on the negative image according to a set index to obtain a negative image after the power operation;

[0017] Obtaining a brightness threshold according to the brightness value of the original RGB image;

[0018] In the negative image after the power operation, the brightness values ​​of the pixels whose brightness values ​​are greater than or equal to the brightness threshold are updated to 255, and the brightness values ​​of the pixels whose brightness values ​​are less than the brightness threshold are updated to 0, so as to obtain the corresponding binary image.

[0019] In an optional implementation manner, the value range of the setting index is (1, 2); and / or

[0020] The step of obtaining a brightness threshold according to the brightness value of the original RGB image comprises:

[0021] According to the brightness value of the original RGB image, a brightness reference value of the original RGB image is calculated; the brightness reference value is the brightness mean value or the brightness median value of the original RGB image;

[0022] The product of the brightness reference value and the set brightness coefficient is calculated to obtain the brightness threshold; wherein the value range of the set brightness coefficient is [0.7, 1.1].

[0023] In an optional implementation, the set index value is 1.6, and / or the set brightness coefficient value is 0.9.

[0024] In an optional implementation, the step of obtaining a brightness enhanced image according to the brightness image includes:

[0025] According to the set brightness enhancement matrix, the brightness transformation matrix is ​​calculated;

[0026] The brightness image is subjected to power operation processing according to the brightness transformation matrix to obtain a brightness enhanced image.

[0027] In an optional implementation, the step of calculating a brightness transformation matrix according to a set brightness enhancement matrix includes:

[0028] By using a first calculation formula and a second calculation formula, respectively based on the brightness enhancement matrix, a first intermediate coefficient matrix and a second intermediate coefficient matrix are calculated;

[0029] Calculate a brightness transformation matrix based on the first intermediate coefficient matrix and the second intermediate coefficient matrix using a third calculation formula;

[0030] The first calculation formula is:

[0031] matMidCoef=1.05+(dLumEnhance / 113.0-0.03);

[0032] The second calculation formula is:

[0033] matBrightenCoef=(dLumEnhance / 113.0-0.03) / 3.0;

[0034] The third calculation formula is:

[0035] matPower=1.0 / matMidCoef / (1-matBrightenCoef);

[0036] Among them, matMidCoef represents the first intermediate coefficient matrix, dLumEnhance represents the brightness enhancement matrix, matBrightenCoef represents the second intermediate coefficient matrix, and matPower represents the brightness transformation matrix.

[0037] In an optional implementation, the step of performing a power operation on the brightness image according to the brightness transformation matrix to obtain a brightness enhanced image includes:

[0038] A brightness enhanced image is obtained by calculation based on the brightness transformation matrix and the brightness image through a fourth calculation formula; in the brightness enhanced image, the brightness increase range of the dark area is greater than the brightness increase range of the bright area;

[0039] The fourth calculation formula is:

[0040] matVBrighten=255*(matV / 255.0)^matPower;

[0041] Among them, matVBrighten represents the brightness enhanced image, matV represents the brightness image, and matPower represents the brightness transformation matrix.

[0042] In an optional implementation, the step of obtaining boundary pixel points at the boundary between the bright area and the dark area in the brightness enhanced image according to the binary image includes:

[0043] Calculating the area of ​​each dark region in the binary image;

[0044] According to the image position of the edge pixel point of the dark area whose area is larger than the set area threshold, the corresponding boundary pixel point is obtained from the brightness enhanced image.

[0045] In an optional implementation, the step of updating the brightness values ​​of the pixels in the to-be-corrected point set according to the brightness values ​​of all the pixels in the to-be-corrected point set includes:

[0046] A brightness reference value is calculated based on the brightness values ​​of all the pixels in the set of points to be corrected; the brightness reference value is the brightness mean or brightness median of all the pixels in the set of points to be corrected;

[0047] Lowering the brightness value of the pixel points in the to-be-corrected point set whose brightness value is higher than the brightness reference value;

[0048] Increase the brightness value of the pixel points in the to-be-corrected point set whose brightness value is lower than the brightness reference value.

[0049] In an optional implementation, for each pixel point with adjusted brightness in the to-be-corrected point set, the brightness adjustment amplitude of the pixel point with adjusted brightness is negatively correlated with the distance between the pixel point with adjusted brightness and the corresponding boundary pixel point.

[0050] According to a second aspect of an embodiment of the present invention, there is provided an image processing device, comprising:

[0051] The acquisition module is configured to: obtain a brightness image according to the HSV image of the original RGB image;

[0052] A binarization processing module is configured to: obtain a binary image according to a negative image of the brightness image;

[0053] A brightness enhancement module is configured to: obtain a brightness enhanced image according to the brightness image;

[0054] The boundary determination module is configured to: obtain boundary pixel points at the boundary between the bright area and the dark area in the brightness enhanced image according to the binary image; for each boundary pixel point, obtain a set of points to be corrected corresponding to the boundary pixel point from the brightness enhanced image; the set of points to be corrected includes the boundary pixel point, and pixel points located on the dark area side and the bright area side of the boundary pixel point respectively; all pixel points in the set of points to be corrected are adjacent to each other and are on the same straight line;

[0055] The correction module is configured to: for each point set to be corrected, update the brightness values ​​of the pixels in the point set to be corrected according to the brightness values ​​of all the pixels in the point set to be corrected, so as to obtain a target point set corresponding to the point set to be corrected; the brightness uniformity of all the pixels in the target point set is higher than the brightness uniformity of all the pixels in the point set to be corrected; according to all the target point sets, update the brightness of the pixels at the corresponding positions in the brightness enhanced image to obtain a target brightness image; according to the target brightness image, update the V channel data in the HSV image to obtain a target HSV image;

[0056] The image conversion module is configured to obtain a target RGB image according to the target HSV image.

[0057] According to a third aspect of an embodiment of the present invention, there is provided an endoscope, comprising:

[0058] Image acquisition device, used for acquiring original RGB images;

[0059] An image processing device, configured to execute the method according to any one of claims 1 to 10, so as to obtain a target RGB image based on the original RGB image;

[0060] and a display device for displaying the target RGB image.

[0061] According to a fourth aspect of an embodiment of the present invention, there is provided an electronic device, comprising a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor can execute the machine executable instructions to implement the image processing method provided in the first aspect above.

[0062] According to a fifth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the image processing method provided in the first aspect is implemented.

[0063] The image processing method, apparatus, endoscope, device and storage medium provided by any of the above embodiments of the present invention have at least the following beneficial technical effects:

[0064] By obtaining a brightness image based on the HSV image of the original RGB image, and then obtaining a binary image and a brightness enhanced image based on the brightness image, and then obtaining the boundary pixel points on the boundary line between the dark area and the bright area in the brightness enhanced image based on the binary image, the boundary recognition accuracy of the dark area and the bright area can be improved. Since the boundary pixel points can be located based on the binary image, the amount of data processing can also be reduced. Then, the corresponding set of points to be corrected is obtained based on the boundary pixel points, and the set of points to be corrected includes boundary pixel points, and pixel points located on the dark area side and the bright area side of the boundary pixel points, respectively, and these pixel points are adjacent to each other and are on the same straight line. It can be seen that all the pixel points in the set of points to be corrected are the points that best represent the brightness difference between the dark area and the bright area. Therefore, the brightness values ​​of the pixels in the to-be-corrected point set are continuously updated based on the brightness values ​​of all the pixels in the to-be-corrected point set, so that the brightness of the pixels after brightness correction has higher uniformity, and then the corresponding target point set is obtained. It can be seen that the brightness of the point that best represents the brightness difference between the dark area and the bright area has been smoothed, which not only can achieve a smooth transition of the brightness between the dark area and the bright area, but also can achieve an improvement in the brightness of the dark area, so that the target HSV image after the V channel data is updated based on the target point set also has the same technical effect, and then the target RGB image obtained based on the target HSV image conversion not only has a good brightness improvement effect in the dark area, but also has a good image color restoration effect as a whole, and the image quality is effectively improved. In addition, since it is only necessary to process the V channel data (i.e., the brightness image) in the HSV image, there is no need to process the image data of the H channel and the S channel, and the image processing does not involve a large amount of computing operations, any of the above-mentioned solutions provided by the embodiments of the present invention also has the beneficial technical effects of high image processing efficiency, the playback speed of the image sequence meets the video playback requirements (30 frames / s, and the processing time for each frame of the image is within 33ms), there will be no freezes, and good clinical use effects.

[0065] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0067] Figure 1 A structural block diagram of an electronic device provided by an embodiment of the present invention is shown;

[0068] Figure 2 A flowchart of an image processing method provided by an embodiment of the present invention is shown;

[0069] Figure 3 A schematic diagram showing the effect comparison between an original RGB image and a target RGB image provided by an embodiment of the present invention is shown;

[0070] Figure 4 A schematic diagram of an endoscope provided by an embodiment of the present invention is shown;

[0071] Figure 5 A functional module diagram of an image processing device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0072] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0073] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0074] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0075] In low-light environments, the images captured by image acquisition devices may have quality problems such as high noise, blurred texture information, low contrast, and incorrect color reproduction, which will affect the quality of subsequent image processing. For example, in the clinical application scenarios of endoscopes, since the human body's cavities and organs are low-light environments, even if the front end of the endoscope can provide lighting, the captured images will still have low-light areas, which will lead to the above-mentioned quality problems in the images.

[0076] To solve the above technical problems, there are three main solutions in the related technologies: one is to use the traditional image enhancement method, the second is the image enhancement method based on deep learning, and the third is the improved method based on the classic histogram equalization and Retinex theory. However, these methods have the following defects:

[0077] Most traditional image enhancement methods are based on histogram equalization to enhance the brightness and contrast of low-light images. However, histogram equalization has problems such as grayscale overlap caused by merging grayscale levels, loss of local image details, obvious image block effects, and blurred image background and target contours. The other two methods have problems such as large computational complexity and long response time, which makes them unsuitable for application scenarios with high timeliness requirements, such as endoscopy scenarios.

[0078] Therefore, in order to at least solve the quality problem of images in low-light environments in the related art, the present invention provides an image processing method, which obtains a brightness image based on the HSV image of the original RGB image, then obtains a binary image and a brightness-enhanced image based on the brightness image, and then obtains the boundary pixel points on the boundary line between the dark area and the bright area in the brightness-enhanced image based on the binary image, which can improve the boundary recognition accuracy of the dark area and the bright area. Since the positioning of the boundary pixel points can be achieved based on the binary image, the amount of data processing can also be reduced. Then, the corresponding set of points to be corrected is obtained based on the boundary pixel points, and the set of points to be corrected includes boundary pixel points, and pixel points located on the dark area side and the bright area side of the boundary pixel points, respectively, and these pixel points are adjacent to each other and are on the same straight line. It can be seen that all the pixel points in the set of points to be corrected are the points that best represent the brightness difference between the dark area and the bright area. Therefore, the brightness values ​​of the pixels in the point set to be corrected are continuously updated based on the brightness values ​​of all the pixels in the point set to be corrected, so that the brightness of the pixels after brightness correction has higher uniformity, and then the corresponding target point set is obtained. It can be seen that the brightness of the point that best represents the brightness difference between the dark area and the bright area has been smoothed, which not only can achieve a smooth transition of the brightness between the dark area and the bright area, but also can achieve an improvement in the brightness of the dark area, so that the target HSV image after the V channel data is updated based on the target point set also has the same technical effect, and then the target RGB image obtained based on the conversion of the target HSV image not only has a good brightness improvement effect in the dark area, but also the overall image has a good image color restoration effect. In addition, since it is only necessary to process the V channel data (i.e., the brightness image) in the HSV image, there is no need to process the image data of the H channel and the S channel, and the image processing does not involve a large amount of computing operations, any of the above-mentioned solutions provided by the embodiments of the present invention also has the beneficial technical effects of high image processing efficiency, the playback speed of the image sequence meets the video playback requirements (30 frames / s, and the processing time for each frame of the image is within 33ms), there will be no freezes, and good clinical use effects.

[0079] The image processing method provided by the present invention can be applied to electronic devices. Figure 1 , is a block diagram of the structure of an electronic device. The electronic device 100 includes a memory 110, a processor 120 and a communication module 130. The memory 110, the processor 120 and the communication module 130 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0080] The memory is used to store programs or data. The memory may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc.

[0081] The processor is used to read / write data or programs stored in the memory and execute corresponding functions.

[0082] The communication module is used to establish a communication connection between the electronic device and other communication terminals through a network, and is used to send and receive data through the network.

[0083] It should be understood that Figure 1 The structure shown is only a schematic diagram of the structure of the electronic device, and the electronic device may also include Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0084] In some embodiments, the electronic device may be an image processing module, which may be sold as a separate product or may be used as a processing module in an image device having both image acquisition and image processing functions to execute the image processing method provided in an embodiment of the present invention to improve image quality.

[0085] The following combination Figure 2 The image processing method provided by the embodiment of the present invention is described. Figure 2 is a flowchart of an image processing method provided by an embodiment of the present invention, the image processing method comprising:

[0086] In step S100, a brightness image is obtained according to the HSV image of the original RGB image;

[0087] In step S200, a binary image is obtained according to the negative image of the brightness image;

[0088] In step S300, a brightness enhanced image is obtained according to the brightness image;

[0089] In step S400, the boundary pixel points at the boundary between the bright area and the dark area in the brightness enhanced image are obtained according to the binary image;

[0090] In step S500, for each boundary pixel point, a set of points to be corrected corresponding to the boundary pixel point is obtained from the brightness enhanced image; the set of points to be corrected includes the boundary pixel point, and pixel points located at the dark area side and the bright area side of the boundary pixel point respectively; all pixel points in the set of points to be corrected are adjacent to each other and are on the same straight line;

[0091] In step S600, for each point set to be corrected, the brightness values ​​of the pixels in the point set to be corrected are updated according to the brightness values ​​of all the pixels in the point set to be corrected, so as to obtain a target point set corresponding to the point set to be corrected; the brightness uniformity of all the pixels in the target point set is higher than the brightness uniformity of all the pixels in the point set to be corrected;

[0092] In step S700, the brightness of the pixel points at the corresponding positions in the brightness enhanced image is updated according to all target point sets to obtain a target brightness image;

[0093] In step S800, according to the target brightness image, the V channel data in the HSV image is updated to obtain a target HSV image;

[0094] In step S900, a target RGB image is obtained according to the target HSV image.

[0095] In a low-light environment, the image processing method provided by the embodiment of the present invention can be used to perform brightness enhancement processing on the original RGB image collected by the image acquisition module. After the processing is completed, the processed target RGB image is displayed on the display module for the user to view. For example, in an endoscope application scenario, the image processing method provided by the embodiment of the present invention can be used to process the image collected by the endoscope in real time, and the processed target RGB image is output to the display module in real time, so as to realize the smooth display of the image sequence collected by the endoscope.

[0096] The execution subject of the image processing method provided by the embodiment of the present invention can start executing the above steps S100 to S900 when receiving the original RGB image. In this case, all received original RGB images are processed by default. Even if there is no low-light area in the original RGB image, the processed original RGB image can have better quality, as described above, which will not be repeated here.

[0097] Alternatively, when it is determined that quality improvement processing is required based on the original RGB image, for example, when the original RGB image is received, the brightness value of the original RGB image can be obtained first. If it is found that the difference between the lowest brightness value and the highest brightness value is greater than the set difference, or the lowest brightness value is lower than the set brightness value, then the above steps S100 to S900 are started again. Because in this case, the quality of the original RGB image is at a good level, further quality improvement is not required, which can reduce the number of images that need to be processed to a certain extent, and can make the image output more immediate.

[0098] In the above, the original RGB image can be acquired by the image acquisition module and sent to the above execution subject, or sent to the execution subject by the server, or obtained after the user selects from the image library of the execution subject. The embodiment of the present invention does not limit the source of the original RGB image. In addition, the setting difference value and the setting brightness value can be set according to experience or experiment, which will not be described in detail here.

[0099] After obtaining the original RGB image, step S100 is executed to convert the original RGB image into an HSV image, and the conversion principle thereof can be referred to in the related art. Then, the image data of the V channel can be extracted from the HSV image to obtain a brightness image, and the extraction principle thereof can also be referred to in the related art. It should be understood that the image data is usually stored in the form of a matrix at the bottom layer.

[0100] After obtaining the brightness image, the corresponding binary image and brightness enhanced image can be obtained according to the brightness image processing, wherein the acquisition of the binary image and the acquisition of the brightness enhanced image can be performed serially or in parallel, that is, the execution order of step S200 and step S300 is not limited.

[0101] In the process of executing step S200, in order to more accurately distinguish the dark area and the bright area in the original RGB image, the original brightness value of the brightness image can be inverted first, that is, 255-the original brightness value of each pixel is used to obtain the corresponding negative image. Then, the negative image is binarized, for example, a brightness threshold is pre-set, and the brightness value of the pixel points with brightness values ​​greater than the brightness threshold in the negative image is set to 255, and the brightness value of the pixel points with brightness values ​​less than the brightness threshold is set to 0. After completing the brightness value processing of all pixels in the negative image, the corresponding binary image can be obtained. The brightness threshold can be set based on experience or experiments.

[0102] However, in order to obtain a better binarization effect and to improve the accuracy of distinguishing dark areas and brightness, in some embodiments, the image processing method provided by the embodiment of the present invention further provides another solution for obtaining a binarized image of a negative image, that is, in the above step S200, the step of obtaining a binary image according to the negative image of the brightness image may include:

[0103] In step S210, the negative image is subjected to a power operation according to a set index to obtain a negative image after the power operation;

[0104] In step S220, a brightness threshold is obtained according to the brightness value of the original RGB image;

[0105] In step S230, the brightness values ​​of pixels whose brightness values ​​are greater than or equal to the brightness threshold in the negative image after the power operation are updated to 255, and the brightness values ​​of pixels whose brightness values ​​are less than the brightness threshold are updated to 0, so as to obtain a corresponding binary image.

[0106] The process of obtaining a binary image based on the negative image processing through steps S210 to S230 is:

[0107] First, step S210 and step S220 are respectively performed to obtain a negative image and a brightness threshold with better effect. It can be seen that the execution order of step S210 and step S220 is not limited, and the two can be executed in series or in parallel.

[0108] In the process of executing step S210, the negative image may be firstly subjected to power operation according to the set index, so that the brightness value of each pixel in the negative image has better difference, thereby facilitating the subsequent distinction between dark and bright areas in the binary image to have higher accuracy. In some examples, in order to obtain a better dark and bright area distinction effect, the set index may have a value range of (1, 2), for example, the set index may be 1.6, which may achieve a better binary image processing effect, so as to facilitate the subsequent distinction between dark and bright areas in the original RGB image based on the binary image. Based on this, the 1.6th power of the brightness value of each pixel in the negative image may be calculated, thereby obtaining the negative image after power operation.

[0109] In the process of executing step S220, a brightness threshold value that is more in line with the actual brightness of the original RGB image can be calculated based on the brightness value of the original RGB image, so that the dark area and the bright area of ​​the binary image obtained subsequently have higher consistency with the dark area and brightness of the original RGB image, respectively. The brightness mean value or the brightness median value of the original RGB image can be used as the brightness threshold value in step S200.

[0110] In the case where the dark area and the bright area of ​​the binary image are more consistent with the dark area and the brightness of the original RGB image, respectively, in order to make the distinction between the dark area and the bright area easier and more accurate, in some embodiments, the image processing method provided by the embodiment of the present invention further provides another brightness threshold acquisition scheme, that is, in the above step S220, the step of obtaining the brightness threshold according to the brightness value of the original RGB image may include:

[0111] In step S221, a brightness reference value of the original RGB image is calculated according to the brightness value of the original RGB image; the brightness reference value is the brightness mean value or the brightness median value of the original RGB image;

[0112] In step S222, the product of the brightness reference value and the set brightness coefficient is calculated to obtain the brightness threshold; wherein the value range of the set brightness coefficient is [0.7, 1.1].

[0113] It can be understood that in the process of obtaining the brightness threshold, step S221 can be executed first to calculate a brightness reference value that can be used to characterize the brightness of the original RGB image based on the brightness value of each pixel in the original RGB image. The brightness reference value can be the brightness mean or the brightness median.

[0114] After obtaining the brightness reference value, step S222 can be performed to calculate the product of the brightness reference value and the set brightness coefficient, and the product is used as the brightness threshold. The value range of the set brightness coefficient is [0.7, 1.1]. On this basis, in order to obtain a brightness condition that can better reflect the original RGB image and enable the subsequent binary image to have a better dark area and light area distinction function, in some examples, the value of the set brightness coefficient is 0.9.

[0115] After obtaining the negative image and brightness threshold after the power operation through any of the above embodiments, step S230 can be executed to set the brightness values ​​of pixels whose brightness values ​​are greater than or equal to the brightness threshold in the negative image after the power operation to 255, and set the brightness values ​​of pixels whose brightness values ​​are less than the brightness threshold to 0, so as to obtain the corresponding binary image.

[0116] In the process of executing step S300, in order to obtain a brightness enhanced image after brightness enhancement, the brightness value of each pixel in the brightness image can be multiplied by the set enhancement coefficient to obtain a new brightness value and update it, so as to obtain the brightness enhanced image. The set enhancement coefficient is a value greater than 1, which can be set according to actual experience or experiments. In other variant embodiments, the multiplication process can also be changed to a power operation process, as long as the purpose of achieving the brightness of the brightness enhanced image is higher than that of the brightness image.

[0117] However, in order to improve the brightness of the dark area, avoid using the same improvement amplitude to improve the brightness of the bright area, which may cause overexposure of the bright area after the brightness improvement, and make the subsequently obtained target RGB image have better color balance, in some embodiments, the image processing method provided by the embodiment of the present invention further provides another brightness enhanced image acquisition solution, that is, in the above step S300, the step of obtaining the brightness enhanced image according to the brightness image may include:

[0118] In step S310, a brightness transformation matrix is ​​calculated according to a set brightness enhancement matrix;

[0119] In step S320, a power operation is performed on the brightness image according to the brightness transformation matrix to obtain a brightness enhanced image.

[0120] It can be understood that in the process of obtaining the brightness enhanced image, step S310 can be performed first to calculate the brightness transformation matrix used to enhance the overall brightness of the brightness image according to the set brightness enhancement matrix, and the brightness enhancement range of the dark area is higher than the brightness enhancement range of the bright area. The number of rows and columns of the brightness transformation matrix is ​​the same as the number of rows and columns of the brightness image, and in the brightness transformation matrix, the value corresponding to the position of each pixel in the dark area is greater than the value corresponding to the position of each pixel in the bright area. In the brightness transformation matrix, the value corresponding to the position of each pixel in the bright area can be in the range of [0, 1], and the values ​​at these positions can be the same or partially the same, or they can be different from each other; and the value corresponding to the position of each pixel in the dark area can be greater than 1, which can be set according to actual needs or experience. Similarly, the values ​​at these positions can be the same or partially the same, or they can be different from each other.

[0121] In some embodiments, in order to obtain a more reasonable brightness transformation matrix, so as to better brighten the brightness image as a whole and achieve better brightening effects for dark areas and brightness, the image processing method provided by the embodiment of the present invention further provides another brightness transformation matrix acquisition scheme, that is, in the above step S310, the step of calculating the brightness transformation matrix according to the set brightness enhancement matrix may include:

[0122] In step S311, a first intermediate coefficient matrix and a second intermediate coefficient matrix are calculated based on the brightness enhancement matrix by using a first calculation formula and a second calculation formula respectively;

[0123] In step S312, a brightness transformation matrix is ​​calculated based on the first intermediate coefficient matrix and the second intermediate coefficient matrix using a third calculation formula.

[0124] Among them, the first calculation formula is:

[0125] matMidCoef=1.05+(dLumEnhance / 113.0-0.03);

[0126] The second calculation formula is:

[0127] matBrightenCoef=(dLumEnhance / 113.0-0.03) / 3.0;

[0128] The third calculation formula is:

[0129] matPower=1.0 / (matMidCoef×(1-matBrightenCoef));

[0130] In the above, matMidCoef represents the first intermediate coefficient matrix; dLumEnhance represents the brightness enhancement matrix, and the value range of each value in the brightness enhancement matrix is ​​(1, 150); matBrightenCoef represents the second intermediate coefficient matrix; matPower represents the brightness transformation matrix. In addition, the number of rows and columns of the above matrices is the same as that of the brightness image.

[0131] After obtaining the brightness transformation matrix through any of the embodiments in step S310, step S320 may be performed to perform power operation on the brightness image according to the brightness transformation matrix to obtain a brightness enhanced image. For example, the brightness transformation matrix may be used as an exponent of a matrix corresponding to the brightness image for operation, and the matrix obtained by the operation is the brightness enhanced image.

[0132] In some embodiments, in order to improve the numerical stability in the brightness enhanced image calculation process and the accuracy of the brightness enhanced image to a certain extent, the image processing method provided by the embodiment of the present invention also provides another brightness enhanced image acquisition scheme, that is, in the above step S320, the step of performing power operation processing on the brightness image according to the brightness transformation matrix to obtain the brightness enhanced image may include:

[0133] In step S321, a brightness enhanced image is calculated based on the brightness transformation matrix and the brightness image by using a fourth calculation formula; in the brightness enhanced image, the brightness increase range of the dark area is greater than the brightness increase range of the bright area.

[0134] The fourth calculation formula is:

[0135] matVBrighten=255*(matV / 255.0)^matPower;

[0136] Among them, matVBrighten represents the brightness enhanced image, matV represents the brightness image, and matPower represents the brightness transformation matrix.

[0137] It can be seen from the fourth calculation formula that the brightness image is first normalized, and then the normalized brightness image is subjected to power operation according to the brightness transformation matrix as an exponent, and then the matrix obtained by the power operation is restored to the scale before regression to obtain a brightness enhanced image. The normalization process can significantly improve the stability and accuracy of the numerical value, thereby improving the brightness accuracy in the brightness enhanced image.

[0138] After obtaining the brightness enhanced image through any of the embodiments in step S300, step S400 may be performed to obtain the boundary pixel points at the boundary of the bright area and the dark area in the brightness enhanced image through the binary image. For example, since the binary image is generated based on the negative image of the brightness image, in the binary image, the position where the value 0 is located represents a point with relatively bright brightness in the original image, and the point belongs to the bright area, while the position where the value 255 is located represents a point with relatively dark brightness in the original image, and the point belongs to the dark area. Therefore, the area formed by continuous values ​​0 can be regarded as the bright area, and the area formed by continuous values ​​255 can be regarded as the dark area. In this way, if the values ​​0 or 255 are not always continuous, there may be multiple dark areas and / or multiple bright areas. Based on this, the edge of each dark area can be used as the boundary line between the corresponding dark area and the bright area. Since the position of each pixel point in the binary image corresponds one-to-one to the position of each pixel point in the brightness enhanced image, when the boundary line between the dark area and the bright area in the binary image is determined, the pixel point at the corresponding position in the brightness enhanced image can be obtained according to the position of the boundary line in the binary image, thereby obtaining the boundary pixel point in step S400.

[0139] In some embodiments, in the case of reducing the amount of image data processing, dark areas with relatively small areas may not be processed, because dark areas with relatively small areas generally have a relatively small impact on the overall quality of the image, so certain simplified operations can be performed. Based on this, the image processing method provided by the embodiment of the present invention also provides another solution for obtaining boundary pixel points, that is, in the above step S400, the step of obtaining boundary pixel points at the boundary between the bright area and the dark area in the brightness enhanced image according to the binary image may include:

[0140] In step S410, the area of ​​each dark region in the binary image is calculated;

[0141] In step S420, corresponding boundary pixel points are obtained from the brightness enhanced image according to the image positions of edge pixel points of dark areas whose areas are larger than a set area threshold.

[0142] In the process of obtaining the boundary pixel points, step S410 can be performed first to calculate the area of ​​each dark area in the binary image. The area calculation principle can be found in the relevant technology. Then step S420 is performed to screen the dark areas according to the area to obtain the dark areas whose area is greater than the set area threshold. The screened dark areas will be applied to the subsequent update of the brightness value. Therefore, for these screened dark areas, the corresponding boundary pixel points will be further obtained from the brightness enhanced image. The unselected dark areas can be ignored and do not need to participate in the subsequent update of the brightness value.

[0143] After obtaining the boundary pixel points through any of the embodiments in step S400, step S500 can be executed to obtain, for each boundary pixel point, a set of points to be corrected corresponding to the boundary pixel point from the brightness enhanced image, wherein one set of points to be corrected includes one boundary pixel point, and pixel points located on the dark area side and the bright area side of the boundary pixel point, respectively, and all pixel points in one set of points to be corrected are adjacent to each other and are on the same straight line.

[0144] It can be understood that all the pixels of a point set to be corrected are in the same column or row. In the process of obtaining the point set to be corrected, the boundary pixel points can be included in the correction point set starting from the boundary pixel points, and then the first set number of dark area pixel points can be obtained from the dark area side of the boundary pixel points, starting from the boundary pixel points next to the boundary pixel points. For example, assuming that the left side of the boundary pixel points is the dark area side, then on the row where the boundary pixel points are located, the dark area pixel points are gradually selected from the pixel points on the left side of the boundary pixel points and adjacent to the boundary pixel points, and included in the point set to be corrected until the total number of dark area pixel points reaches the first set number. Based on the same principle, the second set number of bright area pixel points can be obtained from the bright area side of the boundary pixel points.

[0145] In the above, the first set number and the second set number can be the same or different. In addition, the values ​​of the first set number and the second set number can be configured according to actual needs or experience. In some examples, in order to reduce the amount of calculation to a certain extent, while ensuring a smooth transition between the dark area and the bright area, and making the dark area and the bright area have a better balance, the first set number and the second set number can be set to 4. Based on this example, a set of points to be corrected includes a total of 9 pixels: 1 boundary pixel, 4 dark area pixels, and 4 bright area pixels.

[0146] In the above, the area threshold may be set according to actual needs or experience. For example, the area threshold is 10,000 pixels, but is not limited thereto.

[0147] After obtaining the point set to be corrected by any of the embodiments in step S500, step S600 may be performed to correct the brightness of all or part of the pixels in the point set to be corrected, so that the brightness uniformity of all the pixels after the brightness correction is greater than the brightness uniformity of all the pixels before the correction. The correction principle may refer to the relevant technology.

[0148] In some embodiments, in order to obtain a target RGB image with smoother brightness transition and better color restoration effect, the image processing method provided by the embodiment of the present invention further provides another brightness correction scheme, that is, in the above step S500, the step of updating the brightness values ​​of the pixels in the point set to be corrected according to the brightness values ​​of all the pixels in the point set to be corrected may include:

[0149] In step S510, a brightness reference value is calculated based on the brightness values ​​of all pixels in the set of points to be corrected; the brightness reference value is the brightness mean or brightness median of all pixels in the set of points to be corrected;

[0150] In step S520, the brightness values ​​of the pixels in the set of points to be corrected, whose brightness values ​​are higher than the brightness reference value, are lowered;

[0151] In step S530, the brightness values ​​of the pixels in the to-be-corrected point set whose brightness values ​​are lower than the brightness reference value are increased.

[0152] It can be understood that in the brightness correction scheme shown in steps S510 to S530, step S510 can be executed first, and each point set to be corrected can be processed separately to obtain the brightness reference value corresponding to each point set to be corrected. Among them, each point set to be corrected uses the brightness mean or median of all the pixels contained in it as the brightness reference value. In this way, in the subsequent brightness correction, correction can be made based on the median or mean brightness, and both the median and mean brightness can reflect the brightness and brightness level of the original RGB image, so it can be ensured that the correction result can not only improve the quality of the dark area, but also maintain the brightness characteristics of the original RGB image, and can better restore the original color.

[0153] After obtaining the brightness reference value of each point set to be corrected through step S510, step S520 and step S530 may be executed in series or in parallel. It can be seen that the embodiment of the present invention does not limit the execution order of step S520 and step S530.

[0154] In the process of executing step S520 and step S530, taking a point set to be corrected as an example, all pixel points in the point set to be corrected can be compared with the brightness reference value respectively to obtain pixel points with brightness lower than the brightness reference value and pixel points with brightness higher than the brightness reference value. Then, the brightness of the pixel points with brightness lower than the brightness reference value is increased, and the increase range can be set according to actual needs or experience. Similarly, the brightness of the pixel points with brightness higher than the brightness reference value is reduced, and the reduction range can also be set according to actual needs or experience. Among them, the brightness increase range of the pixel points with brightness lower than the brightness reference value can be the same or different; similarly, the brightness reduction range of the pixel points with brightness higher than the brightness reference value can be the same or different. For the convenience of description hereinafter, the brightness increase range and the brightness reduction range are collectively referred to as the brightness adjustment range.

[0155] In some embodiments, in order to ensure that the brightness transition of the pixels whose brightness is adjusted is natural and to make the boundary transition between the dark area and the bright area more natural and smooth, in the process of adjusting the pixels whose brightness needs to be adjusted in the point set to be corrected, for each pixel whose brightness is adjusted, the brightness adjustment amplitude of the pixel whose brightness is adjusted is negatively correlated with the distance between the pixel whose brightness is adjusted and the corresponding boundary pixel.

[0156] For example, suppose a point set to be corrected includes 9 pixels numbered 1 to 9, and the arrangement order of these 9 pixels from the dark area to the bright area is the same as their numbers, namely: pixel 1, pixel 2, pixel 3, pixel 4, pixel 5, pixel 6, pixel 7, pixel 8, and pixel 9. Among them, the boundary pixel is the middle point, that is, pixel 5; the pixels whose brightness needs to be adjusted are pixel 1, pixel 3, pixel 4, pixel 6, and pixel 8.

[0157] Based on this, it can be seen that pixel 4 and pixel 6 are both the points closest to pixel 5, followed by pixel 3, then pixel 8, and finally pixel 1. Therefore, based on the principle that the brightness adjustment amplitude of a pixel is negatively correlated with the distance between the pixel and the corresponding boundary pixel, the closer the pixel is to the boundary pixel, the greater its brightness adjustment amplitude, and the farther the pixel is from the boundary pixel, the smaller its brightness adjustment amplitude. It can be seen that in the above example, the brightness adjustment amplitude of pixel 4 and pixel 6 is the largest. For example, 10% of the brightness reference value V is used as the maximum brightness adjustment amplitude. Assuming that the brightness of pixel 4 and pixel 6 are A and B respectively, where A is less than the brightness reference value and B is greater than the brightness reference value, the updated brightness value of pixel 4 is: A+10%V, and the updated brightness value of pixel 6 is B-10%V. The distance between pixel 3 and pixel 5 is 1 pixel, and the pixel distance is 1. If 2% is used as the attenuation amplitude, the brightness adjustment amplitude of pixel 3 is: (10%-n×2%)V=8%V, where n is the pixel distance. Similarly, the brightness adjustment amplitudes of pixel 8 and pixel 1, as well as the updated brightness values, can be calculated based on the above principle.

[0158] After the brightness adjustment of all pixels whose brightness needs to be adjusted is completed through any embodiment in step S600, the target point set corresponding to each point set to be corrected can be obtained. Then, step S700 can be executed. During the execution of step S700, the brightness values ​​of all pixels included in all target point sets can be replaced with the brightness values ​​of the pixels at the corresponding positions in the brightness enhanced image, thereby obtaining the target brightness image.

[0159] After obtaining the target brightness image, step S800 can be executed to keep the H channel data and the S channel data in the HSV image unchanged, and update the V channel data to the data corresponding to the target brightness image, that is, replace the original brightness image in the HSV image with the target brightness image, thereby obtaining the target HSV image.

[0160] After obtaining the target HSV image, step S900 may be performed to convert the target HSV image into a target RGB image. The conversion principle may be found in related art.

[0161] Therefore, in the target RGB image obtained by any of the above embodiments, the brightness of the dark area can be greatly improved, and the clarity and color reproduction of the overall image have good effects, such as Figure 3 As shown, Figure 3 It is a schematic diagram comparing the effects of an original RGB image and a target RGB image provided by an embodiment of the present invention. It can be seen that the target RGB image M1 has better image quality than the original RGB image M0.

[0162] After obtaining the target RGB image, in some embodiments, the target RGB image can also be output to a display module for the user to view. For example, in the field of endoscopy applications, the processed target RGB image sequence can be output to a display module for the doctor to observe and make relevant diagnoses. It is worth noting that the technical features or technical solutions in any of the above embodiments of the present invention can be combined with each other as long as there is no combination contradiction.

[0163] In addition, an embodiment of the present invention further provides an endoscope, see Figure 4 , Figure 4 4 is a schematic diagram of an endoscope provided in an embodiment of the present invention, and the endoscope 400 includes:

[0164] An image acquisition device 410 is used to acquire original RGB images;

[0165] The image acquisition device 410 may be located at the tip of the insertion tube 440 of the endoscope operating handle 450;

[0166] An image processing device 420, configured to execute the image processing method provided in any of the above embodiments, so as to obtain a target RGB image based on the original RGB image;

[0167] And a display device 430, used for displaying the target RGB image.

[0168] The endoscope provided in the embodiment of the present invention is used as an illustration. It can be understood that the endoscope may also include Figure 4 More or fewer components as shown, or with Figure 4 Different configurations are shown.

[0169] In order to execute the corresponding steps in the above embodiments and various possible methods, an implementation method of an image processing device is given below. Optionally, the image processing device can adopt the above Figure 1 For further information, please refer to Figure 5 , Figure 5 This is a functional module diagram of an image processing device provided by an embodiment of the present invention. It should be noted that the basic principle and technical effects of the image processing device provided by this embodiment are the same as those of the above embodiments. For the sake of brief description, for parts not mentioned in this embodiment, reference can be made to the corresponding contents in the above embodiments. The image processing device 500 includes:

[0170] The acquisition module 510 is configured to: obtain a brightness image according to the HSV image of the original RGB image;

[0171] The binarization processing module 520 is configured to: obtain a binary image according to the negative image of the brightness image;

[0172] The brightness enhancement module 530 is configured to: obtain a brightness enhanced image according to the brightness image;

[0173] The boundary determination module 540 is configured to: obtain boundary pixel points at the boundary between the bright area and the dark area in the brightness enhanced image according to the binary image; for each boundary pixel point, obtain a set of points to be corrected corresponding to the boundary pixel point from the brightness enhanced image; the set of points to be corrected includes the boundary pixel point, and pixel points located on the dark area side and the bright area side of the boundary pixel point respectively; all pixel points in the set of points to be corrected are adjacent to each other and are on the same straight line;

[0174] The correction module 550 is configured to: for each point set to be corrected, update the brightness values ​​of the pixels in the point set to be corrected according to the brightness values ​​of all the pixels in the point set to be corrected, so as to obtain a target point set corresponding to the point set to be corrected; the brightness uniformity of all the pixels in the target point set is higher than the brightness uniformity of all the pixels in the point set to be corrected; according to all the target point sets, update the brightness of the pixels at the corresponding positions in the brightness enhanced image to obtain a target brightness image; according to the target brightness image, update the V channel data in the HSV image to obtain a target HSV image;

[0175] The image conversion module 560 is configured to obtain a target RGB image according to the target HSV image.

[0176] In some embodiments, the process of obtaining the binary image according to the negative image of the brightness image by the binarization processing module 520 is configured as follows:

[0177] Performing a power operation on the negative image according to a set index to obtain a negative image after the power operation;

[0178] Obtaining a brightness threshold according to the brightness value of the original RGB image;

[0179] In the negative image after the power operation, the brightness values ​​of the pixels whose brightness values ​​are greater than or equal to the brightness threshold are updated to 255, and the brightness values ​​of the pixels whose brightness values ​​are less than the brightness threshold are updated to 0, so as to obtain the corresponding binary image.

[0180] In some embodiments, the value range of the setting index is (1, 2).

[0181] In some embodiments, the process of obtaining the brightness threshold value according to the brightness value of the original RGB image by the binarization processing module 520 is configured as follows:

[0182] According to the brightness value of the original RGB image, a brightness reference value of the original RGB image is calculated; the brightness reference value is the brightness mean value or the brightness median value of the original RGB image;

[0183] The product of the brightness reference value and the set brightness coefficient is calculated to obtain the brightness threshold; wherein the value range of the set brightness coefficient is [0.7, 1.1].

[0184] In some embodiments, the set index value is 1.6, and / or the set brightness coefficient value is 0.9.

[0185] In some embodiments, the process of obtaining the brightness enhanced image according to the brightness image by the brightness enhancement module 530 is configured as follows:

[0186] According to the set brightness enhancement matrix, the brightness transformation matrix is ​​calculated;

[0187] The brightness image is subjected to power operation processing according to the brightness transformation matrix to obtain a brightness enhanced image.

[0188] In some embodiments, the process of the brightness enhancement module 530 calculating the brightness transformation matrix according to the set brightness enhancement matrix is ​​configured as follows:

[0189] By using a first calculation formula and a second calculation formula, respectively based on the brightness enhancement matrix, a first intermediate coefficient matrix and a second intermediate coefficient matrix are calculated;

[0190] Calculate a brightness transformation matrix based on the first intermediate coefficient matrix and the second intermediate coefficient matrix using a third calculation formula;

[0191] The first calculation formula is:

[0192] matMidCoef=1.05+(dLumEnhance / 113.0-0.03);

[0193] The second calculation formula is:

[0194] matBrightenCoef=(dLumEnhance / 113.0-0.03) / 3.0;

[0195] The third calculation formula is:

[0196] matPower=1.0 / (matMidCoef×(1-matBrightenCoef));

[0197] Among them, matMidCoef represents the first intermediate coefficient matrix, dLumEnhance represents the brightness enhancement matrix, matBrightenCoef represents the second intermediate coefficient matrix, and matPower represents the brightness transformation matrix.

[0198] In some embodiments, the process in which the brightness enhancement module 530 performs a power operation on the brightness image according to the brightness transformation matrix to obtain the brightness enhanced image is configured as follows:

[0199] A brightness enhanced image is obtained by calculation based on the brightness transformation matrix and the brightness image through a fourth calculation formula; in the brightness enhanced image, the brightness increase range of the dark area is greater than the brightness increase range of the bright area;

[0200] The fourth calculation formula is:

[0201] matVBrighten=255*(matV / 255.0)^matPower;

[0202] Among them, matVBrighten represents the brightness enhanced image, matV represents the brightness image, and matPower represents the brightness transformation matrix.

[0203] In some embodiments, the process of obtaining the boundary pixel points at the boundary between the bright area and the dark area in the brightness enhanced image according to the binary image by the boundary determination module 540 is configured as follows:

[0204] Calculating the area of ​​each dark region in the binary image;

[0205] According to the image position of the edge pixel point of the dark area whose area is larger than the set area threshold, the corresponding boundary pixel point is obtained from the brightness enhanced image.

[0206] In some embodiments, the process of updating the brightness values ​​of the pixels in the to-be-corrected point set by the correction module 550 according to the brightness values ​​of all the pixels in the to-be-corrected point set is configured as follows:

[0207] A brightness reference value is calculated based on the brightness values ​​of all the pixels in the set of points to be corrected; the brightness reference value is the brightness mean or brightness median of all the pixels in the set of points to be corrected;

[0208] Lowering the brightness value of the pixel points in the to-be-corrected point set whose brightness value is higher than the brightness reference value;

[0209] Increase the brightness value of the pixel points in the to-be-corrected point set whose brightness value is lower than the brightness reference value.

[0210] In some embodiments, for each pixel point with adjusted brightness in the to-be-corrected point set, the brightness adjustment amplitude of the pixel point with adjusted brightness is negatively correlated with the distance between the pixel point with adjusted brightness and the corresponding boundary pixel point.

[0211] Optionally, the above modules can be stored in the form of software or firmware. Figure 1 The memory shown in the figure or solidified in the operating system (OS) of the electronic device, and can be Figure 1 Meanwhile, the data and program codes required for executing the above modules can be stored in the memory.

[0212] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0213] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0214] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0215] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An image processing method, characterized in that: include: According to the HSV image of the original RGB image, the brightness image is obtained; Obtaining a binary image according to the negative image of the brightness image; Obtaining a brightness enhanced image according to the brightness image; According to the binary image, a boundary pixel point at the boundary between a bright area and a dark area in the brightness enhanced image is obtained; For each boundary pixel point, a set of points to be corrected corresponding to the boundary pixel point is obtained from the brightness enhanced image; the set of points to be corrected includes the boundary pixel point and pixel points located on the dark area side and the bright area side of the boundary pixel point respectively; all the pixel points in the set of points to be corrected are adjacent to each other and are on the same straight line; For each point set to be corrected, the brightness values ​​of the pixels in the point set to be corrected are updated according to the brightness values ​​of all the pixels in the point set to be corrected, so as to obtain a target point set corresponding to the point set to be corrected; the brightness uniformity of all the pixels in the target point set is higher than the brightness uniformity of all the pixels in the point set to be corrected; According to all target point sets, the brightness of the pixel points at the corresponding positions in the brightness enhanced image is updated to obtain a target brightness image; According to the target brightness image, update the V channel data in the HSV image to obtain a target HSV image; A target RGB image is obtained according to the target HSV image.

2. The method according to claim 1, characterized in that: The step of obtaining a binary image according to the negative image of the brightness image comprises: Performing a power operation on the negative image according to a set index to obtain a negative image after the power operation; Obtaining a brightness threshold according to the brightness value of the original RGB image; In the negative image after the power operation, the brightness values ​​of the pixels whose brightness values ​​are greater than or equal to the brightness threshold are updated to 255, and the brightness values ​​of the pixels whose brightness values ​​are less than the brightness threshold are updated to 0, so as to obtain the corresponding binary image.

3. The method according to claim 2, characterized in that The value range of the setting index is (1, 2); and / or The step of obtaining a brightness threshold according to the brightness value of the original RGB image comprises: According to the brightness value of the original RGB image, a brightness reference value of the original RGB image is calculated; the brightness reference value is the brightness mean value or the brightness median value of the original RGB image; The product of the brightness reference value and the set brightness coefficient is calculated to obtain the brightness threshold; wherein the value range of the set brightness coefficient is [0.7, 1.1].

4. The method according to claim 3, characterized in that: The set index value is 1.6, and / or the set brightness coefficient value is 0.

9.

5. The method according to claim 1, characterized in that The step of obtaining a brightness enhanced image according to the brightness image comprises: According to the set brightness enhancement matrix, the brightness transformation matrix is ​​calculated; The brightness image is subjected to power operation processing according to the brightness transformation matrix to obtain a brightness enhanced image.

6. The method according to claim 5, characterized in that The step of calculating the brightness transformation matrix according to the set brightness enhancement matrix includes: By using a first calculation formula and a second calculation formula, respectively based on the brightness enhancement matrix, a first intermediate coefficient matrix and a second intermediate coefficient matrix are calculated; Calculate a brightness transformation matrix based on the first intermediate coefficient matrix and the second intermediate coefficient matrix using a third calculation formula; The first calculation formula is: matMidCoef=1.05+(dLumEnhance / 113.0-0.03); The second calculation formula is: matBrightenCoef=(dLumEnhance / 113.0-0.03) / 3.0; The third calculation formula is: matPower=1.0 / (matMidCoef×(1-matBrightenCoef)); Among them, matMidCoef represents the first intermediate coefficient matrix, dLumEnhance represents the brightness enhancement matrix, matBrightenCoef represents the second intermediate coefficient matrix, and matPower represents the brightness transformation matrix.

7. The method according to claim 5, characterized in that The step of performing a power operation on the brightness image according to the brightness transformation matrix to obtain a brightness enhanced image comprises: A brightness enhanced image is obtained by calculation based on the brightness transformation matrix and the brightness image through a fourth calculation formula; in the brightness enhanced image, the brightness improvement range of the dark area is greater than the brightness improvement range of the bright area; The fourth calculation formula is: matVBrighten=255*(matV / 255.0)^matPower; Among them, matVBrighten represents the brightness enhanced image, matV represents the brightness image, and matPower represents the brightness transformation matrix.

8. The method according to claim 1, characterized in that The step of obtaining boundary pixel points at the boundary between the bright area and the dark area in the brightness enhanced image according to the binary image comprises: Calculating the area of ​​each dark region in the binary image; According to the image position of the edge pixel point of the dark area whose area is larger than the set area threshold, the corresponding boundary pixel point is obtained from the brightness enhanced image.

9. The method according to claim 1, characterized in that: The step of updating the brightness values ​​of the pixels in the to-be-corrected point set according to the brightness values ​​of all the pixels in the to-be-corrected point set comprises: A brightness reference value is calculated based on the brightness values ​​of all the pixels in the set of points to be corrected; the brightness reference value is the brightness mean or brightness median of all the pixels in the set of points to be corrected; Lowering the brightness value of the pixel points in the to-be-corrected point set whose brightness value is higher than the brightness reference value; Increase the brightness value of the pixel points in the to-be-corrected point set whose brightness value is lower than the brightness reference value.

10. The method according to claim 9, characterized in that For each pixel point with adjusted brightness in the to-be-corrected point set, the brightness adjustment amplitude of the pixel point with adjusted brightness is negatively correlated with the distance between the pixel point with adjusted brightness and the corresponding boundary pixel point.

11. An image processing device, characterized in that: include: The acquisition module is configured to: obtain a brightness image according to the HSV image of the original RGB image; A binarization processing module is configured to: obtain a binary image according to a negative image of the brightness image; A brightness enhancement module is configured to: obtain a brightness enhanced image according to the brightness image; The boundary determination module is configured to: obtain boundary pixel points at the boundary between the bright area and the dark area in the brightness enhanced image according to the binary image; for each boundary pixel point, obtain a set of points to be corrected corresponding to the boundary pixel point from the brightness enhanced image; the set of points to be corrected includes the boundary pixel point, and pixel points located on the dark area side and the bright area side of the boundary pixel point respectively; all pixel points in the set of points to be corrected are adjacent to each other and are on the same straight line; The correction module is configured to: for each point set to be corrected, update the brightness values ​​of the pixels in the point set to be corrected according to the brightness values ​​of all the pixels in the point set to be corrected, so as to obtain a target point set corresponding to the point set to be corrected; the brightness uniformity of all the pixels in the target point set is higher than the brightness uniformity of all the pixels in the point set to be corrected; According to all target point sets, the brightness of the pixel points at the corresponding positions in the brightness enhanced image is updated to obtain a target brightness image; according to the target brightness image, the V channel data in the HSV image is updated to obtain a target HSV image; The image conversion module is configured to obtain a target RGB image according to the target HSV image.

12. An endoscope, characterized in that: include: Image acquisition device, used for acquiring original RGB images; An image processing device, configured to execute the method according to any one of claims 1 to 10, so as to obtain a target RGB image based on the original RGB image; and a display device for displaying the target RGB image.

13. An electronic device, characterized in that: The invention comprises a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor can execute the machine executable instructions to implement the method according to any one of claims 1 to 10.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images