Method, device and computer equipment for removing smoke from endoscope images

Abstract

The present application relates to a method, device and computer equipment for smoke removal of endoscopic images, wherein the method for smoke removal of endoscopic images comprises: estimating smoke noise of an input image to obtain a first set of noise points corresponding to the input image; the first set of noise points comprises a plurality of noise points of interest in the input image; dividing each noise point of interest into a plurality of noise point combinations, generating a denoising convolution kernel corresponding to each noise point combination; further, determining a target image region corresponding to the noise point combination in the input image, and denoising the target image region through the denoising convolution kernel. Through the present application, the problem that it is impossible to accurately remove smoke from different smoke regions in an image, resulting in reduced image quality in normal imaging regions, is solved, and accurate smoke removal is achieved for different smoke regions in an image, improving smoke removal efficiency while avoiding affecting image quality in normal imaging regions.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device and computer equipment for removing smoke from endoscopic images. Background Art

[0002] Endoscopes are important equipment used in minimally invasive medical treatment of the human body. By inserting the endoscope into organs or cavities inside the human body, images of the parts to be examined are collected and endoscopic images are output to provide high-quality in vivo image information. However, for endoscopic surgeries involving cutting of human tissues, a large amount of smoke is inevitably generated to interfere with the doctor's surgical field of view. Therefore, it is necessary to identify and remove image smoke to improve the quality of endoscopic images.

[0003] In the existing smoke removal method, the original endoscope image is obtained from the lumen of the target object. When smoke exists in the original image, the original image is defogged according to the preset relationship between the fog-free image and the original image. However, the above method performs defogging based on the entire image and cannot accurately remove smoke from different smoke areas in the image, resulting in reduced image quality in the normal imaging area.

[0004] There is no effective solution to the problem in the related art that it is impossible to accurately remove smoke from different smoke areas in an image, resulting in reduced image quality in normal imaging areas. Summary of the invention

[0005] In this embodiment, a method, apparatus and computer device for removing smoke from an endoscopic image are provided to solve the problem in the related art that it is impossible to accurately remove smoke from different smoke areas in an image, resulting in reduced image quality in normal imaging areas.

[0006] In a first aspect, a method for removing smoke from an endoscope image is provided in this embodiment, the method comprising:

[0007] Performing smoke noise estimation on an input image to obtain a first noise point set corresponding to the input image; the first noise point set includes a plurality of noise points of interest in the input image;

[0008] Divide each of the noise points of interest into a plurality of noise point combinations, and generate a denoising convolution kernel corresponding to each of the noise point combinations;

[0009] A target image region corresponding to the noise point combination in the input image is determined, and denoising is performed on the target image region using the denoising convolution kernel.

[0010] In some embodiments, the step of estimating smoke noise on an input image to obtain a first noise point set corresponding to the input image includes:

[0011] Traversing each pixel in the input image, and determining a grayscale difference characteristic value between the pixel and all adjacent pixels;

[0012] When the grayscale difference characteristic value corresponding to the pixel point is greater than a preset threshold, the pixel point is regarded as the noise point of interest;

[0013] Based on each of the noise points of interest, a corresponding first noise point set is generated.

[0014] In some of the embodiments, after determining the grayscale difference characteristic values ​​between the pixel and all adjacent pixels, the method further includes:

[0015] Determine the grayscale mean of each pixel in the input image;

[0016] Based on a preset weight parameter and the grayscale mean, a preset threshold corresponding to the grayscale difference characteristic value is determined.

[0017] In some of the embodiments, after the smoke noise estimation is performed on the input image to obtain the first noise point set corresponding to the input image, the method further includes:

[0018] Traversing each of the noise points of interest, and determining another noise point of interest that is closest to the noise point of interest;

[0019] When the distance between the noise point of interest and the other noise point of interest is less than a preset threshold, suppressing the noise point of interest and the other noise point of interest;

[0020] The midpoint between the noise point of interest and the another noise point of interest is taken as a new noise point of interest.

[0021] In some embodiments, dividing each of the noise points of interest into a plurality of noise point combinations and generating a denoising convolution kernel corresponding to each of the noise point combinations includes:

[0022] Divide every two of the noise points of interest that are closest to each other into a group to obtain a plurality of noise point combinations;

[0023] Based on the two noise points of interest in each noise point combination, a matrix corresponding to the noise point combination is generated as the denoising convolution kernel.

[0024] In some embodiments, generating a matrix corresponding to the noise point combination as the denoising convolution kernel based on two of the noise points of interest in each noise point combination includes:

[0025] Taking the connecting line between two noise points of interest in the noise point combination as the diagonal line, constructing a matrix corresponding to the noise point combination as the denoising convolution kernel;

[0026] Based on a preset scaling parameter, a preset variance parameter and the position coordinates of each pixel point in the matrix, the value of each element in the denoising convolution kernel is determined.

[0027] In some embodiments, dividing each of the noise points of interest into a plurality of noise point combinations and generating a denoising convolution kernel corresponding to each of the noise point combinations includes:

[0028] Performing clustering processing on each of the noise points of interest to obtain corresponding combinations of multiple noise points;

[0029] Generate a denoising convolution kernel corresponding to each noise point combination.

[0030] In a second aspect, in this embodiment, a smoke removal device for an endoscope image is provided, the device comprising: a detection module, a generation module and a denoising module;

[0031] The detection module is used to estimate smoke noise on an input image to obtain a first noise point set corresponding to the input image; the first noise point set includes a plurality of noise points of interest in the input image;

[0032] The generating module is used to divide each of the noise points of interest into a plurality of noise point combinations, and generate a denoising convolution kernel corresponding to each of the noise point combinations;

[0033] The denoising module is used to determine a target image region in the input image corresponding to the noise point combination, and perform denoising processing on the target image region through the denoising convolution kernel.

[0034] In a third aspect, a computer device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for removing smoke from an endoscopic image as described in the first aspect above is implemented.

[0035] In a fourth aspect, in this embodiment, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method for removing smoke from an endoscopic image described in the first aspect above is implemented.

[0036] Compared with the related art, the smoke removal method, device and computer equipment for endoscopic images provided in the present embodiment estimate the smoke noise of the input image to obtain a first noise point set corresponding to the input image; the first noise point set includes multiple noise points of interest in the input image; each noise point of interest is divided into multiple noise point combinations, and a denoising convolution kernel corresponding to each noise point combination is generated; further, a target image area corresponding to the noise point combination in the input image is determined, and the target image area is denoised by the denoising convolution kernel, which solves the problem that it is impossible to accurately remove smoke from different smoke areas in the image, resulting in reduced image quality in the normal imaging area, and realizes accurate smoke removal for different smoke areas in the image, improves the smoke removal efficiency, and avoids affecting the image quality of the normal imaging area.

[0037] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0039] Figure 1 is a flow chart of a method for removing smoke from an endoscopic image provided by an embodiment of the present application;

[0040] Figure 2 is a flow chart of a method for estimating smoke noise of an endoscopic image provided by an embodiment of the present application;

[0041] Figure 3 is a flow chart of a method for removing smoke from an endoscopic image provided by a preferred embodiment of the present application;

[0042] Figure 4 It is a structural block diagram of a smoke removal device for endoscopic images provided in one embodiment of the present application.

[0043] In the figure: 10, detection module; 20, generation module; 30, denoising module. DETAILED DESCRIPTION

[0044] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0045] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with general skills in the technical field to which this application belongs. The words "one", "a", "the", "these" and the like in this application do not indicate a quantitative limitation, and they may be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone. Generally, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0046] In this embodiment, a method for removing smoke from an endoscope image is provided. Figure 1 is a flow chart of the smoke removal method for an endoscope image of this embodiment, as shown in Figure 1 As shown, the process includes the following steps:

[0047] Step S110, performing smoke noise estimation on the input image to obtain a first noise point set corresponding to the input image; the first noise point set includes a plurality of noise points of interest in the input image;

[0048] Step S120, dividing each noise point of interest into a plurality of noise point combinations, and generating a denoising convolution kernel corresponding to each noise point combination;

[0049] Step S130, determining a target image region corresponding to the noise point combination in the input image, and performing denoising processing on the target image region by using a denoising convolution kernel.

[0050] Specifically, the input endoscopic image is subjected to smoke noise estimation, each pixel in the input image is traversed, and the grayscale difference eigenvalue between the current pixel and all adjacent pixels is determined. When it is detected that the grayscale difference eigenvalue is greater than a preset threshold, the current pixel is taken as a noise point of interest, and a first noise point set containing each noise point of interest is generated.

[0051] It should be noted that the smoke noise in this embodiment may be linear smoke generated by cutting or burning operations during endoscopic surgery, linear oil mist caused by sprayed grease, etc.

[0052] Further, each noise point of interest in the first noise point set is traversed to determine another noise point of interest that is closest to the noise point of interest, and to determine the distance between the noise point of interest and another noise point of interest. When it is detected that the distance between the noise point of interest and another noise point of interest is less than a preset threshold, the noise point of interest and the other noise point of interest are suppressed, that is, two noise points whose distance does not reach the preset threshold are eliminated from the first noise point set, and the midpoint between the current noise point of interest and another noise point of interest is used as a new noise point of interest, so as to accurately suppress the noise points of interest with repeated responses, thereby removing redundant and overlapping noise point detection results. The suppression processing algorithm in this embodiment includes but is not limited to a non-maximum suppression algorithm, a threshold suppression algorithm, and a dynamic threshold suppression algorithm.

[0053] Afterwards, all the currently existing noise points of interest are divided to obtain multiple noise point combinations. Each noise point combination represents a local noise area, that is, the target image area corresponding to the noise point combination in the input image. A denoising convolution kernel corresponding to each noise point combination is generated, and the corresponding target image area is denoised by the denoising convolution kernel, so as to accurately remove smoke from different smoke areas in the image, improve the efficiency of smoke removal, and avoid affecting the image quality of the normal imaging area.

[0054] Among them, various division methods can be used to process each noise point of interest, for example, every two noise points of interest with the closest distance among the noise points of interest are divided into a group to obtain multiple noise point combinations; or, each noise point of interest is clustered to obtain corresponding multiple noise point combinations, etc.

[0055] In the existing smoke removal method, the original endoscope image is obtained from the lumen of the target object. When smoke exists in the original image, the original image is defogged according to the preset relationship between the fog-free image and the original image. However, the above method performs defogging based on the entire image and cannot accurately remove smoke from different smoke areas in the image, resulting in reduced image quality in the normal imaging area.

[0056] Compared with the prior art, the present application estimates smoke noise on the input image to obtain a first noise point set corresponding to the input image; the first noise point set includes multiple noise points of interest in the input image; each noise point of interest is divided into multiple noise point combinations, and a denoising convolution kernel corresponding to each noise point combination is constructed; further, the target image area corresponding to the noise point combination in the input image is determined, and the target image area is denoised by the denoising convolution kernel. Based on this, the image is divided into multiple local image areas with noise by noise point grouping, so that the denoising convolution kernel adapted to each local image area can be used to denoise the area, and while accurately denoising, the image will not be degraded in the normal imaging area, which solves the problem that it is impossible to accurately remove smoke from different smoke areas in the image, resulting in reduced image quality in the normal imaging area, and achieves accurate smoke removal for different smoke areas in the image, improves smoke removal efficiency, and avoids affecting the image quality of the normal imaging area.

[0057] In some of the embodiments, Figure 2 As shown, the step S110 of estimating smoke noise on the input image to obtain a first noise point set corresponding to the input image includes the following steps:

[0058] Step S111, traversing each pixel in the input image, and determining the grayscale difference feature value between the pixel and all adjacent pixels;

[0059] Step S112, when the grayscale difference characteristic value corresponding to the pixel point is greater than a preset threshold, the pixel point is regarded as a noise point of interest;

[0060] Step S113: generating a corresponding first noise point set based on each noise point of interest.

[0061] Specifically, traverse each pixel in the input image to determine the grayscale difference eigenvalue between each pixel and all its adjacent pixels, where the adjacent pixels are pixels directly adjacent to the current pixel. The grayscale value variance between each pixel and all its adjacent pixels can be calculated to obtain the grayscale difference eigenvalue; or, the calculation formula in formula (1) can be used to determine the grayscale difference eigenvalue between each pixel and all its adjacent pixels. The specific formula is as follows:

[0062]

[0063] In formula (1), G ij represents the grayscale difference feature value between the current pixel and all its adjacent pixels; p ij Indicates the gray value of the current pixel; q k Represents the grayscale value of adjacent pixels; N represents the total number of adjacent pixels.

[0064] Furthermore, it is determined whether the grayscale difference characteristic value corresponding to the current pixel point is greater than a preset threshold. If the grayscale difference characteristic value is greater than the preset threshold, the pixel point is taken as a noise point of interest, and a first noise point set is generated based on the screened noise points of interest.

[0065] It should be noted that when estimating noise points in the input image, in addition to using grayscale difference eigenvalues ​​for detection, edge detection, texture detection and other algorithms can also be used to process the input image to obtain noise points of interest in the input image and achieve accurate estimation of the smoke noise area.

[0066] Through this embodiment, each pixel point in the input image is traversed, and the grayscale difference eigenvalue between the pixel point and all adjacent pixels is determined. When the grayscale difference eigenvalue corresponding to the pixel point is greater than a preset threshold, the pixel point is regarded as a noise point of interest, and a corresponding first noise point set is generated based on each noise point of interest, so as to detect and obtain areas in the image where smoke noise may exist, achieve accurate estimation of the smoke noise area, and facilitate subsequent precise denoising.

[0067] In some of the embodiments, after determining the grayscale difference characteristic values ​​between the pixel and all adjacent pixels, the following steps are further included:

[0068] Determine the grayscale mean of each pixel in the input image;

[0069] Based on the preset weight parameter and the grayscale mean, a preset threshold corresponding to the grayscale difference characteristic value is determined.

[0070] Specifically, the gray value of each pixel in the input image is obtained, the gray value of each pixel is averaged, and the gray mean of each pixel is obtained. Based on the preset weight parameter and the calculated gray mean, a preset threshold corresponding to the gray difference feature value is set. The specific calculation formula is as follows:

[0071]

[0072] In formula (2), α is the preset threshold corresponding to the grayscale difference feature value; w is the preset weight parameter; I is the grayscale mean of each pixel in the input image.

[0073] It should be noted that, in addition to setting the threshold according to the calculation method of formula (2), a fixed threshold may be set according to prior knowledge, or the grayscale mean of the neighborhood of the current pixel may be calculated and used as the threshold. This embodiment does not limit the method for setting the threshold.

[0074] Through this embodiment, the grayscale mean of each pixel in the input image is determined, and based on the preset weight parameter and the grayscale mean, the preset threshold corresponding to the grayscale difference characteristic value is determined, so that the overall brightness of the input image can be used to accurately set the threshold for judging the size of the grayscale difference, thereby improving the accuracy of subsequent detection of noise points of interest.

[0075] In some of the embodiments, after performing smoke noise estimation on the input image to obtain a first noise point set corresponding to the input image, the following steps are further included:

[0076] Traverse each noise point of interest and determine another noise point of interest that is closest to the noise point of interest;

[0077] When the distance between the noise point of interest and another noise point of interest is less than a preset threshold, suppressing the noise point of interest and the other noise point of interest;

[0078] The midpoint between the noise point of interest and another noise point of interest is taken as a new noise point of interest.

[0079] Specifically, all current noise points of interest are traversed, another noise point of interest that is closest to the current noise point of interest is determined, and the distance between the noise point of interest and the other noise point of interest is calculated. In this embodiment, the distance calculation method between noise points of interest includes but is not limited to Euclidean distance and Manhattan distance.

[0080] Further, it is determined whether the distance between two noise points of interest is less than a preset threshold. If the distance is less than the preset threshold, the current noise point of interest and another noise point of interest are both suppressed, that is, the two noise points whose distance does not reach the preset threshold are eliminated from the first noise point set, and the midpoint between the current noise point of interest and another noise point of interest is used as a new noise point of interest, and the new noise point of interest is added to the first noise point set; if the distance between the noise point of interest and another noise point of interest is greater than or equal to the preset threshold, the current two noise points of interest are retained, that is, the two noise points whose distance reaches the preset threshold are retained in the first noise point set. After all noise points of interest are subjected to the above non-maximum suppression process, the first noise point set is updated to obtain the corresponding second noise point set, and the subsequent image processing process is performed based on the noise points of interest in the second noise point set.

[0081] Through this embodiment, each noise point of interest is traversed to determine another noise point of interest that is closest to the noise point of interest. When the distance between the noise point of interest and another noise point of interest is less than a preset threshold, the noise point of interest and the other noise point of interest are suppressed, and the midpoint between the noise point of interest and the other noise point of interest is used as a new noise point of interest. In this way, the noise points of interest with repeated responses are accurately suppressed, the redundant responses of the noise points are effectively reduced, and the accuracy of subsequent image processing is improved.

[0082] In some embodiments, the step S120 of dividing each noise point of interest into a plurality of noise point combinations and generating a denoising convolution kernel corresponding to each noise point combination includes the following steps:

[0083] Divide every two noise points of interest that are closest to each other into a group to obtain multiple noise point combinations;

[0084] Based on the two noise points of interest in each noise point combination, a matrix corresponding to the noise point combination is generated as a denoising convolution kernel.

[0085] Specifically, all currently existing noise points of interest are divided to obtain multiple noise point combinations, each noise point combination represents a local noise area, that is, the target image area corresponding to the noise point combination in the input image. Among them, various methods can be used to divide and process each noise point of interest, for example, every two noise points of interest with the closest distance among each noise point of interest are divided into a group to obtain multiple noise point combinations; or, each noise point of interest is clustered to obtain corresponding multiple noise point combinations, etc.

[0086] Furthermore, based on the positions of the two noise points of interest in each noise point combination, a matrix containing the two noise points of interest in the current noise point combination is constructed as a denoising convolution kernel, and the element value corresponding to each pixel position in the denoising convolution kernel is calculated.

[0087] Through this embodiment, every two noise points of interest that are closest to each other are divided into a group to obtain multiple noise point combinations. Based on the two noise points of interest in each noise point combination, a matrix corresponding to the noise point combination is generated as a denoising convolution kernel, so that the area where there is noise in the image is divided into multiple local image areas, and a denoising convolution kernel adapted to each local image area is obtained, so as to facilitate the subsequent use of the denoising convolution kernel to achieve accurate smoke removal.

[0088] In some of the embodiments, based on two noise points of interest in each noise point combination, a matrix corresponding to the noise point combination is generated as a denoising convolution kernel, including the following steps:

[0089] Taking the connecting line between two noise points of interest in the noise point combination as the diagonal line, the matrix corresponding to the noise point combination is constructed as the denoising convolution kernel;

[0090] Based on the preset scaling parameter, the preset variance parameter and the position coordinates of each pixel point in the matrix, the value of each element in the denoising convolution kernel is determined.

[0091] Specifically, for each noise point combination, the connecting line between two noise points of interest in the noise point combination is used as the diagonal line, and a matrix corresponding to the noise point combination is constructed as a denoising convolution kernel. For example, the two noise points of interest in the noise point combination are used as the upper left corner point and the lower right corner point of the matrix respectively.

[0092] Furthermore, the position coordinates of each pixel in the matrix are obtained, and the values ​​of each element in the denoising convolution kernel are calculated according to the preset scaling parameter, the preset variance parameter and the position coordinates of each pixel in the matrix. The specific calculation formula is as follows:

[0093]

[0094] In formula (3), (s, t) represents the position coordinates of the pixel point; k(s, t) represents the element value of the pixel point; K represents the preset scaling parameter; σ represents the preset variance parameter. The preset scaling parameter and variance parameter are associated with the denoising strength.

[0095] Through this embodiment, the connecting line between two noise points of interest in the noise point combination is used as the diagonal line, and the matrix corresponding to the noise point combination is constructed as the denoising convolution kernel. Based on the preset scaling parameter, the preset variance parameter and the position coordinates of each pixel point in the matrix, the value of each element in the denoising convolution kernel is determined to realize the construction of the denoising convolution kernel. Different denoising convolution kernels are used to extract features of different scales and frequencies to remove noise, so that the denoising strength of the denoising convolution kernel is more adapted to the corresponding noise area, which helps to improve the image denoising effect.

[0096] In some embodiments, the step S120 of dividing each noise point of interest into a plurality of noise point combinations and generating a denoising convolution kernel corresponding to each noise point combination includes the following steps:

[0097] Perform clustering on each noise point of interest to obtain corresponding multiple noise point combinations;

[0098] Generate a denoising convolution kernel corresponding to each noise point combination.

[0099] Specifically, all the current noise points of interest are clustered, such as the K-means clustering algorithm, the density-based spatial clustering algorithm (Density-Based Spatial Clustering of Applications with Noise, DBSCAN), etc., to form multiple noise point clusters. Each noise point cluster is regarded as a noise point combination, and each noise point combination represents a local noise area, that is, the target image area corresponding to the noise point combination in the input image.

[0100] Furthermore, a matrix corresponding to each noise point combination is constructed as a denoising convolution kernel, and the range of the matrix covers the area where the current noise point combination is located. Based on the characteristic information of each noise point of interest in the noise point combination, such as the position and intensity of the noise point of interest, the values ​​of each element corresponding to each pixel position in the denoising convolution kernel are calculated; or, according to the preset scaling parameter, the preset variance parameter and the position coordinates of each pixel point in the matrix, the values ​​of each element in the denoising convolution kernel are calculated.

[0101] Through this embodiment, each noise point of interest is clustered to obtain corresponding multiple noise point combinations, and a denoising convolution kernel corresponding to each noise point combination is generated to realize the construction of the denoising convolution kernel, so that the denoising strength of the denoising convolution kernel is more adapted to the corresponding noise area, which helps to improve the image denoising effect.

[0102] The present embodiment is described and illustrated below through preferred embodiments.

[0103] Figure 3 is a flow chart of the smoke removal method for an endoscope image of the preferred embodiment, as shown in Figure 3 As shown, the smoke removal method of the endoscope image includes the following steps:

[0104] Step S310, traversing each pixel in the input image, and determining the grayscale difference feature value between the pixel and all adjacent pixels;

[0105] Step S320, when the grayscale difference characteristic value corresponding to the pixel point is greater than a preset threshold, the pixel point is regarded as a noise point of interest, and a corresponding first noise point set is generated based on each noise point of interest;

[0106] Step S330, traversing each noise point of interest, and determining another noise point of interest that is closest to the noise point of interest;

[0107] Step S340, when the distance between the noise point of interest and another noise point of interest is less than a preset threshold, suppressing the noise point of interest and another noise point of interest;

[0108] Step S350, taking the midpoint between the noise point of interest and another noise point of interest as a new noise point of interest, and updating the first noise point set based on the new noise point of interest to obtain a second noise point set;

[0109] Step S360, in the second noise point set, every two noise points of interest that are closest to each other are grouped into one group, to obtain a plurality of noise point combinations;

[0110] Step S370, taking the connecting line between two noise points of interest in the noise point combination as the diagonal line, constructing a matrix corresponding to the noise point combination as a denoising convolution kernel;

[0111] Step S380, determining the value of each element in the denoising convolution kernel based on the preset scaling parameter, the preset variance parameter and the position coordinates of each pixel point in the matrix;

[0112] Step S390, determining a target image region corresponding to the noise point combination in the input image, and performing denoising processing on the target image region by using a denoising convolution kernel.

[0113] Through this embodiment, each pixel point in the input image is traversed, and the grayscale difference characteristic value between the pixel point and all adjacent pixels is determined. When the grayscale difference characteristic value corresponding to the pixel point is greater than a preset threshold, the pixel point is taken as a noise point of interest, and a corresponding first noise point set is generated based on each noise point of interest. Afterwards, each noise point of interest is traversed to determine another noise point of interest that is closest to the noise point of interest. When the distance between the noise point of interest and another noise point of interest is less than a preset threshold, the noise point of interest and the other noise point of interest are suppressed, and the midpoint between the noise point of interest and the other noise point of interest is taken as a new noise point of interest. Based on the new noise point of interest, the first noise point set is updated to obtain a second noise point set, thereby effectively reducing the redundant response of noise points and helping to improve the accuracy of subsequent image processing.

[0114] Furthermore, in the second noise point set, every two noise points of interest with the closest distance among the noise points of interest are divided into a group to obtain a plurality of noise point combinations. The connecting line between two noise points of interest in the noise point combination is taken as the diagonal line, and the matrix corresponding to the noise point combination is constructed as the denoising convolution kernel. Based on the preset scaling parameter, the preset variance parameter and the position coordinates of each pixel point in the matrix, the values ​​of each element in the denoising convolution kernel are determined, and then the target image area corresponding to the noise point combination in the input image is determined. The target image area is denoised by the denoising convolution kernel, which solves the problem that the smoke cannot be accurately removed from different smoke areas in the image, resulting in a reduction in the image quality of the normal imaging area, and realizes accurate smoke removal for different smoke areas in the image, improves the smoke removal efficiency, and avoids affecting the image quality of the normal imaging area.

[0115] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0116] In this embodiment, a smoke removal device for an endoscopic image is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and will not be repeated hereafter. The terms "module", "unit", "subunit", etc. used below may implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0117] Figure 4 : is a structural block diagram of the smoke removal device for endoscope images of this embodiment, as shown in Figure 4 As shown, the device includes: a detection module 10, a generation module 20 and a denoising module 30;

[0118] The detection module 10 is used to estimate smoke noise on the input image to obtain a first noise point set corresponding to the input image; the first noise point set includes a plurality of noise points of interest in the input image;

[0119] A generating module 20, used to divide each noise point of interest into a plurality of noise point combinations, and generate a denoising convolution kernel corresponding to each noise point combination;

[0120] The denoising module 30 is used to determine the target image area corresponding to the noise point combination in the input image, and perform denoising processing on the target image area through a denoising convolution kernel.

[0121] Through the device provided in this embodiment, smoke noise is estimated for an input image to obtain a first noise point set corresponding to the input image; the first noise point set includes a plurality of noise points of interest in the input image; each noise point of interest is divided into a plurality of noise point combinations, and a denoising convolution kernel corresponding to each noise point combination is constructed; further, a target image area corresponding to the noise point combination in the input image is determined, and the target image area is denoised through the denoising convolution kernel, thereby solving the problem that accurate smoke removal cannot be performed for different smoke areas in the image, resulting in reduced image quality in the normal imaging area, and achieving accurate smoke removal for different smoke areas in the image, improving smoke removal efficiency, and avoiding affecting the image quality of the normal imaging area.

[0122] In some of the embodiments, the detection module 10 is also used to traverse each pixel point in the input image to determine the grayscale difference characteristic value between the pixel point and all adjacent pixels; when the grayscale difference characteristic value corresponding to the pixel point is greater than a preset threshold, the pixel point is regarded as a noise point of interest; based on each noise point of interest, a corresponding first noise point set is generated.

[0123] In some of the embodiments, the detection module 10 is further used to determine the grayscale mean of each pixel in the input image; based on the preset weight parameter and the grayscale mean, determine a preset threshold corresponding to the grayscale difference feature value.

[0124] In some of these embodiments, Figure 4 On the basis of, the device also includes a suppression module, which is used to traverse each noise point of interest and determine another noise point of interest that is closest to the noise point of interest; when the distance between the noise point of interest and another noise point of interest is less than a preset threshold, the noise point of interest and the other noise point of interest are suppressed; and the midpoint between the noise point of interest and another noise point of interest is used as a new noise point of interest.

[0125] In some of the embodiments, the generating module 20 is further used to divide every two noise points of interest that are closest to each other among the noise points of interest into a group to obtain multiple noise point combinations; based on the two noise points of interest in each noise point combination, a matrix corresponding to the noise point combination is generated as a denoising convolution kernel.

[0126] In some of the embodiments, the generating module 20 is also used to construct a matrix corresponding to the noise point combination as a denoising convolution kernel with the connecting line between two noise points of interest in the noise point combination as the diagonal line; and determine the value of each element in the denoising convolution kernel based on a preset scaling parameter, a preset variance parameter and the position coordinates of each pixel point in the matrix.

[0127] In some of the embodiments, the generating module 20 is further used to perform clustering processing on each noise point of interest to obtain a corresponding plurality of noise point combinations; and to generate a denoising convolution kernel corresponding to each noise point combination.

[0128] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0129] In this embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0130] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0131] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0132] S1, performing smoke noise estimation on an input image to obtain a first noise point set corresponding to the input image; the first noise point set includes a plurality of noise points of interest in the input image;

[0133] S2, divide each noise point of interest into multiple noise point combinations, and generate a denoising convolution kernel corresponding to each noise point combination;

[0134] S3, determining a target image region corresponding to the noise point combination in the input image, and performing denoising processing on the target image region by using a denoising convolution kernel.

[0135] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0136] In addition, in combination with the smoke removal method for endoscopic images provided in the above embodiments, a storage medium may be provided in this embodiment to implement the smoke removal method for endoscopic images. The storage medium stores a computer program, and when the computer program is executed by a processor, any of the smoke removal methods for endoscopic images in the above embodiments is implemented.

[0137] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0138] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0139] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0140] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A method for removing smoke from an endoscope image, characterized in that: The method comprises: Performing smoke noise estimation on an input image to obtain a first noise point set corresponding to the input image; the first noise point set includes a plurality of noise points of interest in the input image; Divide each of the noise points of interest into a plurality of noise point combinations, and generate a denoising convolution kernel corresponding to each of the noise point combinations; Determine a target image region in the input image corresponding to the noise point combination, and perform denoising processing on the target image region through the denoising convolution kernel; wherein each element value in the denoising convolution kernel corresponds one-to-one to a pixel point position in the target image region.

2. The method for removing smoke from an endoscope image according to claim 1, characterized in that: The step of estimating smoke noise on the input image to obtain a first noise point set corresponding to the input image includes: Traversing each pixel in the input image, and determining a grayscale difference characteristic value between the pixel and all adjacent pixels; When the grayscale difference characteristic value corresponding to the pixel point is greater than a preset threshold, the pixel point is regarded as the noise point of interest; Based on each of the noise points of interest, a corresponding first noise point set is generated.

3. The method for removing smoke from an endoscope image according to claim 2, characterized in that: After determining the grayscale difference characteristic values ​​between the pixel point and all adjacent pixel points, the method further includes: Determine the grayscale mean of each pixel in the input image; Based on a preset weight parameter and the grayscale mean, a preset threshold corresponding to the grayscale difference characteristic value is determined.

4. The method for removing smoke from an endoscope image according to claim 1, characterized in that: After the smoke noise is estimated for the input image to obtain a first noise point set corresponding to the input image, the method further includes: Traversing each of the noise points of interest, and determining another noise point of interest that is closest to the noise point of interest; When the distance between the noise point of interest and the other noise point of interest is less than a preset threshold, suppressing the noise point of interest and the other noise point of interest; The midpoint between the noise point of interest and the another noise point of interest is taken as a new noise point of interest.

5. The method for removing smoke from an endoscope image according to claim 1, characterized in that: The step of dividing each of the noise points of interest into a plurality of noise point combinations and generating a denoising convolution kernel corresponding to each of the noise point combinations comprises: Divide every two of the noise points of interest that are closest to each other into a group to obtain a plurality of noise point combinations; Based on the two noise points of interest in each noise point combination, a matrix corresponding to the noise point combination is generated as the denoising convolution kernel.

6. The method for removing smoke from an endoscope image according to claim 5, characterized in that: The step of generating a matrix corresponding to the noise point combination as the denoising convolution kernel based on the two noise points of interest in each noise point combination includes: Taking the connecting line between two noise points of interest in the noise point combination as the diagonal line, constructing a matrix corresponding to the noise point combination as the denoising convolution kernel; Based on a preset scaling parameter, a preset variance parameter and the position coordinates of each pixel point in the matrix, the value of each element in the denoising convolution kernel is determined.

7. The method for removing smoke from an endoscope image according to claim 1, characterized in that: The step of dividing each of the noise points of interest into a plurality of noise point combinations and generating a denoising convolution kernel corresponding to each of the noise point combinations comprises: Performing clustering processing on each of the noise points of interest to obtain corresponding combinations of multiple noise points; Generate a denoising convolution kernel corresponding to each noise point combination.

8. A smoke removal device for endoscope images, characterized in that: The device comprises: a detection module, a generation module and a denoising module; The detection module is used to estimate smoke noise on an input image to obtain a first noise point set corresponding to the input image; the first noise point set includes a plurality of noise points of interest in the input image; The generating module is used to divide each of the noise points of interest into a plurality of noise point combinations, and generate a denoising convolution kernel corresponding to each of the noise point combinations; The denoising module is used to determine the target image area corresponding to the noise point combination in the input image, and to perform denoising processing on the target image area through the denoising convolution kernel; wherein each element value in the denoising convolution kernel corresponds one-to-one to the pixel point position in the target image area.

9. A computer device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the steps of the method for removing smoke from an endoscopic image according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for removing smoke from an endoscopic image according to any one of claims 1 to 7 are implemented.

Images