A method and apparatus for removing stripe noise

Abstract

This application provides a method and apparatus for removing stripe noise, thereby improving the stripe noise removal effect. The method includes: filtering an image to be processed through two filtering windows of different sizes to obtain first high-frequency data, second high-frequency data, a first weight value, and a second weight value corresponding to each pixel; for each column of pixels, determining a first noise amplitude of that column of pixels using the first high-frequency data and the first weight value; segmenting each column of pixels according to the first weight value and the second weight value, and determining a second noise amplitude corresponding to each pixel segment based on the second high-frequency data and the second weight value; and performing denoising processing on that column of pixels in the image to be processed based on the first noise amplitude corresponding to each column of pixels and multiple second noise amplitudes.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and apparatus for removing stripe noise. Background Technology

[0002] Due to the limitations of its manufacturing process, infrared detectors exhibit significant non-uniformity among pixels in their focal plane array. Stripe noise, a specific type of non-uniformity, is primarily caused by the use of different amplification circuits in detector units across different rows or columns on the infrared focal plane. Differences in readout circuit voltages ultimately result in stripes of varying brightness on the infrared image, severely impacting image quality. Existing technologies address relatively prominent, narrow stripes, but do not consider removing wider stripes with less brightness difference from the background, thus resulting in insufficient stripe removal performance. Furthermore, existing stripe removal methods are prone to creating inverted stripes in scenes with partially obscured surfaces and vertically edged objects, such as buildings and trees, in areas that were originally stripe-free. Summary of the Invention

[0003] This application provides a method and apparatus for removing stripe noise, which improves the stripe noise removal effect.

[0004] In a first aspect, embodiments of this application provide a method for removing stripe noise, comprising:

[0005] For the j-th column of pixels in the image to be processed, if the first number of noise points in the j-th column of pixels is greater than the first noise number threshold, then the first noise amplitude corresponding to the j-th column of pixels is determined based on the first high-frequency data of the noise points in the j-th column of pixels, wherein the first high-frequency data of the noise points is determined after the image to be processed is filtered by the first filtering window;

[0006] For any pixel segment of the j-th column of the image to be processed, if the second number of noise points in the arbitrary pixel segment is greater than the second noise number threshold corresponding to the arbitrary pixel segment, then the second noise amplitude corresponding to the arbitrary pixel segment is determined based on the second high-frequency data of the noise points in the arbitrary pixel segment, wherein the j-th column of pixels includes multiple pixel segments, and the second high-frequency data of the noise points is determined after the image to be processed is filtered by the second filtering window, and the width of the first filtering window is smaller than the width of the second filtering window;

[0007] The j-th column of pixels in the image to be processed is denoised based on the first noise amplitude and multiple second noise amplitudes corresponding to the j-th column of pixels.

[0008] Based on the above scheme, this application extracts noise through two windows of different sizes, and then determines different noise amplitudes to remove stripe noise. This method can remove both fine and wide stripes. Furthermore, by segmenting each column of pixels in the image and determining the stripe noise amplitude for each pixel segment, different noise amplitudes are used to process the image. This ensures effective stripe removal without introducing reverse stripes, thus improving the stripe noise removal effect.

[0009] In one possible implementation, the first high-frequency data is determined by: filtering the image to be processed through the first filtering window to obtain the first low-frequency data corresponding to each pixel in the image to be processed; for any pixel, subtracting the pixel value of the arbitrary pixel in the image to be processed from the first low-frequency data of the arbitrary pixel to obtain the first high-frequency data corresponding to the arbitrary pixel.

[0010] The second high-frequency data is determined by: filtering the image to be processed through the second filtering window to obtain the second low-frequency data corresponding to each pixel in the image to be processed; for any pixel, subtracting the pixel value of the pixel in the image to be processed from the second low-frequency data of the corresponding pixel to obtain the second high-frequency data corresponding to the pixel.

[0011] In one possible implementation, the j-th column of pixels refers to the column of pixels in the image to be processed that have adjacent left and right columns of pixels; noise points in the j-th column of pixels are determined as follows:

[0012] Determine a first difference between the first weight value of the i-th pixel and the first weight value of its left neighbor, and a second difference between the first weight value of the i-th pixel and the first weight value of its right neighbor, wherein the i-th pixel is any pixel in the j-th column; when both the first difference and the second difference are less than a first preset threshold and the first weight value of the i-th pixel is greater than a second preset threshold, determine the i-th pixel as a noise point; and / or,

[0013] Noise points in a pixel segment are determined as follows: a third difference is determined between the second weight value of the s-th pixel and the second weight value of its left neighboring pixel, and a fourth difference is determined between the second weight value of the s-th pixel and the second weight value of its right neighboring pixel, wherein the s-th pixel is any pixel in the pixel segment; when both the third difference and the fourth difference are less than a third preset threshold and the second weight value of the s-th pixel is greater than a fourth preset threshold, the s-th pixel is determined to be a noise point in the pixel segment; wherein the first weight value and the second weight value are used to characterize the pixel differences between adjacent pixels.

[0014] Based on the above scheme, in the stripe noise extraction process, multiple thresholds (weight threshold, weight difference threshold) and multiple judgment conditions (weight judgment, noise point count judgment) are used to strictly distinguish the pixel attributes, avoid the problem of insufficient or excessive stripe noise removal intensity, and better protect the effective edge details while effectively removing stripe noise.

[0015] In one possible implementation, the first weight value is determined by: filtering the image to be processed through the first filtering window to obtain a first coefficient value for each pixel; determining the first weight value for each pixel based on the first coefficient value for each pixel; and determining the second weight value by: filtering the image to be processed through the second filtering window to obtain a second coefficient value for each pixel; determining the second weight value for each pixel based on the second coefficient value for each pixel.

[0016] In one possible implementation, each pixel segment in the plurality of pixel segments corresponding to the j-th column of pixels includes the same number of pixels; or, each pixel segment in the plurality of pixel segments includes scene signal points that are continuously greater than a first set threshold and target points that are greater than a second set threshold. The scene signal points are pixels where the difference between the first weight value and the second weight value is greater than a fifth set threshold, and the target points are pixels where the difference between the first weight value and the second weight value is less than or equal to the third set threshold.

[0017] In one possible implementation, when the number of pixels included in each of the plurality of pixel segments is the same, the second noise quantity threshold corresponding to each pixel segment is the first quantity threshold; or, when the number of pixels included in each of the plurality of pixel segments is different, the second noise quantity threshold corresponding to any one of the plurality of pixel segments is the product of the number of pixels included in that one pixel segment and a set ratio.

[0018] Based on the above scheme, different noise amplitudes are used for correction in different segment intervals, ensuring effective removal of stripe noise while avoiding the introduction of reverse stripe problems. In segment noise processing, a relationship is established between the segment length and a threshold for the number of noise points, giving this threshold a certain degree of adaptability and further improving the accuracy of pixel attribute judgment.

[0019] Secondly, embodiments of this application provide a stripe noise removal apparatus, comprising:

[0020] The first determining module is used to determine the first noise amplitude corresponding to the j-th column of pixels for the j-th column of the image to be processed, if the first number of noise points in the j-th column of pixels is greater than the first noise number threshold, and the first high-frequency data of the noise points in the j-th column of pixels is determined by filtering the image to be processed through the first filtering window.

[0021] The second determining module is used to determine the second noise amplitude corresponding to any pixel segment of the j-th column of the image to be processed, if the second number of noise points in the arbitrary pixel segment is greater than the second noise number threshold corresponding to the arbitrary pixel segment, then based on the second high-frequency data of the noise points in the arbitrary pixel segment, wherein the j-th column of pixels includes multiple pixel segments, and the second high-frequency data of the noise points is determined by filtering the image to be processed through a second filtering window, and the width of the first filtering window is smaller than the width of the second filtering window;

[0022] The noise reduction module is used to perform noise reduction processing on the j-th column pixels of the image to be processed based on the first noise amplitude and multiple second noise amplitudes corresponding to the j-th column pixels.

[0023] In one possible implementation, the first determining module is further configured to determine the first high-frequency data by: filtering the image to be processed through the first filtering window to obtain the first low-frequency data corresponding to each pixel in the image to be processed; and for any pixel, subtracting the pixel value of the arbitrary pixel in the image to be processed from the first low-frequency data of the arbitrary pixel to obtain the first high-frequency data corresponding to the arbitrary pixel.

[0024] The second determining module is further configured to determine the second high-frequency data by: filtering the image to be processed through the second filtering window to obtain the second low-frequency data corresponding to each pixel in the image to be processed; and for any pixel, subtracting the pixel value of the pixel in the image to be processed from the second low-frequency data of the pixel at the corresponding position to obtain the second high-frequency data corresponding to the pixel.

[0025] In one possible implementation, the j-th column pixel is a column pixel in the image to be processed that has left and right adjacent pixel columns;

[0026] The first determining module is further configured to determine noise points in the j-th column of pixels by: determining a first difference between the first weight value of the i-th pixel and the first weight value of its left neighboring pixel, and a second difference between the first weight value of the i-th pixel and the first weight value of its right neighboring pixel, wherein the i-th pixel is any pixel in the j-th column of pixels; when both the first difference and the second difference are less than a first preset threshold and the first weight value of the i-th pixel is greater than a second preset threshold, the i-th pixel is determined to be a noise point; and / or,

[0027] The second determining module is further configured to determine noise points in a pixel segment by: determining a third difference between the second weight value of the s-th pixel and the second weight value of its left neighboring pixel, and a fourth difference between the second weight value of the s-th pixel and the second weight value of its right neighboring pixel, wherein the s-th pixel is any pixel in the pixel segment; when both the third difference and the fourth difference are less than a third preset threshold and the second weight value of the s-th pixel is greater than a fourth preset threshold, the s-th pixel is determined to be a noise point in the pixel segment; wherein the first weight value and the second weight value are used to characterize the pixel differences between adjacent pixels.

[0028] In one possible implementation, the first determining module is further configured to determine the first weight value by: filtering the image to be processed through the first filtering window to obtain a first coefficient value for each pixel; and determining the first weight value for each pixel by the first coefficient value for each pixel.

[0029] The second determining module is further configured to determine the second weight value by: filtering the image to be processed through the second filtering window to obtain the second coefficient value of each pixel; and determining the second weight value of each pixel by the second coefficient value of each pixel.

[0030] In one possible implementation, each pixel segment in the plurality of pixel segments corresponding to the j-th column of pixels includes the same number of pixels; or, each pixel segment in the plurality of pixel segments includes scene signal points that are continuously greater than a first set threshold and target points that are greater than a second set threshold. The scene signal points are pixels where the difference between the first weight value and the second weight value is greater than a fifth set threshold, and the target points are pixels where the difference between the first weight value and the second weight value is less than or equal to the third set threshold.

[0031] In one possible implementation, when the number of pixels included in each of the plurality of pixel segments is the same, the second noise quantity threshold corresponding to each pixel segment is the first quantity threshold; or, when the number of pixels included in each of the plurality of pixel segments is different, the second noise quantity threshold corresponding to any one of the plurality of pixel segments is the product of the number of pixels included in that one pixel segment and a set ratio.

[0032] Thirdly, embodiments of this application provide an execution device, including:

[0033] Memory, used to store program instructions;

[0034] A processor is configured to acquire program instructions stored in the memory and execute the method described in the first aspect and different implementations of the first aspect according to the acquired program instructions.

[0035] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to perform the methods described in the first aspect and different implementations of the first aspect.

[0036] The technical effects of any of the implementation methods in the second to fourth aspects can be found in the first aspect and the technical effects of different implementation methods of the first aspect, which will not be repeated here. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 A schematic diagram illustrating a method for removing stripe noise provided in an embodiment of this application;

[0039] Figure 2A This is a schematic diagram of an application scenario provided by an embodiment of this application;

[0040] Figure 2B A schematic diagram of a server structure provided in an embodiment of this application;

[0041] Figure 3 A schematic flowchart illustrating a method for removing stripe noise provided in an embodiment of this application;

[0042] Figure 4This is a schematic diagram of the pixels of an image to be processed, provided in an embodiment of this application.

[0043] Figure 5 A schematic diagram of a stripe noise removal device provided in an embodiment of this application;

[0044] Figure 6 This is a schematic diagram of an execution device provided in an embodiment of this application. Detailed Implementation

[0045] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can be arranged and designed in various different configurations.

[0046] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0047] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0048] Due to the limitations of its manufacturing process, infrared detectors exhibit significant non-uniformity among pixels in their focal plane array. Stripe noise, a specific type of non-uniformity, is primarily caused by the use of different amplification circuits in detector units across different rows or columns on the infrared focal plane. Differences in readout circuit voltages ultimately result in stripes of varying brightness on the infrared image, severely impacting image quality. Existing technologies address relatively prominent, narrow stripes, but do not consider removing wider stripes with less brightness difference from the background, thus resulting in insufficient stripe removal performance. Furthermore, existing stripe removal methods are prone to creating inverted stripes in scenes with partially obscured surfaces and vertically edged objects, such as buildings and trees, in areas that were originally stripe-free.

[0049] To address the aforementioned problems, embodiments of this application provide a method and apparatus for removing stripe noise, the algorithm flow of which is as follows: Figure 1 As shown. Taking vertical stripe noise as an example, the details are as follows:

[0050] The image data to be processed is filtered using a small filtering window to obtain first low-frequency data and a first weight value, and then filtered using a large filtering window to obtain second low-frequency data and a second weight value. First high-frequency data and second high-frequency data are obtained by subtracting the image data to be processed from the first low-frequency data and the second low-frequency data, respectively. Noise is extracted using the first high-frequency data and the first weight value to obtain the noise amplitude of the small filtering window. Pixel columns are adaptively segmented using the first and second weight values, and for each pixel segment, noise extraction is performed based on the second high-frequency data and the second weight value to obtain the noise amplitude of the large filtering window. The noise from the small filtering window and the noise from the large filtering window are weighted to obtain the total stripe noise, and the difference between the total stripe noise and the image data to be processed is used to obtain the image with stripe noise removed.

[0051] The stripe noise removal method provided in this application can be implemented by an execution device. In some embodiments, the execution device can be an electronic device, which can be implemented by one or more servers. Figure 2A Let's take a server with 100 servers as an example. (Reference) Figure 2AThe diagram illustrates a possible application scenario provided by an embodiment of this application, including a server 100 and a data acquisition device 200. The server 100 can be implemented as a physical server or a virtual server. The server can be implemented as a single server or as a server cluster consisting of multiple servers. The stripe noise removal method provided in this application can be implemented using either a single server or a server cluster. The data acquisition device 200 is a device with image acquisition capabilities, including electronic police equipment, electronic monitoring equipment, surveillance cameras, video recorders, and terminal devices with video acquisition capabilities (such as laptops, computers, mobile phones, and televisions). The data acquisition device 200 can send the acquired image with stripe noise to be removed to the server 100 via a network. Optionally, the server 100 can be connected to a terminal device 300, receive stripe noise removal tasks sent by the terminal device 300, and perform stripe noise removal based on the received image with stripe noise to be removed sent by the data acquisition device 200. In some scenarios, the server 100 can send the image with stripe noise removed to the terminal device 300. The terminal device 300 can be a television, mobile phone, tablet computer, personal computer, etc.

[0052] As an example, see Figure 2B As shown, server 100 may include processor 110, communication interface 120, and memory 130. Of course, server 100 may also include other components. Figure 2B Not shown in the image.

[0053] The communication interface 120 is used to communicate with the acquisition device 200 and the terminal device 300, to receive the image to be striped noise removed sent by the acquisition device 200, or to receive the striped noise removal task sent by the terminal device 300, or to send the striped noise removal result or (image with striped noise removed) to the terminal device 300.

[0054] In the embodiments of this application, the processor 110 may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0055] Processor 110 is the control center of server 100, connecting various parts of server 100 through various interfaces and routes. It executes various functions and processes data by running or executing software programs and / or modules stored in memory 130, and by calling data stored in memory 130. Optionally, processor 110 may include one or more processing units. Processor 110 may be, for example, a processor, microprocessor, controller, or other control component. It may be a general-purpose central processing unit (CPU), a general-purpose processor, a digital signal processing unit (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof.

[0056] The memory 130 can be used to store software programs and modules. The processor 110 executes various functional applications and data processing by running the software programs and modules stored in the memory 130. The memory 130 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function, etc.; the data storage area may store data created according to business processing, etc. As a non-volatile computer-readable storage medium, the memory 130 can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The memory 130 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. The memory 130 is any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. The memory 130 in this embodiment can also be a circuit or any other means capable of implementing storage functions for storing program instructions and / or data.

[0057] In other embodiments, the execution device may be a terminal device. In some scenarios, the terminal device can receive an image with stripe noise to be removed sent by the acquisition device, and identify foreign objects in the camera lens based on the image with stripe noise to be removed. The terminal device may include a display device, which may be a liquid crystal display, an organic light-emitting diode (OLED) display, a projection display device, etc., and this application does not specifically limit it.

[0058] It should be noted that the above Figure 2A and Figure 2B The structure shown is merely an example, and the embodiments of this application are not limited thereto.

[0059] This application provides a method for removing stripe noise. Figure 3 An exemplary flowchart of a method for removing stripe noise is shown, which can be performed by a stripe noise removal device, which can be located as follows: Figure 2B The server 100 shown could be, for example, a processor 110 or the server 100 itself. The stripe noise removal device can also be located in the terminal device. The specific process is as follows:

[0060] 301. For the j-th column of pixels in the image to be processed, if the first number of noise points in the j-th column of pixels is greater than the first noise number threshold, then the first noise amplitude corresponding to the j-th column of pixels is determined based on the first high-frequency data of the noise points in the j-th column of pixels.

[0061] In some embodiments, the image to be processed is an image frame captured by an acquisition device. The acquisition device can be an electronic police device, electronic monitoring device, surveillance camera, video recorder, or a terminal device with image acquisition capabilities (such as a laptop, computer, mobile phone, or television). For example, after the acquisition device acquires the image frame, the server obtains the image to be processed from the acquisition device.

[0062] In some embodiments, the first high-frequency data of the noise points is determined after the image to be processed is filtered by a first filtering window. In some scenarios, the first high-frequency data can be determined by filtering the image to be processed by a first filtering window to obtain the first low-frequency data corresponding to each pixel in the image to be processed. The first filtering window is a 1×N1 window; for example, N1 can be 11.

[0063] Furthermore, for any given pixel, the first high-frequency data corresponding to that pixel can be obtained by subtracting its pixel value from the first low-frequency data of that pixel in the image to be processed. In some scenarios, filtering methods such as Gaussian filtering, bilateral filtering, and guided filtering can be used, which have good filtering effects and can reflect the differences between the pixel to be processed and other pixels in the neighborhood window. This application takes guided filtering as an example, where the image to be processed can be used as a guide for guided filtering, and the filtering result (the first low-frequency data corresponding to each pixel) can be obtained.

[0064] In some embodiments, when the j-th column of pixels is a column of pixels in the image to be processed that has left and right adjacent pixel columns, noise points in the j-th column of pixels can be determined as follows: Determine the first difference between the first weight value of the i-th pixel and the first weight value of its left adjacent pixel, and the second difference between the first weight value of the i-th pixel and the first weight value of its right adjacent pixel. Here, the i-th pixel is any pixel in the j-th column of pixels. As an example, the index position of the i-th pixel in the image to be processed can be represented as (i, j), then the index position of the pixel to the left of the i-th pixel in the image to be processed can be represented as (i-1, j), and the index position of the pixel to the right of the i-th pixel in the image to be processed can be represented as (i+1, j), and so on. Figure 4 As shown. Calculate the first difference between the first weight value of the pixel at index position (i-1, j) and the first weight value of the pixel at index position (i, j), and the second difference between the first weight value of the pixel at index position (i+1, j) and the first weight value of the pixel at index position (i, j). When both the first difference and the second difference are less than a first preset threshold and the first weight value of the i-th pixel is greater than the second preset threshold, the i-th pixel is determined to be a noise point.

[0065] In other embodiments, when the j-th column pixel is the 1st column pixel, there is no left adjacent pixel to the 1st column. In this case, the pixel value of the 2nd column pixel can be copied as the 0th column pixel, and then the first difference in pixel values ​​between the pixels in the 0th column and the pixels in the same row number as the pixels in the 1st column pixel is calculated. When the j-th column pixel is the last column pixel, there is no right adjacent pixel to the last column. Assuming the size of the image to be processed is h×k, the last column pixel is the k-th column pixel. In this case, the pixel value of the (k-1)-th column pixel can be copied as the (k+1)-th column pixel, and then the second difference in pixel values ​​between the pixels in the k-th column and the pixels in the (k+1)-th column pixel is calculated.

[0066] Furthermore, after identifying the noise points in the j-th column of pixels, if the first number of noise points in the j-th column of pixels is greater than a first noise quantity threshold, then the first noise amplitude corresponding to the j-th column of pixels is determined based on the first high-frequency data of the noise points. For example, the first noise quantity threshold can be 15. When the first number of noise points in the j-th column of pixels is determined to be 25, the first number of noise points is greater than the first noise quantity threshold. Therefore, the average of the first high-frequency data corresponding to each of the 25 noise points is used as the first noise amplitude of the j-th column of pixels. In some scenarios, the first noise amplitude corresponding to the j-th column of pixels is 0.

[0067] In some embodiments, the first weight value can be determined as follows: the image to be processed is filtered through a first filtering window to obtain the first coefficient value of each pixel; the first weight value of each pixel is determined by the first coefficient value of each pixel. The first weight value is used to characterize the pixel difference between adjacent pixels. In some scenarios, for ease of calculation, the first coefficient value can be converted to an integer, then the first weight value satisfies the condition shown in the following formula: weight1=A×(1-mean) a1 Where weight represents the first weight value, and mean... a1 This represents the first coefficient value, where A is an integer and can take values ​​such as 4096. A smaller first weight value indicates a greater difference between the pixel and its neighboring pixels, while a larger first weight value indicates a smaller difference. In some scenarios, the first threshold can be set to 100, and the second threshold can be set to 3600.

[0068] 302. For any pixel segment of the j-th column of the image to be processed, if the second number of noise points in any pixel segment is greater than the second noise number threshold corresponding to any pixel segment, then the second noise amplitude corresponding to any pixel segment is determined based on the second high-frequency data of the noise points in any pixel segment.

[0069] In some embodiments, the second high-frequency data of noise points in the j-th column of pixels is determined after the image to be processed is filtered by a second filtering window. The second high-frequency data can be determined as follows: The image to be processed is filtered by the second filtering window to obtain the second low-frequency data corresponding to each pixel in the image to be processed. The second filtering window is a 1×N² window, and the width of the first filtering window is smaller than the width of the second filtering window. As an example, if N² is greater than N1, then N² can be 31.

[0070] In some embodiments, for any given pixel, the second high-frequency data corresponding to that pixel can be obtained by subtracting the pixel value of any pixel in the image to be processed from the second low-frequency data of the corresponding pixel.

[0071] In some embodiments, each column of pixels can be segmented. In some scenarios, each column of pixels can be divided into multiple pixel segments. Taking the j-th column of pixels as an example, it can be divided into M pixel segments, where each of the M pixel segments includes the same number of pixels. As an example, the j-th column of pixels can be divided into 3 pixel segments. When the image size is 1024×768, each pixel segment includes 256 pixels. In other scenarios, each column of pixels can be adaptively segmented. Taking the j-th column of pixels being divided into multiple pixel segments as an example, each pixel segment includes scene signal points that are continuously greater than a first set threshold and target points that are greater than a second set threshold. Scene signal points are pixels where the difference between a first weight value and a second weight value is greater than a fifth set threshold, and target points are pixels where the difference between a first weight value and a second weight value is less than or equal to the fifth set threshold. The fifth set threshold can be set to 100, the second set threshold can be set to 15, and the first set threshold can be set to 10. In some embodiments, the j-th column of pixels can be segmented into multiple pixel segments based on other characteristic indicators such as gradient characteristics or variance characteristics, which will not be elaborated in this application.

[0072] Furthermore, after dividing the j-th column of pixels into multiple pixel segments, noise points in the pixel segments can be determined as follows: a third difference between the second weight value of the s-th pixel and the second weight value of its left neighbor, and a fourth difference between the second weight value of the s-th pixel and the second weight value of its right neighbor. Here, the s-th pixel is any pixel in the pixel segment. When the j-th column is either the first or last column of pixels, the column pixels to the left of the first column and the column pixels to the right of the last column can be determined using the method in step 301, which will not be elaborated further here.

[0073] In some scenarios, when both the third and fourth differences are less than the third set threshold and the second weight value of the s-th pixel is greater than the fourth set threshold, the s-th pixel is determined to be a noise point within the pixel segment.

[0074] In some scenarios, the second weight value can be determined as follows: The image to be processed is filtered through a second filtering window to obtain the second coefficient value for each pixel; the second weight value for each pixel is then determined using the second coefficient value. Similarly, for ease of calculation, the second weight value can be converted to an integer. The second weight value then satisfies the condition shown in the following formula: weight2 = A × (1 - mean) a2 Where weight2 represents the second weight value, mean a1This represents the first coefficient value, where A is an integer and can take values ​​such as 4096. At this point, the third threshold can be set to 100, and the fourth threshold can be set to 3700.

[0075] In some embodiments, after determining the noise points in each pixel segment of the j-th column of pixels, if the second number of noise points in any pixel segment is greater than the second noise number threshold corresponding to that pixel segment, then the second noise amplitude corresponding to that pixel segment is determined based on the second high-frequency data of the noise points in that pixel segment. For example, the second noise number threshold can be 15. When the second number of noise points in any pixel segment is determined to be 25, the second number of noise points is greater than the second noise number threshold. Therefore, the average of the second high-frequency data corresponding to each of the 25 noise points is used as the second noise amplitude corresponding to that pixel segment in the j-th column of pixels. In some scenarios, when the second number of noise points in a certain pixel segment is less than or equal to the second noise number threshold, the second noise amplitude corresponding to that pixel segment is 0.

[0076] In some embodiments, when the number of pixels in each of the multiple pixel segments is the same, the second noise quantity threshold for each pixel segment is the first quantity threshold. When the number of pixels in each of the multiple pixel segments is different, the second noise quantity threshold for any pixel segment is the product of the number of pixels in that pixel segment and a set ratio. As an example, when a pixel segment includes 256 pixels, the set ratio can be 0.3, then the second noise quantity threshold for that pixel segment is 77.

[0077] 303. Based on the first noise amplitude and multiple second noise amplitudes corresponding to the j-th column of pixels, perform noise reduction processing on the j-th column of pixels in the image to be processed.

[0078] In some embodiments, denoising can be performed on each column of pixels after determining the first noise amplitude and multiple second noise amplitudes corresponding to each column of pixels. In other embodiments, after determining the first noise amplitude and multiple second noise amplitudes corresponding to all pixel rows, the first noise amplitude and multiple second noise amplitudes corresponding to each pixel row can be weighted to obtain the total noise, and then the image to be processed can be denoised using the total noise.

[0079] It should be noted that the above embodiments are illustrated using vertical stripe noise as an example. The processing steps for horizontal stripe noise are the same as those for vertical stripe noise, only requiring a slight modification in the direction, which will not be repeated here.

[0080] In some scenarios, multi-size windows can be used to specifically process stripe noise. The calculation method is the same as the stripe noise removal method described above, and will not be repeated here.

[0081] Based on the same technical concept, embodiments of this application provide a stripe noise removal device 500, such as... Figure 5 As shown. The device 500 can perform any step in the above-described stripe noise removal method, which will not be described again here to avoid repetition. The device 500 includes a first determining module 501, a second determining module 502, and a noise reduction module 503.

[0082] The first determining module 501 is used to determine the first noise amplitude corresponding to the j-th column of pixels in the image to be processed if the first number of noise points in the j-th column of pixels is greater than the first noise number threshold. The first high-frequency data of the noise points in the j-th column of pixels is determined by filtering the image to be processed through the first filtering window.

[0083] The second determining module 502 is used to determine the second noise amplitude corresponding to any pixel segment of the j-th column of the image to be processed, if the second number of noise points in the arbitrary pixel segment is greater than the second noise number threshold corresponding to the arbitrary pixel segment, then based on the second high-frequency data of the noise points in the arbitrary pixel segment, wherein the j-th column of pixels includes multiple pixel segments, and the second high-frequency data of the noise points is determined by filtering the image to be processed through a second filtering window, and the width of the first filtering window is smaller than the width of the second filtering window;

[0084] The denoising module 503 is used to perform denoising processing on the j-th column of pixels of the image to be processed based on the first noise amplitude and multiple second noise amplitudes corresponding to the j-th column of pixels.

[0085] In some embodiments, the first determining module 501 is further configured to determine the first high-frequency data by: filtering the image to be processed through the first filtering window to obtain the first low-frequency data corresponding to each pixel in the image to be processed; and for any pixel, subtracting the pixel value of the pixel in the image to be processed from the first low-frequency data of the pixel to obtain the first high-frequency data corresponding to the pixel.

[0086] The second determining module 502 is further configured to determine the second high-frequency data in the following manner: filtering the image to be processed through the second filtering window to obtain the second low-frequency data corresponding to each pixel in the image to be processed; for any pixel, subtracting the pixel value of the pixel in the image to be processed from the second low-frequency data of the pixel at the corresponding position to obtain the second high-frequency data corresponding to the pixel.

[0087] In some embodiments, the j-th column of pixels refers to the column pixels in the image to be processed that have left and right adjacent pixel columns; the first determining module 501 is further configured to determine noise points in the j-th column of pixels by: determining a first difference between the first weight value of the i-th pixel and the first weight value of the left adjacent pixel, and a second difference between the first weight value of the i-th pixel and the first weight value of the right adjacent pixel, wherein the i-th pixel is any pixel in the j-th column of pixels; when both the first difference and the second difference are less than a first preset threshold and the first weight value of the i-th pixel is greater than a second preset threshold, the i-th pixel is determined to be a noise point; and / or,

[0088] The second determining module 502 is further configured to determine noise points in a pixel segment by: determining a third difference between the second weight value of the s-th pixel and the second weight value of its left neighboring pixel, and a fourth difference between the second weight value of the s-th pixel and the second weight value of its right neighboring pixel, wherein the s-th pixel is any pixel in the pixel segment; when both the third difference and the fourth difference are less than a third preset threshold and the second weight value of the s-th pixel is greater than a fourth preset threshold, the s-th pixel is determined to be a noise point in the pixel segment; wherein the first weight value and the second weight value are used to characterize the pixel differences between adjacent pixels.

[0089] In some embodiments, the first determining module 501 is further configured to determine the first weight value by: filtering the image to be processed through the first filtering window to obtain a first coefficient value for each pixel; and determining the first weight value for each pixel by the first coefficient value for each pixel.

[0090] The second determining module 502 is further configured to determine the second weight value by: filtering the image to be processed through the second filtering window to obtain the second coefficient value of each pixel; and determining the second weight value of each pixel by the second coefficient value of each pixel.

[0091] In some embodiments, each pixel segment in the plurality of pixel segments corresponding to the j-th column of pixels includes the same number of pixels, or each pixel segment in the plurality of pixel segments includes a scene signal point that is continuously greater than a first set number threshold and a target point that is greater than a second set number threshold. The scene signal point is a pixel whose difference between the first weight value and the second weight value is greater than a fifth set threshold, and the target point is a pixel whose difference between the first weight value and the second weight value is less than or equal to the fifth set threshold.

[0092] In some embodiments, when the number of pixels included in each of the plurality of pixel segments is the same, the second noise quantity threshold corresponding to each pixel segment is the first quantity threshold; or, when the number of pixels included in each of the plurality of pixel segments is different, the second noise quantity threshold corresponding to any one of the plurality of pixel segments is the product of the number of pixels included in that one pixel segment and a set ratio.

[0093] Based on the same technical concept, embodiments of this application provide an execution device 600, which can implement any step of the stripe noise removal method discussed above. Please refer to... Figure 6 The device includes a memory 601 and a processor 602.

[0094] The memory 601 is used to store program instructions;

[0095] The processor 602 is used to call the program instructions stored in the memory and execute the above-mentioned stripe noise removal method according to the obtained program.

[0096] In the embodiments of this application, the processor 602 may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0097] Memory 601, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 601 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 601 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. Memory 601 in the embodiments of this application may also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0098] Based on the same technical concept, embodiments of this application provide a computer-readable storage medium, including: computer program code, which, when executed on a computer, causes the computer to perform the stripe noise removal method as described above. Since the principle of the problem solved by the above-described computer-readable storage medium is similar to that of the stripe noise removal method, the implementation of the above-described computer-readable storage medium can be referred to the implementation of the method, and repeated details will not be elaborated further.

[0099] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0100] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0101] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0102] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0103] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method of removing stripe noise, characterized by, The method comprises: For the jth column of pixel points of the to-be-processed image, if the first number of noise points in the jth column of pixel points is greater than a first noise number threshold, a first noise amplitude corresponding to the jth column of pixel points is determined based on first high-frequency data of the noise points in the jth column of pixel points, wherein the first high-frequency data of the noise points is determined after the to-be-processed image is filtered by a first filter window; Wherein, the jth column of pixel points is a column of pixel points with left and right adjacent pixel columns in the to-be-processed image, and the noise points in the jth column of pixel points are determined in the following manner: A first difference between a first weight value of an ith pixel point and a first weight value of a left adjacent pixel point, and a second difference between the first weight value of the ith pixel point and a first weight value of a right adjacent pixel point are determined, the ith pixel point being any pixel point in the jth column of pixel points; the first weight value is used to represent the pixel difference between adjacent pixel points; When the first difference and the second difference are both less than a first set threshold and the first weight value of the ith pixel point is greater than a second set threshold, the ith pixel point is determined to be a noise point; For any one pixel segment of the jth column of pixel points of the to-be-processed image, if the second number of noise points in the any one pixel segment is greater than a second noise number threshold corresponding to the any one pixel segment, a second noise amplitude corresponding to the any one pixel segment is determined based on second high-frequency data of the noise points in the any one pixel segment, wherein the jth column of pixel points comprises a plurality of pixel segments, the second high-frequency data of the noise points is determined after the to-be-processed image is filtered by a second filter window, and the width of the first filter window is less than the width of the second filter window; The jth column of pixel points of the to-be-processed image is denoised according to the first noise amplitude corresponding to the jth column of pixel points and a plurality of second noise amplitudes.

2. The method of claim 1, wherein, The first high-frequency data is determined in the following manner: First low-frequency data corresponding to each pixel point in the to-be-processed image is obtained after the to-be-processed image is filtered by the first filter window; For any one pixel point, a first high-frequency data corresponding to the any one pixel point is obtained by subtracting the first low-frequency data of the any one pixel point from the pixel value of the any one pixel point in the to-be-processed image; The second high-frequency data is determined in the following manner: Second low-frequency data corresponding to each pixel point in the to-be-processed image is obtained after the to-be-processed image is filtered by the second filter window; For any one pixel point, a second high-frequency data corresponding to the any one pixel point is obtained by subtracting the second low-frequency data of the corresponding pixel point from the pixel value of the any one pixel point in the to-be-processed image.

3. The method of claim 1, wherein, The method further comprises: The noise points in the pixel segment are determined in the following manner: A third difference between a second weight value of an sth pixel point and a second weight value of a left adjacent pixel point, and a fourth difference between the second weight value of the sth pixel point and a second weight value of a right adjacent pixel point are determined, the sth pixel point being any pixel point in the pixel segment; determining that the s-th pixel point is a noise point in the pixel segment when the third difference value and the fourth difference value are both less than a third set threshold value and the second weight value of the s-th pixel point is greater than a fourth set threshold value; wherein the second weight value is used to represent a pixel difference between adjacent pixel points.

4. The method of claim 3, wherein, The first weight value is determined in the following manner: filtering the to-be-processed image through the first filter window to obtain a first coefficient value of each pixel point; determining the first weight value of each pixel point through the first coefficient value of each pixel point; The second weight value is determined in the following manner: filtering the to-be-processed image through the second filter window to obtain a second coefficient value of each pixel point; determining the second weight value of each pixel point through the second coefficient value of each pixel point.

5. The method of claim 3 or 4, wherein, The number of pixel points included in each pixel segment of the plurality of pixel segments corresponding to the j-th column of pixel points is the same, or each pixel segment of the plurality of pixel segments includes a scene signal point which is greater than a first set number threshold value and a target point which is greater than a second set number threshold value, the scene signal point is a pixel point whose difference between the first weight value and the second weight value is greater than a fifth set threshold value, and the target point is a pixel point whose difference between the first weight value and the second weight value is less than or equal to the third set threshold value.

6. The method of claim 5, wherein, When the number of pixel points included in each pixel segment of the plurality of pixel segments is the same, the second noise number threshold value corresponding to each pixel segment is a first number threshold value; or, When the number of pixel points included in each pixel segment of the plurality of pixel segments is different, the second noise number threshold value corresponding to any one of the plurality of pixel segments is a product of the number of pixel points included in the any one of the plurality of pixel segments and a set proportion.

7. A device for removing stripe noise, characterized in that, comprising: The first determining module is configured to, for a j-th column of pixel points of a to-be-processed image, if a first number of noise points in the j-th column of pixel points is greater than a first noise number threshold value, determine a first noise amplitude corresponding to the j-th column of pixel points based on first high-frequency data of the noise points in the j-th column of pixel points, wherein the first high-frequency data of the noise points is determined after the to-be-processed image is filtered through a first filter window. The j-th column of pixel points is a column of pixel points with left and right adjacent pixel columns in the to-be-processed image, and the first determining module is further configured to: determine a noise point in the j-th column of pixel points in the following manner: determine a first difference between a first weight value of an i-th pixel point and a first weight value of a left adjacent pixel point, and a second difference between the first weight value of the i-th pixel point and a first weight value of a right adjacent pixel point, the i-th pixel point being any pixel point in the j-th column of pixel points; the first weight value is used to represent a pixel difference between adjacent pixel points; determine that the i-th pixel point is a noise point when the first difference and the second difference are both less than a first set threshold value and the first weight value of the i-th pixel point is greater than a second set threshold value; The second determining module is configured to: for any one pixel segment of the jth column of pixel points of the to-be-processed image, if a second quantity of noise points in the any one pixel segment is greater than a second noise quantity threshold corresponding to the any one pixel segment, determine a second noise amplitude corresponding to the any one pixel segment based on second high-frequency data of the noise points in the any one pixel segment, wherein the jth column of pixel points comprises a plurality of pixel segments, and the second high-frequency data of the noise points is determined after the to-be-processed image is filtered by a second filter window, and a width of the first filter window is less than a width of the second filter window; The denoising module is configured to: perform denoising processing on the jth column of pixel points of the to-be-processed image according to the first noise amplitude corresponding to the jth column of pixel points and the plurality of second noise amplitudes.

8. The apparatus of claim 7, wherein, The first determining module is further configured to determine the first high-frequency data in the following manner: obtain first low-frequency data corresponding to each pixel point in the to-be-processed image after filtering the to-be-processed image by the first filter window; for any one pixel point, obtain first high-frequency data corresponding to the any one pixel point by subtracting the first low-frequency data of the any one pixel point from a pixel value of the any one pixel point in the to-be-processed image; The second determining module is further configured to determine the second high-frequency data in the following manner: obtain second low-frequency data corresponding to each pixel point in the to-be-processed image after filtering the to-be-processed image by the second filter window; for any one pixel point, obtain second high-frequency data corresponding to the any one pixel point by subtracting the second low-frequency data of the any one pixel point from a pixel value of the any one pixel point in the to-be-processed image.

9. The apparatus of claim 7, wherein, The second determining module is further configured to determine noise points in a pixel segment in the following manner: determine a third difference between a second weight value of an s-th pixel point and a second weight value of a left adjacent pixel point, and a fourth difference between the second weight value of the s-th pixel point and a second weight value of a right adjacent pixel point, the s-th pixel point being any one pixel point in the pixel segment; when the third difference and the fourth difference are both less than a third set threshold and the second weight value of the s-th pixel point is greater than a fourth set threshold, determine that the s-th pixel point is a noise point in the pixel segment; wherein the second weight value is used to represent a pixel difference between adjacent pixel points.

10. The apparatus of claim 9, wherein, The first determining module is further configured to determine the first weight value in the following manner: obtain a first coefficient value of each pixel point after filtering the to-be-processed image by the first filter window; determine the first weight value of each pixel point through the first coefficient value of each pixel point; The second determining module is further configured to determine the second weight value in the following manner: obtain a second coefficient value of each pixel point after filtering the to-be-processed image by the second filter window; determine the second weight value of each pixel point through the second coefficient value of each pixel point.

11. The apparatus of claim 9 or 10, wherein, The number of pixel points included in each of the plurality of pixel segments corresponding to the jth column of pixel points is the same, or each of the plurality of pixel segments includes a scene signal point greater than a first set number threshold and a target point greater than a second set number threshold, the scene signal point is a pixel point with a difference between the first weight value and the second weight value greater than a fifth set threshold, and the target point is a pixel point with a difference between the first weight value and the second weight value less than or equal to a third set threshold.

12. The apparatus of claim 11, wherein, When the number of pixel points included in each of the plurality of pixel segments is the same, the second noise number threshold corresponding to each of the plurality of pixel segments is a first number threshold; or, When the number of pixel points included in each of the plurality of pixel segments is different, the second noise number threshold corresponding to any one of the plurality of pixel segments is a product of the number of pixel points included in the any one of the plurality of pixel segments and a set proportion.

13. An execution device, characterized by Comprise: a memory for storing program instructions; a processor for obtaining the program instructions stored in the memory, and executing the method of any one of claims 1-6 according to the obtained program instructions.

14. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions, when the computer instructions run on the computer, make the computer execute the method of any one of claims 1-6.

Images