Fast image filtering methods, systems, devices and media for embedded airborne scenarios

Abstract

This invention provides a fast image filtering method, system, device, and medium for embedded airborne scenarios. The method includes: determining a filtering region for each pixel in the image to be filtered, centered on each pixel to be filtered and using a set radius as the filtering radius; adaptively determining the sampling step size for each filtering region based on its gradient information and radius; sampling each filtering region using the sampling step size to obtain multiple sampled pixels; and using a filtering algorithm corresponding to the embedded airborne scenario, performing filtering processing on each filtering region using the multiple sampled pixels of each filtering region, and outputting the filtered value of each filtering region. This invention solves the technical problem that existing filtering methods require an exponential increase in the number of nested loops as the filtering region increases, thus significantly reducing image processing efficiency. It achieves high execution efficiency and excellent filtering performance.

Description

Technical Field

[0001] This invention relates to the field of airborne digital image processing technology, specifically to a fast image filtering method, system, device, and medium for embedded airborne scenarios. Background Technology

[0002] In embedded scenarios, chip power consumption and heat dissipation are subject to strict requirements, resulting in relatively low chip computing power. As the most basic image processing method, filtering methods have wide applications in the field of image processing. Their real-time performance is very important in embedded environments, and slow processing speed will be detrimental to the real-time performance of other image processing tasks in the later stages.

[0003] Currently, image filtering methods are widely used in numerous computer vision and image processing tasks, such as guided filtering, airborne super-resolution methods based on iterative back-projection, and airborne image dehazing methods based on dark channels. Among these, guided filtering uses Box Filter to average pixel values ​​in repetitive regions, airborne super-resolution methods based on iterative back-projection achieve resolution degradation through Gaussian filtering, and airborne image dehazing methods based on dark channels obtain dark channel information through minimum value filtering.

[0004] For airborne embedded image processing applications, a large number of image filtering methods are also required. Currently, CPUs, GPUs, FPGAs, and DSPs, as commonly used embedded platform signal processors, typically require nested loops to index the surrounding pixels (in two-dimensional, three-dimensional, or multi-dimensional space) of each central pixel when performing image filtering tasks. However, as the filtering area increases, the number of nested loops required increases exponentially, significantly reducing image processing efficiency and posing a huge challenge to the real-time image filtering requirements in embedded scenarios. Summary of the Invention

[0005] To address the technical problem that existing filtering methods require an exponential increase in nested loops as the filtering area grows, thus significantly reducing image processing efficiency, this invention discloses a fast image filtering method, system, device, and medium for embedded airborne scenarios. The method of this invention can overcome the problem of high time overhead caused by the large filtering area in traditional image filtering methods.

[0006] The technical solution to achieve the purpose of the invention is as follows:

[0007] This invention provides a fast image filtering method for embedded airborne scenarios, comprising:

[0008] Step 1: Using each pixel to be filtered as the center and a set radius as the filtering radius, determine the filtering area for each pixel in the image to be processed.

[0009] Step 2: Based on the gradient information and radius of each filtering region, adaptively determine the sampling step size of each filtering region;

[0010] Step 3: Using the sampling step size, sample each of the filtered regions to obtain multiple sampled pixels;

[0011] Step 4: Using a filtering algorithm corresponding to the embedded airborne scenario, multiple sampled pixels of each filtering region are used to perform filtering processing on each filtering region and output the filtered value of each filtering region.

[0012] Optionally, in step 2, the gradient information and radius of each of the filtering regions are used to adaptively determine the sampling step size of each filtering region, including:

[0013] Step 21: Obtain the gradient information by calculating the gradient between each pixel to be filtered and other pixels in each of the filtering regions, and determine the position of the pixel to be filtered in each of the filtering regions in the image to be processed based on the gradient information;

[0014] Step 22: Determine the initial sampling step size for each filtering region based on the position of each pixel to be filtered in the image to be processed;

[0015] Step 23: Based on the radius of each filtering region, when the radius is not an integer multiple of the initial sampling step size, adjust the initial sampling step size to obtain an adjusted sampling step size, so that the radius is an integer multiple of the adjusted sampling step size, and determine the adjusted sampling step size as the final sampling step size.

[0016] Optionally, in step 22, determining the initial sampling step size of each filtering region based on the position of each pixel to be filtered in the image to be processed includes:

[0017] When each pixel to be filtered is located in a texture region in the image to be processed, the first step size is selected as the initial sampling step size.

[0018] When each pixel to be filtered is located in the background region of the image to be processed, a second step size is selected as the initial sampling step size, wherein the first step size is smaller than the second step size.

[0019] Optionally, in step 4, the filtering algorithm corresponding to the embedded airborne scenario includes any one of the following: Gaussian filtering algorithm, mean filtering algorithm, maximum filtering algorithm, minimum filtering algorithm, and median filtering algorithm.

[0020] Optionally, in step 1, the image to be processed includes any one of a two-dimensional grayscale image, a three-dimensional color image, and a high-dimensional data image.

[0021] This invention also provides a fast image filtering system for embedded airborne scenarios, comprising:

[0022] The filtering region determination module is used to determine the filtering region of each pixel to be filtered in the image to be processed, with each pixel to be filtered as the center and a set radius as the filtering radius.

[0023] The sampling step size determination module is used to adaptively determine the sampling step size of each filtering region based on the gradient information and radius of each filtering region;

[0024] The sampling module is used to sample each of the filtered regions to obtain multiple sampled pixels using the sampling step size.

[0025] An embedded platform computing module is used to perform filtering processing on each of the filtering regions by using a filtering algorithm corresponding to the embedded airborne scenario, employing multiple sampled pixels of each filtering region, and outputting the filtered value of each filtering region.

[0026] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the fast image filtering method for any of the above-mentioned embedded airborne scenarios, in order to solve the technical problem that existing filtering methods require an exponential increase in the number of nested loops as the filtering area increases, thereby significantly reducing image processing efficiency.

[0027] This invention also provides a computer-readable storage medium storing a computer program that executes the fast image filtering method for any of the above-described embedded airborne scenarios, in order to solve the technical problem that existing filtering methods require an exponential increase in the number of nested loops as the filtering area increases, thereby significantly reducing image processing efficiency.

[0028] Compared with the prior art, the beneficial effects achieved by at least one of the above-mentioned technical solutions in the embodiments of this specification include at least the following: The fast image filtering method of the present invention, by using an adaptively determined sampling step size for each filtering region, uniformly samples the filtering region of each pixel to be filtered in the image to be processed, and uses multiple sampled pixels in each filtering region to perform filtering processing using a selected filtering algorithm, which can meet the fast filtering method requirements of airborne embedded platforms. For example, when the traditional method uses the Box Filter filtering algorithm to filter a 31*31 pixel region, it needs to perform 31 pixel indexing operations in both the horizontal and vertical directions. The method disclosed in this invention adaptively sets the sampling step size to 5 or 3, and only indexes 6 or 10 times in the horizontal and vertical directions respectively, which greatly improves the computational efficiency. Furthermore, since the present invention uses an adaptive sampling step size for pixel sampling, the overhead during indexing and calculation is theoretically no greater than the original 49 / 961 or 121 / 961. And through experimental acceleration comparison, it can be seen that the method disclosed in this invention not only has high execution efficiency, but also achieves excellent filtering performance. Attached Figure Description

[0029] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a flowchart of a fast image filtering method for embedded airborne scenarios disclosed in an embodiment of the present invention;

[0031] Figure 2 This is a schematic diagram illustrating the operation of the fast image filtering method for embedded airborne scenarios disclosed in an embodiment of the present invention;

[0032] Figure 3 This is a schematic diagram of the computer device disclosed in an embodiment of the present invention;

[0033] Figure 4 This is a schematic diagram of a fast image filtering system for embedded airborne scenarios disclosed in an embodiment of the present invention;

[0034] Among them, 301 is the memory; 302 is the processor; 401 is the filter region determination module; 402 is the sampling step size determination module; 403 is the sampling module; and 404 is the embedded platform computing module. Detailed Implementation

[0035] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0036] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. This application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, in the absence of conflict, the following embodiments and features of the embodiments can be combined with each other. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0037] This invention provides a fast image filtering method for embedded airborne scenarios. See [link to relevant documentation]. Figure 1 and Figure 2 As shown, fast image filtering methods include:

[0038] Step 1: Using each pixel to be filtered as the center and a set radius as the filtering radius, determine the filtering area for each pixel in the image to be processed.

[0039] Step 2: Based on the gradient information and radius of each filtering region, adaptively determine the sampling step size of each filtering region;

[0040] Step 3: Using the sampling step size, sample each of the filtered regions to obtain multiple sampled pixels;

[0041] Step 4: Using a filtering algorithm corresponding to the embedded airborne scenario, multiple sampled pixels of each filtering region are used to perform filtering processing on each filtering region and output the filtered value of each filtering region.

[0042] Optionally, in step 1, the image to be processed includes any one of a two-dimensional grayscale image, a three-dimensional color image, and a high-dimensional data image.

[0043] Optionally, when implementing step 1, the horizontal and vertical coordinate indices centered on the pixel to be filtered (i, j) can be defined as (0, 0), and the filtering radius is h. In this case, the indices of the filtering region of the pixel to be filtered in the horizontal and vertical directions are (-h, -(h-1), ..., -1, 0, 1, ... (h-1), h).

[0044] Optionally, in step 2, the gradient information and radius of each of the filtering regions are used to adaptively determine the sampling step size of each filtering region, including:

[0045] Step 21: Obtain the gradient information by calculating the gradient between each pixel to be filtered and other pixels in each of the filtering regions, and determine the position of each pixel to be filtered in the image to be processed based on the gradient information.

[0046] In practice, the gradient information between the pixel to be filtered (i, j) and other pixels in the filtering region is calculated in the horizontal and vertical directions. judge and The filter checks whether the gradient difference between the pixels (i, j) is greater than a set threshold, where h is the horizontal direction and v is the vertical direction. If the gradient difference is greater than the set threshold, the pixel (i, j) to be filtered belongs to the texture region; otherwise, the pixel (i, j) to be filtered belongs to the background region (i.e., a flat region without texture).

[0047] Step 22: Determine the initial sampling step size for each filtering region based on the position of each pixel to be filtered in the image to be processed.

[0048] Optionally, when each pixel to be filtered is located in a texture region in the image to be processed, a first step length is selected as the initial sampling step length; when each pixel to be filtered is located in a background region in the image to be processed, a second step length is selected as the initial sampling step length, wherein the first step length is smaller than the second step length.

[0049] In practice, the initial sampling step size is determined as follows: Given the filter radius as h, the filter diameter is 2h+1. For textured regions, the initial sampling step size s can be set to s = (2h+1) / 5. For non-textured regions, the second sampling step size s can be set to s = (2h+1) / 3. Since a larger filter diameter results in a more significant acceleration effect, when the filter diameter is less than 5, the initial sampling step size s can be set to 1.

[0050] Step 23: Based on the radius of each filtering region, when the radius is not an integer multiple of the initial sampling step size, adjust the initial sampling step size to obtain an adjusted sampling step size, so that the radius is an integer multiple of the adjusted sampling step size, and determine the adjusted sampling step size as the final sampling step size.

[0051] In practice, due to the uncertainty of the radius of the filtering region and the initial sampling step size, there may be cases where the index of the sampled pixel obtained after sampling according to the initial sampling step size is a non-positive integer. Therefore, it is necessary to adjust the initial sampling step size according to the gradient information of the center pixel so that the index of the sampled pixel is an integer. The specific adjustment method is as follows: when the pixel to be filtered belongs to the texture region, the adjusted sampling step size is s = int((2h+1) / 5+0.5); when the pixel to be filtered belongs to the non-texture region, the adjusted sampling step size is s = int((2h+1) / 3-0.5), where int(·) represents the rounding operation to obtain the final sampling step size.

[0052] Optionally, in specific implementation, step 3 involves sampling each filtering region using the sampling step size determined in step 2 to obtain multiple sampled pixels, with the indices of the multiple sampled pixels being (…, -2s, -s, 0, s, 2s, …).

[0053] Optionally, in step 4, the filtering algorithm corresponding to the embedded airborne scenario includes any one of the following: Gaussian filtering algorithm, mean filtering algorithm, maximum filtering algorithm, minimum filtering algorithm, and median filtering algorithm.

[0054] In practical implementation, for filtering algorithms that require filter coefficients, such as Gaussian filtering and mean filtering, the mathematical formula is expressed as: f(i,j)=∑ a ∑ b The expression w(a,b)x(i+as,j+bs) is used, where w is the filtering coefficient, and its specific value is determined by the type of filtering algorithm, such as Gaussian filtering or Box Filtering. s is the sampling step size, a positive integer greater than or equal to 1. The values ​​of a and b must satisfy as≤h, bs≤h, -as≥-h, and -bs≥-h. For common filtering algorithms such as maximum filtering, minimum filtering, and median filtering, sorting of the sampled pixels is required. As the filtering area increases, the execution time of the sorting method increases exponentially. Replacing all values ​​with the sampled values ​​(…, x(i-2s,j-2s), x(is,js), x(i,j), x(i+s,j+s), x(i+2s,j+2s),…) of the filtering area to calculate the sorting result can reduce the computational load and thus execute the filtering method faster.

[0055] In practical implementation, there are two types of embedded platforms, see [link / reference] Figure 2As shown, one approach is to use a CPU or DSP, which typically performs filtering operations serially (or in parallel across multiple cores) on each pixel. Because CPUs or DSPs have higher clock speeds and relatively fewer cores, they can accelerate filtering for lower-resolution images. The other approach is to use a GPU or FPGA, which have lower clock speeds but more computing cores, enabling large-scale parallel computing and better accelerating filtering for high-resolution images. By mapping each sampled filtering region in the image to each computing thread of the GPU or FPGA, fast filtering of each region in the image can be performed in parallel, resulting in high execution efficiency.

[0056] For embedded airborne scenarios, the runtime and filtering effect of the filtering algorithm are equally important. The effectiveness of this invention is demonstrated by comparing traditional filtering methods with the fast sampling filtering method proposed in this invention.

[0057] This invention selects the Guided Image Filter (GIF) method as the verification method. Numerous image processing applications utilize the GIF method, such as image dehazing, low-light enhancement, denoising, and multi-source image fusion. The GIF method, in its self-guided filtering mode, requires four Box Filter operations, which constitute the majority of its computational overhead. This invention applies the proposed accelerated filtering method to the Box Filter method, thereby speeding up the GIF method.

[0058] This embodiment uses four sets of images with resolutions of 1280x720, 1920*1080, 2560*1440 and 4096*3112 as examples for illustration.

[0059] Table 1: Acceleration time of Box Filter method by domestic GPUs in embedded environments

[0060]

[0061] Table 2: Shows the acceleration time of the GIF method using domestic GPUs in embedded environments.

[0062]

[0063] This invention was verified on a domestically produced GPU, specifically by comparing the acceleration performance of the proposed fast filtering method on BoxFilter and GIF methods on domestic GPUs to demonstrate the applicability of the proposed method to an airborne embedded platform. Table 1 above shows the acceleration time of the BoxFilter method on domestic GPUs in an embedded environment, and Table 2 above shows the acceleration time of the GIF method on domestic GPUs in an embedded environment. From Tables 1 and 2, we can see that: 1) The proposed accelerated filtering method achieves approximately 18 times the acceleration effect for BoxFilter and approximately 16 times the acceleration effect for GIF filtering; 2) Since GIF contains four BoxFilter operations, some BoxFilter operations can be combined into a single GPU execution function to reduce computational overhead. Therefore, Tables 1 and 2 do not strictly satisfy the relationship that the computational time overhead is greater than 4 times; 3) At a resolution of 1280*720, the proposed accelerated filtering method achieves a computation speed of approximately 301 frames per second for BoxFilter and approximately 77 frames per second for GIF.

[0064] In this embodiment, a computer device is provided, such as... Figure 3 As shown, it includes a memory 301, a processor 302, and a computer program stored in the memory 301 and executable on the processor 302. When the processor executes the computer program, it implements the fast image filtering method for any of the above-mentioned embedded airborne scenarios.

[0065] Specifically, the computer device can be a computer terminal, a server, or a similar computing device.

[0066] In this embodiment, a computer-readable storage medium is provided, which stores a computer program that performs the fast image filtering method for any of the above-described embedded airborne scenarios.

[0067] Specifically, computer-readable storage media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer-readable storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable storage media does not include transient media, such as modulated data signals and carrier waves.

[0068] Based on the same inventive concept, this invention also provides a fast image filtering system for embedded airborne scenarios, as described in the following embodiments. Since the principle of the fast image filtering system for embedded airborne scenarios is similar to the fast image filtering method for embedded airborne scenarios described above, the implementation of the fast image filtering system can refer to the implementation of the fast image filtering method, and repeated details will not be elaborated further. As used below, the terms "unit" or "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0069] Figure 4 This is a schematic diagram of a fast image filtering system for an embedded airborne scenario according to an embodiment of the present invention, such as... Figure 4 As shown, the fast image filtering system includes a filtering region determination module 401, a sampling step size determination module 402, a sampling module 403, and an embedded platform calculation module 404. The structure is described below.

[0070] The filtering region determination modulus 401 is used to determine the filtering region of each pixel to be filtered in the image to be processed, with each pixel to be filtered as the center and a set radius as the filtering radius.

[0071] The sampling step size determination module 402 is used to adaptively determine the sampling step size of each filtering region based on the gradient information and radius of each filtering region;

[0072] Sampling module 403 is used to sample each of the filtered regions using the sampling step size to obtain multiple sampled pixels;

[0073] The embedded platform computing module 404 is used to use a filtering algorithm corresponding to the embedded airborne scenario, employing multiple sampled pixels of each filtering region to perform filtering processing on each filtering region and output the filtered value of each filtering region.

[0074] The embodiments of this invention achieve the following technical effects: The fast image filtering method of this invention, by adopting an adaptively determined sampling step size for each filtering region, uniformly samples the filtering region of each pixel to be filtered in the image to be processed, and uses multiple sampled pixels in each filtering region to perform filtering processing using a selected filtering algorithm, which can meet the fast filtering method requirements of airborne embedded platforms. For example, when the traditional method uses the Box Filter algorithm to filter a 31*31 pixel region, it needs to perform 31 pixel indexing operations in both the horizontal and vertical directions. The method disclosed in this invention adaptively sets the sampling step size to 5 or 3, and only indexes 6 or 10 times in the horizontal and vertical directions respectively, which greatly improves the computational efficiency. Furthermore, since this invention uses an adaptive sampling step size for pixel sampling, the overhead during indexing and calculation is theoretically no greater than the original 49 / 961 or 121 / 961. And through experimental acceleration comparison, it can be seen that the method disclosed in this invention not only has high execution efficiency, but also achieves excellent filtering performance.

[0075] Obviously, those skilled in the art should understand that the modules or steps of the above-described embodiments of the present invention can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of the present invention are not limited to any particular hardware and software combination.

[0076] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A fast image filtering method for embedded airborne scenarios, characterized in that, include: Step 1: Using each pixel to be filtered as the center and a set radius as the filtering radius, determine the filtering area for each pixel in the image to be processed. Step 2: Based on the gradient information and radius of each filtering region, adaptively determine the sampling step size of each filtering region; Step 3: Using the sampling step size, sample each of the filtered regions to obtain multiple sampled pixels; Step 4: Using a filtering algorithm corresponding to the embedded airborne scenario, multiple sampled pixels of each filtering region are used to perform filtering processing on each filtering region and output the filtered value of each filtering region; In step 2, the gradient information and radius of each filtering region are used to adaptively determine the sampling step size of each filtering region, including: Step 21: Obtain the gradient information by calculating the gradient between each pixel to be filtered and other pixels in each of the filtering regions, and determine the position of the pixel to be filtered in each of the filtering regions in the image to be processed based on the gradient information; Step 22: Determine the initial sampling step size for each filtering region based on the position of each pixel to be filtered in the image to be processed, including: when the position of each pixel to be filtered in the image to be processed is a texture region, select the first step size as the initial sampling step size; when the position of each pixel to be filtered in the image to be processed is a background region, select the second step size as the initial sampling step size, wherein the first step size is smaller than the second step size. Step 23: Based on the radius of each filtering region, when the radius is not an integer multiple of the initial sampling step size, adjust the initial sampling step size to obtain an adjusted sampling step size, so that the radius is an integer multiple of the adjusted sampling step size, and determine the adjusted sampling step size as the final sampling step size.

2. The fast image filtering method for embedded airborne scenarios according to claim 1, characterized in that, In step 4, the filtering algorithm corresponding to the embedded airborne scenario includes any one of the following: Gaussian filtering algorithm, mean filtering algorithm, maximum filtering algorithm, minimum filtering algorithm, and median filtering algorithm.

3. The fast image filtering method for embedded airborne scenarios according to claim 1, characterized in that, In step 1, the image to be processed includes any one of two-dimensional grayscale images, three-dimensional color images, and high-dimensional data images.

4. A fast image filtering system for embedded airborne scenarios, characterized in that, The system is used to implement the fast image filtering method for embedded airborne scenes as described in any one of claims 1 to 3, comprising: The filtering region determination module is used to determine the filtering region of each pixel to be filtered in the image to be processed, with each pixel to be filtered as the center and a set radius as the filtering radius. The sampling step size determination module is used to adaptively determine the sampling step size of each filtering region based on the gradient information and radius of each filtering region; The sampling module is used to sample each of the filtered regions to obtain multiple sampled pixels using the sampling step size. An embedded platform computing module is used to perform filtering processing on each of the filtering regions by using a filtering algorithm corresponding to the embedded airborne scenario, employing multiple sampled pixels of each filtering region, and outputting the filtered value of each filtering region.

5. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the fast image filtering method for embedded airborne scenes as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that performs a fast image filtering method for an embedded airborne scene according to any one of claims 1 to 3.

Images