Image filtering method, device, equipment, storage medium and program product
By utilizing the cooperation between the filtering data buffer unit and the image filtering unit during the image filtering process, efficient data updating and continuous filtering are achieved, solving the problem of low image filtering efficiency in the prior art, improving the computing speed and reducing latency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GRAVITYXR ELECTRONICS & TECH CO LTD
- Filing Date
- 2024-12-20
- Publication Date
- 2026-06-23
AI Technical Summary
Existing technologies are inefficient in image filtering processes in FPGAs, requiring repeated reading of pixel values at the same location and resulting in low computational efficiency.
An image filtering method is adopted. By cooperating with the image filtering unit and the filtering data buffer unit, the data in the filtering data buffer unit is shifted and updated by using the filtering sliding direction of the image filtering unit, so as to avoid repeated data reading and ensure that the required data is read in each clock cycle.
It improves the operation speed and continuity of image filtering, reduces image filtering delay, and achieves continuity and high efficiency in image filtering.
Smart Images
Figure CN122265042A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to image filtering methods, apparatus, devices, storage media, and program products. Background Technology
[0002] Currently, in the process of image filtering, it is necessary to use the filter to traverse the entire image. In FPGA (Field-Programmable Gate Array) implementation, image buffering is usually implemented using row buffers, that is, each row of the image is stored in the same RAM (Random Access Memory). During the process of the filter traversing the entire image, the pixel value at the same position needs to be read repeatedly and stored in the register. Moreover, depending on the size of the filter, it is usually necessary to retrieve N data from the same row when performing filtering calculations. For a single-port RAM, this requires N clock cycles. Based on this, the efficiency of image filtering calculations is currently low. Summary of the Invention
[0003] This application provides an image filtering method, apparatus, device, storage medium, and program product, to provide a technical solution that can read each pixel data in an image only once and can continuously perform image filtering processing.
[0004] In a first aspect, this application provides an image filtering method, the method comprising:
[0005] According to the filtering sliding direction of the image filtering unit, the data in the filtering data buffer unit is shifted, wherein the size of the filtering data buffer unit and the image filtering unit is the same;
[0006] The data in the filtered data buffer unit is updated based on the current pixel data and historical pixel data in the acquired image data stream;
[0007] The system acquires the data from the shifted and updated filtered data buffer unit, filters the data from the shifted and updated filtered data buffer unit using the image filtering unit, and outputs the filtered image data.
[0008] Optionally, shifting the data in the filter data buffer unit according to the filtering sliding direction of the image filtering unit includes:
[0009] Obtain the filtering sliding direction of the image filtering unit, and determine the direction opposite to the filtering movement direction as the data shifting direction in the filtering data buffer unit;
[0010] The data in the filtered data buffer unit is shifted based on the data shift direction.
[0011] Optionally, shifting the data in the filtered data buffer unit based on the data shift direction includes:
[0012] Based on the data shifting direction, the data in the overlapping area of the filter data buffer unit is shifted; wherein, the overlapping area is the part that overlaps between the next filtering position of the image filter unit in the coverage area of the filter data buffer unit and the current filtering position of the image filter unit in the coverage area of the filter data buffer unit.
[0013] Optionally, shifting the column data in the overlapping area of the filtered data buffer unit includes:
[0014] The data in the overlapping area of the filtered data buffer unit is shifted forward by m columns; where m is greater than or equal to 1 and less than or equal to n-1, and n is the number of columns that the image filtering unit can filter at each time.
[0015] Optionally, updating the data in the filtered data buffer unit based on the current pixel data and historical pixel data in the acquired image data stream includes:
[0016] Based on the current pixel data and historical pixel data in the acquired image data stream, the last m columns of data in the filtering data buffer unit are updated; where m is greater than or equal to 1 and less than or equal to n-1, and n is the number of columns that the image filtering unit can filter each time.
[0017] Optionally, the historical pixel data is the data in the row cache unit;
[0018] Based on the current pixel data and historical pixel data in the acquired image data stream, updating the last m columns of data in the filtered data buffer unit includes:
[0019] Based on the address of the current pixel data, and according to the current pixel data and the historical data corresponding to the address in the row cache unit, the last m columns of data in the filter data cache unit are updated.
[0020] Optionally, the last m columns of data are the last column of data. The row cache unit includes n-1 row caches, which are used to store n-1 rows of data in the image. The n-1 rows are the first n-1 rows of the current filtered data of the image filtering unit, where n is the number of rows that the image filtering unit can filter at each time.
[0021] Based on the address of the current pixel data, and according to the current pixel data and the historical data corresponding to the address in the row cache unit, the last m columns of data in the filter data cache unit are updated, including:
[0022] Based on the current pixel data, update the data corresponding to the last column and last row in the filter data cache unit;
[0023] Based on the address corresponding to the current pixel data, the target data in the last column of the filtered data cache unit is updated according to the data corresponding to the column address of the address in each row buffer; wherein, the row address of the row where the target data is located is the same as the row address of the corresponding row buffer.
[0024] Optionally, the method also includes:
[0025] Based on the current pixel data and the historical data in n-1 row buffers, update the data in n-1 row buffers corresponding to the address of the current pixel data.
[0026] Optionally, based on the current pixel data and the historical data in the n-1 line buffers, updating the data corresponding to the address of the current pixel data in the n-1 line buffers includes:
[0027] Based on the data in the (n-1)th row buffer of the n-1 row buffer that corresponds to the column address of the current pixel data, update the data in the (n-2)th row buffer of the n-1 row buffer that corresponds to the column address of the current pixel data;
[0028] Based on the current pixel data, update the data corresponding to the column address of the current pixel data in the (n-1)th row buffer of the n-1 row buffers.
[0029] Optionally, acquiring the data in the shifted and updated filtered data buffer unit, and filtering the data in the shifted and updated filtered data buffer unit using the image filtering unit includes:
[0030] Obtain the shifted and updated filtered data from the buffer unit and write it into the filter buffer unit;
[0031] The image filtering unit is used to filter the data in the filter buffer unit.
[0032] Optionally, the filtered data buffer unit includes multiple registers arranged in a matrix, each register used to store one pixel data to be filtered.
[0033] Optionally, the current pixel data is pixel data obtained sequentially from the image data stream;
[0034] The image data stream is a data stream formed sequentially based on the row data of the image.
[0035] Secondly, this application provides an image filtering device, including: a filtering data buffer unit, an image filtering unit, and a data processing unit, wherein the filtering data buffer unit and the image filtering unit are the same size.
[0036] The data processing unit is used for:
[0037] According to the filtering sliding direction of the image filtering unit, the data in the filtering data buffer unit is shifted, wherein the size of the filtering data buffer unit and the image filtering unit is the same;
[0038] The data in the filtered data buffer unit is updated based on the current pixel data and historical pixel data in the acquired image data stream;
[0039] The system acquires the data from the shifted and updated filtered data buffer unit, filters the data from the shifted and updated filtered data buffer unit using the image filtering unit, and outputs the filtered image data.
[0040] Thirdly, this application provides an image device, including: a processor, and a memory communicatively connected to the processor;
[0041] The memory stores the instructions that the computer executes.
[0042] The processor executes computer-executable instructions stored in memory to implement the method as described in any of the first aspects.
[0043] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, are used to implement the method as described in any of the first aspects.
[0044] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method as described in any of the first aspects.
[0045] The technical solution provided in this application caches the image data to be filtered in a filtering data cache unit during image filtering. The image data in the filtering data cache unit is continuously updated according to the image data stream during the filtering process, with each update corresponding to one filtering operation of the image filtering unit. Each update of the filtering data cache unit includes shifting the data in the filtering data cache unit according to the filtering sliding direction of the image filtering unit to update a portion of the data in the filtering data cache unit. This operation does not involve acquiring data from the image data stream. Each update of the filtering data cache unit also includes updating based on the current pixel data and historical pixel data in the image data stream. Pixel data is read only once per image filtering process, avoiding repeated data reading. Furthermore, the data required by the image filtering unit can be read within each clock cycle, improving the computational speed and ensuring the continuity of image filtering. Based on this, the image filtering unit can perform continuous filtering during continuous data inflow and outflow, reducing the latency of the image filtering unit and significantly improving the computational speed. Attached Figure Description
[0046] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0047] Figure 1 A flowchart illustrating the steps of an image filtering method provided in this application embodiment;
[0048] Figure 2 A schematic diagram of an overlapping region formed during the sliding process of an image filtering unit provided in an embodiment of this application;
[0049] Figure 3 This is a schematic diagram illustrating the data update process of each unit from the inflow of image data stream to the output of the filtering result, provided as an embodiment of this application.
[0050] Figure 4 A flowchart of an image filtering method provided in an embodiment of this application;
[0051] Figure 5 A schematic diagram illustrating the update process of a filtered data cache unit provided in an embodiment of this application;
[0052] Figure 6 This is a schematic diagram of the structure of an image filtering device provided in an embodiment of this application.
[0053] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0054] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. It is understood that the terms “first,” “second,” etc., as used herein may be used to describe various information or data, but these elements are not limited by these terms. These terms are only used to distinguish first information from another type of information. For example, without departing from the scope of this application, first action information may be referred to as second action information, and similarly, second action information may be referred to as first action information. Both first action information and second action information are action information, but they are not the same action information.
[0056] First, let me explain the terms used in this application:
[0057] FPGA: Field-Programmable Gate Array;
[0058] RAM: Random Access Memory;
[0059] REG: Register, a high-speed storage device in a computer system, used for temporary storage of data and instructions;
[0060] Line buffer: A storage structure used in image and video processing systems to temporarily store one or more lines of image data.
[0061] Filter buffer: filter_buffer, a storage structure used in image processing and video processing systems to temporarily buffer one or more pieces of data to be filtered.
[0062] Image filtering unit: A device or module used to process image signals, which improves image quality or extracts specific image features through various filtering techniques.
[0063] Image data stream: piecel_flow refers to a sequence of image data that flows according to preset rules.
[0064] Current pixel data refers to the pixel information flowing out of the image data stream at the current moment.
[0065] Currently, in the process of image filtering, it is necessary to use the filter to traverse the entire image. In FPGA implementation, image buffering is usually implemented using a row buffer, that is, each row of the image is stored in the same RAM. During the process of the filter traversing the entire image, the pixel value at the same position needs to be read repeatedly and stored in the REG register. Moreover, depending on the size of the filter, it is usually necessary to retrieve N data from the same row when performing filtering calculations. For a single-port RAM, this requires N clock cycles. Based on this, the efficiency of image filtering calculations is currently low.
[0066] Based on this, this application proposes a pipelined image filtering technique. During the image filtering process, the data at the same position in the image data stream only needs to be read once, avoiding repeated data reading. Furthermore, all the data required by the image filtering unit can be obtained in each clock cycle, thereby improving the filtering rate of the image filtering unit and reducing the image filtering delay.
[0067] Firstly, this application proposes an image filtering method, referring to... Figure 1 The method may include the following steps:
[0068] S101, according to the filtering sliding direction of the image filtering unit, the data in the filtering data buffer unit is shifted, wherein the number of filtering data buffer units is the same as the size of the image filtering unit.
[0069] It should be understood that due to imperfections in imaging systems, transmission media, and recording equipment, image data is often contaminated by various types of noise during its formation, transmission, and recording. Therefore, image data filtering is necessary. The aforementioned image filtering unit is used to filter image data to remove noise present in the image data.
[0070] The embodiments of this application can be applied to image processing scenarios, such as super-resolution algorithm processing and high dynamic range image generation.
[0071] Optionally, the image filtering unit described above may include an image filter, such as a nonlinear filter, a median filter, or a morphological filter.
[0072] During image filtering, the image filter slides across the image data, enabling filtering of pixel data at different locations within the image. The size of the image filter can be set according to actual filtering requirements. For example, the image filter size can be set to N*N, where N = 3, 4, or 5. It is worth noting that this application does not impose any special limitations on the size of the image filter.
[0073] Optionally, embodiments of this application can be applied to an FPGA. The aforementioned filtered data buffer unit may include multiple registers arranged in a matrix, each register storing one pixel data to be filtered. The number of registers in the filtered data buffer unit is N*N.
[0074] In this embodiment, the filter data buffer unit is used to buffer the image data to be filtered, that is, the image data required by the image filtering unit for filtering in the next clock cycle. Therefore, the data in the filter data buffer unit can be shifted according to the sliding direction of the image filtering unit on the image to update part of the data in the filter data buffer unit. This operation does not require reading the corresponding pixel data from the image data stream, saving pixel data reading time and improving the filtering efficiency of this embodiment.
[0075] Furthermore, in this embodiment, the number of registers in the filter data cache unit is the same as the size of the image filter unit, so that the filter data cache unit can cache the pixel data required by the image filter unit each time.
[0076] Optionally, in this embodiment of the application, while the image filtering unit is filtering the current image data, a portion of the data in the filter data buffer unit can be shifted according to the current image filtering data to update a portion of the data in the filter data buffer unit, thereby saving the filtering time of the entire image data.
[0077] S102, based on the current pixel data and historical pixel data in the acquired image data stream, update the data in the filtered data cache unit.
[0078] In this embodiment, the remaining data in the filtered data cache unit can be updated based on the current pixel data and historical pixel data in the acquired image data stream, thereby achieving a complete update of the data in the entire filtered data cache unit. The historical pixel data can be data cached in the row cache unit.
[0079] S103: Obtain the data in the shifted and updated filtered data buffer unit, use the image filtering unit to filter the data in the shifted and updated filtered data buffer unit, and output the filtered image data.
[0080] After shifting and updating the data in the entire filtered data buffer unit, the data is obtained, and the image filtering unit is used to filter the shifted and updated data in the filtered data buffer unit to obtain the filtered image data.
[0081] In some examples, the data acquired from the shifted and updated filtered data buffer unit can be written into the filter buffer unit. The number of registers in this filter buffer unit is the same as the size of the image filtering unit. For example, if the image filtering unit is 3x3, then the number of registers in the filter buffer unit is also 3x3.
[0082] Based on the above description, in this embodiment of the application, when performing image filtering, the image data to be filtered is cached in a filtering data cache unit. The image data in the filtering data cache unit is continuously updated during the filtering process, with each update corresponding to one filtering operation of the image filtering unit. Each update of the filtering data cache unit includes shifting the data in the filtering data cache unit according to the filtering sliding direction of the image filtering unit to update a portion of the data in the filtering data cache unit. This operation does not involve acquiring data from the image data stream. Each update of the filtering data cache unit also includes updating based on the current pixel data and historical pixel data in the image data stream. Pixel data is read once per image filtering process, avoiding repeated data reading. Furthermore, all the data required by the image filtering unit can be read within each clock cycle, improving the computational speed and ensuring the continuity of image filtering. Based on this, the image filtering unit can perform continuous filtering during continuous data inflow and outflow, reducing the latency of the image filtering unit and significantly improving the computational speed.
[0083] The above content describes the overall solution of this application. The following content describes in detail each step of the image filtering method provided in the embodiments of this application.
[0084] Specifically, shifting the data in the filter data buffer unit according to the filtering sliding direction of the image filtering unit in step S101 may include:
[0085] S1011, Obtain the filtering sliding direction of the image filtering unit, and determine the direction opposite to the filtering movement direction as the data shifting direction in the filtering data buffer unit.
[0086] Optionally, the filtering direction of the image filtering unit can be the arrangement direction of the pixel data. For example, the filtering direction of the image filtering unit can be set along the arrangement direction of the row data in the image data.
[0087] It should be understood that as the image filtering unit slides across the image data, the filtered data of the image filtering unit is updated accordingly each time. The sliding distance of the image filtering unit each time can be set to slide one pixel or multiple pixel units. It is worth noting that, to improve the effectiveness and reliability of image filtering, assuming the size of the image filtering unit is N*N, the sliding distance of the image filtering unit each time should be less than N pixel units to ensure that the current filtering area of the image filtering unit overlaps with the previous filtering area.
[0088] Based on the above description, embodiments of this application can update a portion of the data in the filtered data buffer unit by shifting the data within it. Specifically, the shift direction can be set to the opposite direction to the filtering movement direction, used to shift the data at the next pixel position in the filtered data buffer unit to the previous pixel position.
[0089] S1012 shifts the data in the filtered data buffer unit based on the data shift direction.
[0090] Based on the foregoing, it has been determined that the data shifting direction in the filter data buffer unit is opposite to the filtering sliding direction of the image filter unit.
[0091] Optionally, when the filtering sliding direction of the image filtering unit is the arrangement direction of the row data in the image, the column data in the filtering data cache unit can be shifted based on this data shift direction to update part of the data in the filtering data cache unit.
[0092] In some examples, step S1012 above may include: shifting the data in the overlapping region of the filtered data buffer unit based on the data shifting direction. This overlapping region can be interpreted as the portion overlapping between the next filtering position of the image filtering unit in the coverage area of the filtered data buffer unit and the current filtering position of the image filtering unit in the coverage area of the filtered data buffer unit.
[0093] Based on this, the data in the overlapping areas of the filtered data buffer unit can be shifted to update a portion of the data in the filtered data buffer unit.
[0094] When the filtering sliding direction of the image filtering unit is the same as the arrangement direction of the row data in the image, this part of the data can be the first few columns of data in the filtering data cache unit.
[0095] In a specific example, refer to Figure 2The next filtering position of the image filtering unit is located in the coverage area S2 of the filter data buffer unit, and the current filtering position of the image filtering unit is located in the coverage area S1 of the filter data buffer unit, with an overlapping area of St. It can be seen that the data in the overlapping area consists of the first two columns of data for the next filtering position of the image filtering unit and the last two columns of data for the current filtering position of the image filtering unit. The filter data buffer unit is used to buffer the data required for the next filtering position of the image filtering unit. The data currently buffered in the filter data buffer unit is the data required for the current filtering position of the image filtering unit. Therefore, when updating the data in the filter data buffer unit, the column data in the overlapping area of the current filter data buffer unit can be shifted in the opposite direction to the filtering sliding direction of the image filtering unit, thereby updating the first few columns of data in the filter data buffer unit.
[0096] Optionally, shifting the data in the overlapping region of the filtered data buffer unit may include:
[0097] The data in the overlapping area of the filtered data buffer unit is shifted forward by m columns. Here, m is greater than or equal to 1 and less than or equal to n-1, where n is the number of columns that the image filtering unit can filter at any given time.
[0098] In this embodiment, m can be the pixel distance that the image filtering unit slides after each image filtering operation during the filtering process. This pixel distance can be represented by the number of pixels. For example, when the pixel distance that the image filtering unit slides after each image filtering operation is 1 pixel, m equals 1.
[0099] For example, when the pixel distance that the image filtering unit slides after each image filtering operation is 2 pixels, then m equals 2.
[0100] It should be understood that, to ensure the accuracy of image filtering, the pixel distance that an image filtering unit slides after each image filtering operation should generally be less than the number of pixels that the image filter can cover in each row of pixels in a single filtering operation. For example, if the number of pixels that the image filter can cover in each row of pixels in a single filtering operation is 3, then the pixel distance that the image filtering unit slides after each image filtering operation can be 1 pixel or 2 pixels.
[0101] For example, if the number of pixels that an image filter can cover in each row of pixels in a single filtering operation is 5, then the distance of the pixels that the image filtering unit slides after each image filtering operation can be any number of pixels from 1 to 4.
[0102] In a specific example, to ensure the accuracy of image filtering, m can be set to 1. Then, shifting the data in the overlapping area of the filtered data buffer unit forward by m columns can be achieved as follows:
[0103] The data in the next column of the filtered data buffer unit is shifted to the data in the previous column of the filtered data buffer unit.
[0104] As mentioned above, the data shift direction is determined to be opposite to the filtering sliding direction of the image filtering unit, which is the row data arrangement direction. Therefore, since the data shift direction is opposite to the row data arrangement direction, updating the data in the filtering data buffer unit can be done by shifting the data in the next column of the filtering data buffer unit to the data in the previous column of the filtering data buffer unit.
[0105] The above content provides a detailed description of step S101. The following content describes step S102.
[0106] Optionally, step S102 above may include:
[0107] S1021, based on the current pixel data and historical pixel data in the acquired image data stream, update the last m columns of data in the filtered data buffer unit.
[0108] As mentioned above, the data in the overlapping area of the filtered data cache unit has been updated. The data in the overlapping area can include the first nm columns of data in the filtered data cache unit. Therefore, the data in other areas of the filtered data cache unit can include the last m columns of data in the filtered data cache unit. Here, n is the number of columns in the filtered data cache unit.
[0109] In this embodiment of the application, the current pixel data can be pixel data obtained sequentially from the image data stream.
[0110] An image data stream can be a data stream formed sequentially based on the row data of an image. The address of each pixel in the image data stream corresponds to its position in the image, i.e., its row and column number.
[0111] For example, Pix(x,y) can be used to represent the pixel with address x,y, where x is the row number of the pixel and y is the column number of the pixel.
[0112] In one example, historical pixel data can be data in a row cache unit; wherein, the row cache unit is used to cache multiple rows of image data, and the current pixel data in the image data stream is the data required for the current filtering. Therefore, the address of the current pixel data can be used to characterize the pixel position of the area to be updated in the filtering data cache unit. Thus, the current pixel data and the data in the row cache unit corresponding to the address can be used to update the last m columns of data in the filtering data cache unit.
[0113] Based on this, step S1021 above may include:
[0114] Based on the address of the current pixel data, and according to the current pixel data and the historical data corresponding to the address of the current pixel data in the row cache unit, the last m columns of data in the filter data cache unit are updated.
[0115] Based on this, it is possible to update all data in the filter data cache unit, and the update process of all data in the filter data cache unit only reads the current pixel data in image data six once, which can avoid repeated reading of data.
[0116] Optionally, to ensure the effectiveness of filtering, the image filtering unit can be set to move one pixel at a time along the row direction of the image. In this case, the unupdated area of the filtering data buffer unit can be the last column of data in the filtering data buffer unit.
[0117] The row buffer unit can be further explained here as comprising n-1 row buffers, which are used to store n-1 rows of data in the image. These n-1 rows are the n-1 rows preceding the current filtering data of the image filtering unit, where n is the number of rows that the image filtering unit can filter at any given time.
[0118] Based on the above description, updating the last m columns of data in the filtered data cache unit based on the address of the current pixel data, the current pixel data, and the historical data corresponding to the address of the current pixel data in the row cache unit can include:
[0119] Step S10211: Update the data corresponding to the last column and last row in the filter data cache unit according to the current pixel data.
[0120] It should be understood that, based on the characteristics of the image data stream, the data at the maximum address position in the filter data cache unit can be updated using the current pixel data. Therefore, it is possible to set the data corresponding to the last column and last row in the filter data cache unit to be updated using the current pixel data.
[0121] Step S10212: Based on the address corresponding to the acquired current pixel data, update the target data in the last column of the filtered data cache unit according to the data corresponding to the column address of the address in each row buffer; wherein, the row address of the row where the target data is located is the same as the row address of the corresponding row buffer.
[0122] As mentioned above, the filtering data buffer unit is used to buffer the data corresponding to the next filtering position of the image filtering unit. Therefore, updating the first m-1 data in the last column of the filtering data buffer unit based on the data corresponding to the column address of the current pixel address in n-1 row buffers can be further interpreted as: updating the target data in the last column of the filtering data buffer unit based on the data corresponding to the column address of the address corresponding to the current pixel data in each row buffer, wherein the row address of the row containing the target data is the same as the row address of the corresponding row buffer.
[0123] The above describes how a single update can be performed on all data in the entire filtered data cache unit.
[0124] In this embodiment, during the process of updating the filter data buffer unit using the current data stream in the image data stream, the data in the above-mentioned row buffer unit is updated simultaneously using the current pixel data in the image data stream. This ensures that the data in the row buffer unit is also updated in real time along with the incoming data in the image data stream, thereby ensuring that the data in the row buffer unit is updated accordingly as the image filter unit slides.
[0125] Based on this, the method provided in this application may further include: updating the data corresponding to the address of the current pixel data in the n-1 line buffers according to the current pixel data and the historical data in the n-1 line buffers.
[0126] More specifically: Updating the data corresponding to the address of the current pixel data in the n-1 line buffers can include:
[0127] Based on the data in the (n-1)th row buffer of the n-1 row buffer that corresponds to the column address of the current pixel data, update the data in the (n-2)th row buffer of the n-1 row buffer that corresponds to the column address of the current pixel data.
[0128] In other words, the data corresponding to the column address of the current pixel data in the previous row buffer is updated using the data in the next row buffer that corresponds to the column address of the current pixel data.
[0129] The update of the data in the last line cache in the line cache unit can be:
[0130] Based on the current pixel data, update the data corresponding to the column address of the current pixel data in the (n-1)th row buffer of the n-1 row buffers.
[0131] In other words, update the data in the last row buffer corresponding to the column address of the current pixel data based on the current pixel data.
[0132] It is worth noting here that the row address of the next row cache, the previous row cache, and the last row cache are determined according to the size of the row address of the row cache. The row address of the next row cache is greater than the row address of the previous row cache, and the last row cache has the largest row address in the row cache unit.
[0133] Based on the above description, real-time updates of the row cache unit can be achieved to ensure that historical data in the row cache can be used to update the filtered data cache unit in real time.
[0134] The following specific embodiment describes how to update the data in the row cache unit and the filtered data cache unit.
[0135] Reference Figure 3 and Figure 4 The image filtering method in this embodiment is based on an FPGA. Assuming the image filtering unit size is N*N, this embodiment requires instantiating n-1 row buffers (row buffer_0, row buffer_1...row buffer_n-2), a filtering data buffer unit (including N*N registers), and a filtering buffer. The N-1 row buffers store the data of the first N-1 rows of the image, and the N*N registers buffer all the image data needed for each filtering calculation. The image data stream is the inflow channel for the image data. The filtering buffer stores the data updated by the filtering data buffer unit each time. As the image data stream flows in, the values of the row buffers and registers (N-1, N-1) are updated one by one until the entire image data input is completed. Among them, the value of register (*, N-1) is updated by the row buffers or the image data stream, and the values of the other registers are updated by the previous register value. Each update only requires one clock cycle to complete. After the update is completed, the data is written into the corresponding filtering buffer for use by the subsequent image filtering unit. The N*N independent filter buffers ensure that all the data required for calculation can be read within one clock cycle in this embodiment, and the updates of the row buffer and registers are also completed within one clock cycle, and finally the filtering results are continuously output.
[0136] For step S103, obtaining the updated data in the filtered data cache unit and filtering the updated data in the filtered data cache unit using the image filtering unit may include:
[0137] S1031: Obtain the updated data from the filter data cache unit and write it into the filter buffer unit.
[0138] The size of the filter buffer unit is the same as that of the filter data buffer unit and the image filter unit. It can be used to store the data after the filter data buffer unit has been updated. Then, the filter data buffer unit continues to be updated continuously according to the data in the image data stream to ensure the continuous filtering of the image filter unit and improve the efficiency of image filtering.
[0139] S1032 uses the image filtering unit to filter the data in the filtering buffer unit to form a continuous filtering process until the entire image data input is completed.
[0140] In a specific instance, refer to Figure 5 Assuming the image filtering unit is 3*3 in size, the filtering data buffer unit includes 3*3 arrayed registers, there are 2 row buffers, and the number of pixels in the filtering buffer unit is also 3*3. The image filtering process is the process of the image filtering unit sliding on the image. When the current pixel in the image data stream is image data with coordinates (x,y), row buffer_0[y] updates register (0,2) in the current filtering data buffer unit, row buffer_1[y] updates row buffer_0[y] and register (1,2) in the current filtering data buffer unit, the current pixel data in the image data stream updates row buffer_1[y] and register (2,2) in the current filtering data buffer unit, and at the same time, register (2,1) in the previous filtering data buffer unit is updated with register (2,2), register (2,1) is updated with register (2,0), and the updates of the remaining registers are similar. In other words, the first column of data in the filtering data buffer unit is updated using the second column of data, and the second column of data is updated using the third column of data. At the same time, the first row of data in the last column of the filtering data buffer unit is updated using the data in the first row buffer corresponding to the column address of the current pixel address. The second row of data in the last column of the filtering data buffer unit is updated using the data in the second row buffer corresponding to the column address of the current pixel address. The last row of data in the last column of the filtering data buffer unit is updated using the current pixel. All of the above data updates are completed within the same clock cycle. After the filtering data buffer unit completes the update, the data can be written to the filtering buffer unit for use by the image filtering unit during filtering.
[0141] Table 1 compares the time required for a single filtering operation with that required in the prior art, where the prior art solution is based on the solution in the background art of this application.
[0142]
[0143] It can be seen that the method provided in this application has greatly improved the filtering rate compared with related technologies.
[0144] In this embodiment, the method can be run in an FPGA. Based on the parallel computing characteristics of the FPGA, image filtering calculations are performed in a pipelined manner. During the calculation process, image data at the same position only needs to be read from the row buffer once, avoiding repeated data reading. Furthermore, the calculation module can retrieve all the data required for calculation in each clock cycle. For an N*N filter, the operation speed is increased by N times, ensuring that the calculation can be performed continuously with continuous data inflow and outflow. Due to this pipelined design, the operation speed of the image filtering unit is greatly improved, reducing the latency of the entire module.
[0145] Based on the above description, the image filtering method provided in this application embodiment can update all the data required by the image filtering unit within one clock cycle, ensuring that the image filtering unit is not idle, the filtering process is continuous, and the filtering results can be continuously output, thereby improving filtering efficiency, reducing latency, and enhancing real-time performance.
[0146] Secondly, embodiments of this application also provide an image filtering device, including: a filtering data buffer unit, an image filtering unit, and a data processing unit, wherein the filtering data buffer unit and the image filtering unit are of the same size.
[0147] The data processing unit is used for:
[0148] According to the filtering sliding direction of the image filtering unit, the data in the filtering data buffer unit is shifted, wherein the size of the filtering data buffer unit and the image filtering unit is the same;
[0149] The data in the filtered data buffer unit is updated based on the current pixel data and historical pixel data in the acquired image data stream;
[0150] The updated data in the filter data buffer unit is obtained, the image filtering unit is used to filter the data in the updated filter data buffer unit, and the filtered image data is output.
[0151] Optionally, the data processing unit is specifically used to: obtain the filtering sliding direction of the image filtering unit, and determine the direction opposite to the filtering movement direction as the data shifting direction in the filtering data buffer unit.
[0152] The data in the filtered data buffer unit is shifted based on the data shift direction.
[0153] Optionally, the data processing unit is further configured to: shift the data in the overlapping region of the filter data buffer unit based on the data shifting direction; wherein the overlapping region is the part that overlaps between the next filtering position of the image filtering unit in the coverage area of the filter data buffer unit and the current filtering position of the image filtering unit in the coverage area of the filter data buffer unit.
[0154] Optionally, the data processing unit is further specifically used to: shift the data of the overlapping area in the filtered data buffer unit forward by m columns; where m is greater than or equal to 1 and less than or equal to n-1, and n is the number of columns that the image filtering unit can filter each time.
[0155] Optionally, the data processing unit is specifically used to: update the last m columns of data in the filtered data cache unit based on the current pixel data and historical pixel data in the acquired image data stream; where m is greater than or equal to 1 and less than or equal to n-1, and n is the number of columns that the image filtering unit can filter each time.
[0156] Optionally, the historical pixel data is the data in the row cache unit;
[0157] The data processing unit is specifically used to update the last m columns of data in the filtered data cache unit based on the address of the current pixel data, the current pixel data, and the historical data corresponding to the address in the row cache unit.
[0158] Optionally, the last m columns of data are the last column of data. The row cache unit includes n-1 row caches, which are used to store n-1 rows of data in the image. The n-1 rows are the first n-1 rows of the current filtered data of the image filtering unit, where n is the number of rows that the image filtering unit can filter at each time.
[0159] The data processing unit is also specifically used for:
[0160] Based on the current pixel data, update the data corresponding to the last column and last row in the filter data cache unit;
[0161] Based on the address corresponding to the current pixel data, the target data in the last column of the filtered data cache unit is updated according to the data corresponding to the column address of the address in each row buffer; wherein, the row address of the row where the target data is located is the same as the row address of the corresponding row buffer.
[0162] Optionally, the data processing unit is also specifically used to: update the data corresponding to the address of the current pixel data in the n-1 line buffers based on the current pixel data and the historical data in the n-1 line buffers.
[0163] Optionally, the data processing unit is further configured to: update the data corresponding to the column address of the current pixel data in the (n-2)th row buffer of the n-1 row buffers based on the data corresponding to the column address of the current pixel data in the (n-1)th row buffer of the n-1 row buffers;
[0164] Based on the current pixel data, update the data corresponding to the column address of the current pixel data in the (n-1)th row buffer of the n-1 row buffers.
[0165] Optionally, the data processing unit is also specifically used to: obtain the updated data in the filtered data cache unit and write it into the filtered buffer unit.
[0166] The image filtering unit is used to filter the data in the filter buffer unit.
[0167] Optionally, the filtered data buffer unit includes multiple registers arranged in a matrix, each register used to store one pixel data to be filtered.
[0168] Optionally, the current pixel data is pixel data obtained sequentially from the image data stream.
[0169] The image data stream is a data stream formed sequentially based on the row data of the image.
[0170] It should be understood that the beneficial effects of the image filtering device in the second aspect are the same as those of the image filtering method in the first aspect, and will not be repeated here.
[0171] Thirdly, this application also provides an image filtering device. Figure 6 This is a schematic diagram of the structure of an image filtering device provided in an embodiment of this application. Figure 6 As shown, the image filtering device may include: transceiver 121, processor 122, and memory 123.
[0172] Processor 122 executes computer execution instructions stored in memory, causing processor 122 to perform the multi-channel data synchronization method provided in the above embodiments. Processor 122 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital data processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0173] The memory 123 is connected to the processor 122 via the system bus and completes communication between them. The memory 123 is used to store computer program instructions.
[0174] Transceiver 121 can be used to obtain the task to be run and its configuration information.
[0175] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. Transceivers are used to enable communication between database access devices and other computers (e.g., clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and may also include non-volatile memory.
[0176] This application also provides a chip for executing instructions, which is used to execute the image filtering method described in the above embodiments.
[0177] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to perform the image filtering method described in the above embodiments.
[0178] Fifthly, embodiments of this application also provide a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when the at least one processor executes the computer program, it can implement the technical solution of the image filtering method in the above embodiments.
[0179] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.
[0180] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.
[0181] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit composed of the above modules can be implemented in hardware or in the form of hardware plus software functional units.
[0182] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.
[0183] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.
[0184] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.
[0185] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.
[0186] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.
[0187] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. The processor and storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic control unit or main control device.
[0188] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.
[0189] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.
Claims
1. An image filtering method, characterized in that, The method includes: According to the filtering sliding direction of the image filtering unit, the data in the filtering data buffer unit is shifted, wherein the filtering data buffer unit and the image filtering unit are the same size; Based on the current pixel data and historical pixel data in the acquired image data stream, the data in the filtered data cache unit is updated; The data in the shifted and updated filtered data buffer unit is obtained, the image filtering unit is used to filter the data in the shifted and updated filtered data buffer unit, and the filtered image data is output.
2. The image filtering method according to claim 1, characterized in that, The step of shifting the data in the filter data buffer unit according to the filtering sliding direction of the image filtering unit includes: Obtain the filtering sliding direction of the image filtering unit, and determine the direction opposite to the filtering movement direction as the data shifting direction in the filtering data buffer unit; Based on the data shift direction, the data in the filtered data buffer unit is shifted.
3. The image filtering method according to claim 2, characterized in that, Shifting the data in the filtered data buffer unit based on the data shift direction includes: Based on the data shifting direction, the data in the overlapping area of the filter data buffer unit is shifted; wherein, the overlapping area is the part where the next filtering position of the image filtering unit overlaps with the current filtering position of the image filtering unit in the coverage area of the filter data buffer unit.
4. The image filtering method according to claim 3, characterized in that, Shifting the data in the overlapping region of the filtered data buffer unit includes: The data in the overlapping area of the filtered data buffer unit is shifted forward by m columns; where m is greater than or equal to 1 and less than or equal to n-1, and n is the number of columns that the image filtering unit can filter each time.
5. The image filtering method according to claim 1, characterized in that, The step of updating the data in the filtered data cache unit based on the current pixel data and historical pixel data in the acquired image data stream includes: Based on the current pixel data and historical pixel data in the acquired image data stream, the last m columns of data in the filtering data cache unit are updated; where m is greater than or equal to 1 and less than or equal to n-1, and n is the number of columns that the image filtering unit can filter each time.
6. The method according to claim 5, characterized in that, The historical pixel data is the data in the row cache unit; Updating the last m columns of data in the filtered data cache unit based on the current pixel data and historical pixel data in the acquired image data stream includes: Based on the address of the current pixel data, and according to the current pixel data and the historical data corresponding to the address in the row cache unit, the last m columns of data in the filter data cache unit are updated.
7. The method according to claim 6, characterized in that, The last m columns of data are the last column of data. The row cache unit includes n-1 row caches, which are used to store n-1 rows of data in the image. The n-1 rows are the first n-1 rows of the row where the current filtered data of the image filtering unit is located, where n is the number of rows that the image filtering unit can filter at each time. Based on the address of the current pixel data, updating the last m columns of data in the filtered data cache unit according to the current pixel data and the historical data corresponding to the address in the row cache unit includes: Update the data corresponding to the last column and last row in the filtered data cache unit based on the current pixel data; Based on the address corresponding to the current pixel data, the target data in the last column of the filtered data cache unit is updated according to the data in each row cache corresponding to the column address of the address; wherein, the row address of the row where the target data is located is the same as the row address of the corresponding row cache.
8. The method according to claim 7, characterized in that, The method further includes: Based on the current pixel data and the historical data in the n-1 row buffers, update the data in the n-1 row buffers corresponding to the address of the current pixel data.
9. The method according to claim 8, characterized in that, Updating the data corresponding to the address of the current pixel data in the n-1 row buffers based on the current pixel data and the historical data in the n-1 row buffers includes: Based on the data in the (n-1)th row buffer of the n-1 row buffer that corresponds to the column address of the current pixel data, update the data in the (n-2)th row buffer of the n-1 row buffer that corresponds to the column address of the current pixel data; Based on the current pixel data, update the data in the (n-1)th row buffer of the n-1 row buffer that corresponds to the column address of the current pixel data.
10. The image filtering method according to any one of claims 1-9, characterized in that, The step of acquiring the shifted and updated data in the filtered data cache unit, and then filtering the shifted and updated data in the filtered data cache unit using the image filtering unit, includes: Obtain the shifted and updated data from the filtered data cache unit and write it into the filtered buffer unit; The image filtering unit is used to filter the data in the filtering buffer unit.
11. The image filtering method according to any one of claims 1-9, characterized in that, The filtered data buffer unit includes multiple registers arranged in a matrix, each register being used to store one pixel data to be filtered.
12. The image filtering method according to any one of claims 1-9, characterized in that, The current pixel data is the pixel data obtained sequentially from the image data stream; The image data stream is a data stream formed sequentially based on the row data of the image.
13. An image filtering device, characterized in that, include: The system includes a filtered data buffer unit, an image filtering unit, and a data processing unit, wherein the filtered data buffer unit and the image filtering unit are of the same size. The data processing unit is used for: According to the filtering sliding direction of the image filtering unit, the data in the filtering data buffer unit is shifted, wherein the filtering data buffer unit and the image filtering unit are the same size; Based on the current pixel data and historical pixel data in the acquired image data stream, the data in the filtered data cache unit is updated; The data in the shifted and updated filtered data buffer unit is obtained, the image filtering unit is used to filter the data in the shifted and updated filtered data buffer unit, and the filtered image data is output.
14. An image filtering device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-12.
15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-12.
16. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-12.