Image processing method, device and chip
By dividing the image into multiple image blocks and using the path of the DDR module and SRAM space, the problem of large SRAM area during high-resolution image processing in traditional methods is solved, and more efficient data storage and processing is achieved.
Patent Information
- Application Number
- CN202411543672.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-31
- Publication Date
- 2025-02-18
AI Technical Summary
When traditional image processing methods process high-resolution images, they require a large amount of SRAM space to store the statistical values of the image segmentation area, resulting in a large area of SRAM.
By dividing the image to be processed into a first image block and a second image block, and obtaining the statistical value of each pixel point respectively, the paths of the DDR module and the SRAM space are used to coordinate data flow and avoid data backlogs.
It effectively reduces the area of SRAM when processing high-resolution images, improves data storage efficiency, avoids data backlog, and improves the stability and efficiency of image processing.
Smart Images
Figure CN119477666A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of image processing technology, and in particular, relates to an image processing method, device and chip. Background Art
[0002] With the widespread use of visual electronic products, image processing has become an indispensable part of various fields. For high-resolution images, the traditional image processing algorithm divides the image into two parts, processes them one by one, and then stitches them together to output the whole image. However, the higher the resolution of the image, the higher the storage cost required. This defect is more obvious in the image statistics module.
[0003] Image statistics will first divide the image into different areas, then extract useful information from each area, and finally pass the statistical information of each area to the next module in the order of division from left to right and from top to bottom. If the position of the image blocks just divides the same image segmentation area into two different blocks, the statistical value of the other block will also be used when processing one of the blocks. The current common practice is to open up enough static random access memory (SRAM, Static Random-Access Memory) space internally, a type of semiconductor memory used to store data, usually used to build CPU cache and other high-speed memories), store the statistical values of all image segmentation areas, and then read the statistical values of another block from the SRAM when processing one of the images, and then superimpose the statistical values of the two, and finally output them in order. This approach has the defect that the larger the image segmentation area, the larger the area occupied by the SRAM. Summary of the invention
[0004] The present application provides an image processing method, device and chip, which can reduce the occupied area of SRAM.
[0005] In a first aspect, the present application provides an image processing method, which is applied to an image processing system, the image processing system includes a DDR module, and the image processing method includes:
[0006] The image to be processed is divided into a first image block and a second image block; the first image block and the second image block each include a plurality of rows of pixels, and each row of pixels in the plurality of rows of pixels includes a plurality of pixels;
[0007] The statistical values of each pixel point of the first image block and the second image block are respectively obtained, and a path for writing the statistical values into the DDR module is determined; wherein the path includes writing into the DDR module through the SRAM space, and the SRAM space can store statistical information of a preset data volume.
[0008] Optionally, the image processing system further includes a DMA module and an image statistics module provided with an SRAM space; the first image block and the second image block each include a plurality of segmented regions, each segmented region includes a plurality of rows of pixel points; respectively obtaining a statistical value of each pixel point of the first image block and the second image block, and determining a path for writing the statistical value into the DDR module, including:
[0009] Input each pixel point of the first image block and the second image block into the image statistics module in sequence, and obtain the statistical value of each pixel point input through the image statistics module;
[0010] According to the arrangement order of the pixels, one pixel is taken as the current pixel in turn;
[0011] The path for writing the statistical value of the segmented area to which the current pixel belongs into the DDR module is determined according to the back pressure information sent by the DMA module.
[0012] Optionally, determining a path for writing the statistical value of the segmented area to which the current pixel belongs into the DDR module according to the back pressure information sent by the DMA module includes:
[0013] S1, determine whether the current pixel point meets the preset writing condition, if the current pixel point meets the preset writing condition, execute S2, otherwise, execute S3;
[0014] S2, judging whether the DDR module can receive data according to the back pressure information sent by the DMA module, if the DDR module can receive data, the statistical value of the segmented area to which the current pixel belongs is directly written into the DDR module; otherwise, when the amount of data in the SRAM space does not reach the preset amount of data, the statistical value of the segmented area to which the current pixel belongs is sequentially written into the SRAM space until the DDR module can receive data, and then the statistical value is written into the DDR module through the SRAM space;
[0015] S3, determine whether the current pixel point meets the preset storage conditions. If the current pixel point meets the preset storage conditions, write the statistical value into the SRAM space, and use the next pixel point as the current pixel point in S1, and return to execute S1; otherwise, write the statistical value into the register module, and use the next pixel point as the current pixel point to execute S1 until the current pixel point meets the preset storage conditions.
[0016] Optionally, determining whether the current pixel satisfies a preset writing condition includes:
[0017] If the current pixel is the last pixel of a segmented area, or the pixel in the last row of a segmented area and the last column of the current image block, it is determined that the current pixel satisfies the preset writing condition; otherwise, it is determined that the current pixel does not satisfy the preset writing condition.
[0018] Optionally, determining whether the current pixel satisfies a preset storage condition includes:
[0019] If the pixel corresponding to the current pixel is the last column of pixels in a segmented area, or the last column of pixels in the current image block, it is determined that the current pixel meets the preset storage condition; otherwise, it is determined that the current pixel does not meet the preset storage condition.
[0020] Optionally, respectively obtaining a statistical value of each pixel point of the first image block and the second image block includes:
[0021] Obtain the statistical value of each pixel point of the first image block. After obtaining the statistical value of each pixel point of the first image block and writing it to the DDR module, obtain the statistical value of each pixel point of the second image block, and read the statistical values of the pixel points in the first image block in a preset order and rewrite them into the DDR module.
[0022] Optionally, obtaining the statistical value of each pixel point of the second image block, and reading the statistical values of the pixel points in the first image block in a preset order and rewriting them into the DDR module includes:
[0023] For each pixel of the second image block, perform the following steps:
[0024] If the segmented area to which the pixel point belongs is completely divided into blocks, while obtaining the statistical values of the pixel points of the second image block row by row, the statistical values of the pixel points of the corresponding row in the first image block are read row by row from the DDR module in a preset order and rewritten into the DDR module; wherein the image to be processed includes multiple segmented areas; the complete block refers to the first image block and the second image block not containing the same segmented area of the image to be processed;
[0025] If the segmented area to which the pixel point belongs is incompletely divided, when processing the pixel points of the second image block row by row, it is necessary to first read the statistical values of the last segmented area of the corresponding row of the first image block from the DDR module according to the preset order, and then obtain the statistical values of the pixel points of the corresponding row of the second image block based on the read statistical values and rewrite the statistical values of the pixel points of the corresponding row of the first image block into the DDR module; wherein, the incomplete blocking situation refers to different parts of the same segmented area of the image to be processed contained in the first image block and the second image block.
[0026] Optionally, the preset data amount is smaller than the data amount corresponding to the pixel points in the first image block.
[0027] In a second aspect, the present application provides an image processing device, comprising:
[0028] An image division module, used for dividing the image to be processed into a first image block and a second image block, wherein the first image block and the second image block each include a plurality of rows of pixels, and each row of pixels in the plurality of rows of pixels includes a plurality of pixels;
[0029] An image processing module is used to obtain the statistical value of each pixel point of the first image block and the second image block respectively, and determine the path for writing the statistical value into the DDR module; wherein the path includes writing into the DDR module through the SRAM space, and the SRAM space can store the statistical information of a preset data volume.
[0030] In a third aspect, the present application provides an image processing chip, comprising the above-mentioned image processing device.
[0031] In a fourth aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned image processing method when executing the above-mentioned computer program.
[0032] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program implements the above-mentioned image processing method when executed by a processor.
[0033] The above solution of the present application has the following beneficial effects:
[0034] The image processing method provided by the present application divides the image to be processed into a first image block and a second image block, and then obtains the statistical value of each pixel point of the first image block and the second image block respectively; wherein the path includes writing into the DDR module through the SRAM space, and the SRAM space can store the statistical information of the preset data volume. The opening of the SRAM space can store the statistical information, realize the subsequent processing of the image, determine the path to write into the DDR module, coordinate the data flow between the modules, avoid the backlog of a large amount of data in the SRAM module, improve the data storage efficiency, and can effectively control and reduce the occupied area of the SRAM when processing high-resolution images.
[0035] Other beneficial effects of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0037] Figure 1 A flowchart of an image processing method provided by an embodiment of the present application;
[0038] Figure 2 A schematic diagram of satisfying a preset writing condition provided in an embodiment of the present application;
[0039] Figure 3 A schematic diagram of a case where a preset writing condition is not satisfied provided by an embodiment of the present application;
[0040] Figure 4 A schematic diagram of a flow chart of whether to write into a DDR module according to an embodiment of the present application;
[0041] Figure 5 A schematic diagram of an execution flow of determining whether to write into a register provided in an embodiment of the present application;
[0042] Figure 6 A schematic diagram of an execution flow of determining a statistical value writing path provided by an embodiment of the present application;
[0043] Figure 7 A schematic diagram of the structure of an image processing device provided in one embodiment of the present application;
[0044] Figure 8 A schematic diagram of the structure of an image processing chip provided in one embodiment of the present application. DETAILED DESCRIPTION
[0045] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.
[0046] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.
[0047] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0048] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.
[0049] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.
[0050] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.
[0051] In view of the problem that the SRAM space occupies a large area when processing high-resolution images by traditional methods, the present application provides an image processing method, which divides the image to be processed (dividing the image to be processed into a first image block and a second image block), and then obtains the statistical value of each pixel point of the first image block and the second image block respectively; wherein the path includes writing to the DDR module through the SRAM space, and the SRAM space can store the statistical information of the preset data volume. Opening up the SRAM space can store the statistical information, realize the subsequent processing of the image, determine the path to write to the DDR module, coordinate the data flow between the modules, avoid a large amount of data backlog in the SRAM module, improve the data storage efficiency, and effectively control and reduce the area occupied by the SRAM when processing high-resolution images.
[0052] The image processing method provided by this application is described below.
[0053] In an embodiment of the present application, the image processing method is applied to a designed image processing system, which includes a DDR module (Double Data Rate, a memory module used in a computer system, which adopts a double data transfer rate technology, that is, two data transfers are performed in each clock cycle), a DMA module (Direct Memory Access), a register module, and an image statistics module. Among them, the image statistics module is designed with an SRAM space that can only store statistical information of a preset amount of data.
[0054] Among them, DDR module is a memory module used to store and read data. It provides high-speed data transfer rate and is usually used as main memory in computer systems. DDR module can improve system performance by increasing the data transfer rate.
[0055] The DMA module is a hardware device for high-speed data transmission. It allows peripheral devices (such as hard disk drives, network cards, etc.) to directly access system memory without CPU intervention. The DMA module can improve the efficiency of data transmission and reduce the burden on the CPU. In an embodiment of the present application, the DMA module is used to send back pressure information to indicate whether the DDR module can receive data. If the DDR module cannot receive data, the data needs to be back-pressed (i.e., the device sending data needs to be current-limited or data transmission stopped).
[0056] The register module is a high-speed storage component with limited storage capacity that can be used to temporarily store instructions, data and addresses.
[0057] The image statistics module is a module for analyzing and processing images. It can extract various features and statistical information from the image, such as the brightness, contrast, histogram, etc. of the image. The image statistics module can be used in the fields of image processing, computer vision, pattern recognition, etc., for analyzing and understanding the image content; in addition, the present invention is designed to set an SRAM space for storing statistical information of a preset data amount inside the image statistics module, thereby effectively controlling and reducing the occupied area of the SRAM when processing high-resolution images.
[0058] like Figure 1 As shown, the image processing method comprises the following steps:
[0059] Step 11: divide the image to be processed into a first image block and a second image block.
[0060] The image to be processed may be a part of a complete image.
[0061] Specifically, the first image block and the second image block each include a plurality of rows of pixels, and each row of pixels in the plurality of rows of pixels includes a plurality of pixels.
[0062] Exemplarily, in one embodiment of the present application, the process of dividing the image to be processed is as follows:
[0063] First, determine the size of the image block.
[0064] In the embodiment of the present application, the sizes of the first image block and the second image block are both determined to be image blocks including M rows and N columns (half of the image to be processed) of pixels.
[0065] Then, traverse the image and divide it into blocks.
[0066] Starting from the upper left corner of the image to be processed, the entire image is traversed with the image block size as the step size. For the pixels in the window, each pixel is taken as the current pixel in turn, starting with the pixel in the first row and first column in the window, and the current pixel and the pixels around it are extracted. For example, for the pixel (1,1), the pixel and all the adjacent pixels of the pixel are taken as the pixels in the same image block, and then the pixel (1,2) is taken as the current pixel. When an image with a size of M rows and N columns of pixels is formed, it is recorded as an image block, and the first image block and the second image block are obtained.
[0067] It should be noted that, in the embodiment of the present application, the first image block and the second image block both include multiple segmented areas, each segmented area includes multiple rows of pixels, and the above-mentioned preset data amount is less than the data amount corresponding to the pixels in the first image block. The specific data amount depends on the actual use requirements, as long as the SRAM space size can be controlled by controlling the number of data stored in the SRAM, thereby reducing the area cost caused by storing statistical information corresponding to the pixels. For example, the preset data amount is the data amount corresponding to a row of segmented areas of the first image block. If the first image block is 6 rows and 12 columns, including 6×3 segmented areas, that is, each row of the first image block can include 6 segmented areas (each segmented area is 2 rows and 2 columns relative to itself), then the preset data amount is the data amount corresponding to 6 segmented areas.
[0068] It is worth mentioning that by dividing the image to be processed into a first image block and a second image block, the data of the image to be processed can be segmented, which can reduce the storage cost of storing a single image block.
[0069] Step 12, respectively obtain the statistical value of each pixel point of the first image block and the second image block, and determine the path for writing the statistical value into the DDR module; wherein the path includes writing into the DDR module through the SRAM space, and the SRAM space can store the statistical information of a preset data volume.
[0070] The above path also includes writing directly to the DDR module.
[0071] The above statistical values include the average brightness value of the pixel and the RGB channel mean (the mean of the pixel values of the R channel, G channel, and B channel).
[0072] The following describes the process of respectively obtaining the statistical value of each pixel point of the first image block and the second image block and determining the path for writing the statistical value into the DDR module, which specifically includes:
[0073] In the first step, each pixel point of the first image block and the second image block is input into an image statistics module in turn, and a statistical value of each input pixel point is obtained through the image statistics module.
[0074] In an embodiment of the present application, the above statistical values include the average brightness value of the pixel points and the mean value of the RGB channels.
[0075] In the second step, one pixel is taken as the current pixel in turn according to the arrangement order of the pixels.
[0076] The third step is to determine the path for writing the statistical value of the segmented area to which the current pixel belongs into the DDR module according to the back pressure information sent by the DMA module. This step specifically includes the following steps S1 to S3:
[0077] S1, determine whether the current pixel meets the preset writing condition. If the current pixel meets the preset writing condition, execute S2, otherwise, execute S3.
[0078] In an embodiment of the present application, the process of determining whether the current pixel point meets the preset writing conditions is as follows: if the current pixel point is the last pixel point of a segmented area, or is the pixel point of the last row of a segmented area and the last column of the first image block, then it is determined that the current pixel point meets the preset writing conditions; otherwise, it is determined that the current pixel point does not meet the preset writing conditions.
[0079] The above-mentioned preset writing condition is used to determine whether all the pixel points in the segmented area have been counted, and provides a judgment on whether the segmented area is written into the DDR module, thereby improving the logic and rationality of the image processing method of the present application.
[0080] By taking pixel points as current pixel points in turn and determining the path for writing the statistical value corresponding to the current pixel point into the DDR module according to the back pressure information, the situation where statistical values are omitted is reduced and the stability of the image processing method is improved. By determining the path according to the back pressure information, the rationality of the path for writing the statistical value into the DDR module is improved, thereby improving the stability and rationality of the image processing method in practical applications.
[0081] This step is illustrated below with reference to a specific example.
[0082] Taking the segmented area A of the first image block as an example, the pixel points of the segmented area A satisfy the preset writing condition as follows: Figure 2 As shown, the segmented area A includes two rows of pixels, each row includes 5 pixels, the numbers are the numbers of the pixels, the 10th pixel is taken as the current pixel, the current pixel is the last pixel of the segmented area, and the preset writing condition is met; a situation in which the pixels of the segmented area A do not meet the preset writing condition is as follows Figure 3 As shown, the first pixel is used as the current pixel. The current pixel is not the last pixel of the segmented area and does not meet the preset writing condition.
[0083] S2, judge whether the DDR module can receive data according to the back pressure information sent by the DMA module. If the DDR module can receive data, the statistical value of the segmented area to which the current pixel belongs is directly written into the DDR module; otherwise, when the amount of data in the SRAM space does not reach the preset data amount, the statistical value of the segmented area to which the current pixel belongs is written into the SRAM space in sequence until the DDR module can receive data, and then the statistical value is written into the DDR module through the SRAM space.
[0084] It should be noted that when the amount of data in the SRAM space reaches a preset amount of data, data back pressure is performed to wait for the DDR module to receive data and write the data stored in the SRAM space into the DDR module.
[0085] When the back pressure information (including transmission pause information, idle state information, and data reception ready information) sent by the DMA module reflects that the DDR module cannot receive data, the multiple statistical values obtained are written into the SRAM space, and when the bus is idle, the obtained statistical values are written into the DDR module. Since the SRAM space is pre-set to a preset data volume, and the image statistics module obtains statistical values line by line, the data volume corresponding to the multiple statistical values obtained at this time does not exceed the storage range of the SRAM space.
[0086] for Figure 2 The pixel points of the segmented area A shown in FIG. 1 meet the preset writing conditions and execute the process of this step as shown in FIG. Figure 4 As shown, if the DDR module can receive data, the statistical value of the partition area A is written into the DDR module; if the DDR module cannot receive data, the statistical value of the partition area A is written into the SRAM space.
[0087] S3, determine whether the current pixel point meets the preset storage conditions. If the current pixel point meets the preset storage conditions, write the statistical value into the SRAM space, and use the next pixel point as the current pixel point in S1, and return to execute S1; otherwise, write the statistical value into the register module, and use the next pixel point as the current pixel point to execute S3 until the current pixel point meets the preset storage conditions.
[0088] It should be noted that in the process of obtaining the statistical value of the current pixel point that meets the preset storage conditions, since this process is a process of accumulating statistical values, in this process, the statistical values of the pixel points that do not meet the preset storage conditions are temporarily stored in the register module to participate in the process of accumulating statistical values.
[0089] Specifically, the process of determining whether the current pixel point meets the preset storage conditions is as follows: if the pixel point corresponding to the current pixel point is the last column of pixel points in a segmented area, or the last column of pixel points in the first image block, then it is determined that the current pixel point meets the preset storage conditions; otherwise, it is determined that the current pixel point does not meet the preset storage conditions.
[0090] for Figure 3 The pixel points of the segmented area A do not meet the preset writing condition and the process of executing S3 is as follows: Figure 5 As shown, the current pixel point is the first pixel point, and it is determined whether the pixel point meets the preset storage condition. If the first pixel point does not meet the preset storage condition, the statistical value is written into the register module, and the next pixel is used as the current pixel point, and the step of determining whether the preset writing condition is met is returned. If the current pixel point position does not meet the preset writing condition, the step of determining whether the preset storage condition is met is performed until the current pixel point is the fifth pixel point and meets the preset storage condition. The accumulated statistical value of the first five pixel points is written into the SRAM space, and the next pixel is used as the current pixel point, and the execution of S1 in the above text is returned.
[0091] It should be noted that in the embodiment of the present application, the image statistics module obtains the statistical values corresponding to each pixel point line by line. For example, the segmented area corresponding to the first image block includes two rows of pixel points P1 and P2. Where P1 = {P11, P12, ..., P1 i , ..., P1 I}, P2={P21, P22,..., P2 i , ..., P2 I}, then input P1 first, the image statistics module starts from pixel P11, and obtains the statistical values corresponding to each pixel in P1 one by one (which can be the average brightness value and RGB channel mean of the pixel). When the last pixel P1 in P1 is obtained I After obtaining the statistical value, the statistical value of P1 in the segmented area corresponding to the first image block is sent to the DDR module, and then input to P2. The process of obtaining the statistical value is the same as above. I The statistical value is for all pixels in P1, that is, the average brightness value or RGB channel mean of all pixels in P1.
[0092] Through the above-mentioned preset storage conditions, the judgment of whether the statistical values of the segmented area are written into the SRAM space or into the register module is given. The participation of the register module reduces the storage pressure of the SRAM space, further improves the logic and rationality of the image processing method of the present application, and enables the statistical values in different situations to be stored in the DDR module in an orderly manner, thereby reducing congestion and improving the image processing speed.
[0093] Specifically, the process of sequentially inputting each pixel point of the first image block and the second image block into the image statistics module and obtaining the statistical value of each pixel point input by the image statistics module is as follows:
[0094] The statistical value of each pixel point of the first image block is obtained through the image statistics module. After the statistical value of each pixel point of the first image block is obtained and written into the DDR module, the statistical value of each pixel point of the second image block is obtained through the image statistics module, and the statistical values of the pixel points in the first image block are read in a preset order and rewritten into the DDR module.
[0095] When obtaining the statistical value of each pixel point of the second image block, the following steps are performed for each pixel point of the second image block:
[0096] If the segmented area to which the pixel point belongs is completely divided into blocks, while obtaining the statistical values of the pixel points of the second image block row by row through the image statistics module, the statistical values of the pixel points of the corresponding row in the first image block are read row by row from the DDR module in a preset order and rewritten into the DDR module; wherein the complete block refers to the first image block and the second image block not containing the same segmented area of the image to be processed;
[0097] If the segmented area to which the pixel point belongs is incompletely divided, when processing the pixel points of the second image block row by row, it is necessary to first read the statistical values of the last segmented area of the corresponding row of the first image block from the DDR module according to the preset order, and then obtain the statistical values of the pixel points of the corresponding row of the second image block based on the read statistical values and rewrite the statistical values of the pixel points of the corresponding row of the first image block into the DDR module; wherein, the incomplete blocking situation refers to different parts of the same segmented area of the image to be processed contained in the first image block and the second image block.
[0098] Specifically, the image block to be processed is divided into a first image block and a second image block (whether the sizes of the two image blocks are the same depends on the actual situation of the division of the image block to be processed. If the sizes of the two image blocks divided on average are the same, and the number of segmented areas is the same and one-to-one corresponding, the image to be processed can also be divided into two image blocks of different sizes). If the segmented area to which the pixel point belongs is completely divided into blocks, the statistical values of the pixel points of the corresponding row in the first image block are read row by row from the DDR module in a preset order in order to ensure that the statistical values of the second image block are written in order, that is, when obtaining the statistical values of the pixel points of the second image block, the statistical values of the segmented area corresponding to the segmented area to which the pixel point belongs in the first image block are read out, and the first image block All statistical values of the corresponding row in the first image block (the corresponding row here refers to a row of segmented areas corresponding to the corresponding row) are read out, indicating that all statistical values of the corresponding row in the second image block have been obtained (it is also possible to choose to obtain the statistical values of the second image block after all the statistical values of the corresponding row in the first image block are read out, as long as the statistical values of the corresponding row in the first image block are written into the DDR before the statistical values of the second image block are written), the above-mentioned steps of determining the path of writing the statistical values of the segmented area to which the current pixel belongs into the DDR module according to the back pressure information sent by the DMA module can be performed, and the DDR module is written in sequence; if not all are read out, it means that all statistical values of the corresponding row in the second image block have not been obtained and cannot be written into the DDR module. Before the statistical values of the corresponding row in the second image block are written into the DDR module in sequence, the statistical values of the corresponding row of the first image block read out are rewritten into the DDR module in a preset order.
[0099] If the segmented area to which the pixel point belongs is incompletely divided, when processing the pixel points of the second image block row by row, it is necessary to first read the statistical values of the last segmented area of the corresponding row of the first image block from the DDR module according to the preset order, and then obtain the statistical values of the pixel points of the corresponding row of the second image block based on the read statistical values and rewrite the statistical values of the pixel points of the corresponding row of the first image block into the DDR module; wherein, the incomplete blocking situation refers to different parts of the same segmented area of the image to be processed contained in the first image block and the second image block.
[0100] That is, when a segmented area is divided into two image blocks, when collecting the statistical value of the part of the segmented area in the second image block, if the statistical value is directly obtained, the part of the segmented area in the first image block is not taken into account, which makes the statistical value inaccurate. In order to ensure the accuracy of the statistical value, it is necessary to read the statistical value of the part of the segmented area in the first image block, and then add the statistical value and the statistical value of the pixel point in the corresponding row in the second image block to obtain the final statistical value of the pixel point.
[0101] The above-mentioned preset order is the order in which the statistical values of the segmented areas are written into the DDR module by row (the preset order can be selected as "first in first out", and the preset order is the same as the order in which the image blocks are written into the DDR module). For example, the segmented areas of the first row of the image block to be processed are numbered A, B, and C, respectively, the segmented areas of the second row are numbered D, E, and F, respectively, and the segmented areas of the third row are numbered G, H, and I, respectively. Then the preset order can be ABC, DEF, and GHI. The statistical values of the segmented areas are written into the DDR module in this order, and the statistical values are read out in the same order when reading. If there is an incomplete block situation in the segmented areas B, E, and H, assuming that the incomplete segmented area parts of the segmented areas B, E, and H in the first image block are B1, E1, and H1, and the incomplete segmented area parts of the segmented areas B, E, and H in the second image block are B2, E2, and H2, then the first image block is written into the DDR module in the order of A, B1, D, E1, G, and H1. When the statistical values of the second image block are obtained row by row after the first image block is processed, the statistical values of the pixel points in the corresponding row of the first image block are read from the DDR module row by row in a preset order and written into the DDR module row by row. For example, when obtaining B2, When obtaining the statistical value of C, the statistical values of all the segmented areas in the first row of the first image block are read out. After all the statistical values of the segmented area B2 in the first row of the second image block (the statistical value of B2 is obtained by processing the statistical value of B1 of the first image block) and C are obtained, the statistical values of all the segmented areas in the first row of the first image block are rewritten into the DDR module, and then the statistical values of all the segmented areas in the first row of the second image block are written into the DDR module in the order of acquisition. After the statistical values of the segmented areas in the last row of the second image block are obtained and written into the DDR module, the storage order of the statistical values of each segmented area in the DDR module is still ABC, DEF, GHI.
[0102] It should be noted that, for the statistical values of the pixel points in the corresponding row of the first image block read from the DDR module, the statistical values of the current row of the first image block can be re-written into the DDR module during the process of obtaining the pixel points of the second image block in the current row, or after the pixel points of the second image block in the current row are obtained, the statistical values of the segmented areas in the same row of the first image block and the second image block can be written into the DDR in the order of first writing the statistical values of the first image block and then writing the statistical values of the second image block, thereby controlling the subsequent statistical information to be output in the original order.
[0103] It should be noted that the statistical value of the pixel points of the corresponding row of the second image block can be obtained based on the statistical value read out, which can be obtained by adding or weighted averaging the statistical value of the incompletely segmented area of the second image block obtained by the image statistics module and the statistical value of the corresponding segmented area of the first image block read out from the corresponding row to obtain the complete statistical value. The addition can be carried out by first obtaining the initial statistical value of the first pixel point of the segmented area in the second image block for the pixel points of different parts of the same segmented area contained in the second image block, and then accumulating the initial statistical value and the statistical value of the last pixel point of the corresponding segmented area in the first image block to obtain the statistical value of the first pixel point of the segmented area of the second image block.
[0104] Exemplarily, in some embodiments, information of the first image block is first obtained, and the information includes a valid flag signal of the second image block obtained from the first image block, a column index corresponding to the next column of the last column of the first image block, a block index of the segmented area, and a column index inside the segmented area.
[0105] When inputting the pixel points of the second image block by row, the above information is loaded synchronously. When the valid flag signal of the second image block is 1, it is judged whether it reaches the set statistical area according to the column index. If not, it continues to wait for the next pixel point. If yes, it distinguishes the following two situations according to the internal column index of the block:
[0106] Case 1: When the complete image segmentation region is divided into blocks (i.e., one segmentation region only exists in one image block), while the second image block is being processed, the statistical information (i.e., statistical value) of the segmentation region of the first image block corresponding to the current segmentation region in the DDR module is read back and re-output. After each row of segmentation block data of the second image block is processed, it is determined whether the statistical information of the segmentation region before the segmentation region information of the first image block corresponding to the current segmentation region is fully output. If so, the statistical information of the current segmentation region of the second image block is output to the DDR module to obtain the feature information of the image from the statistical information for improving the quality problems of the image such as overexposure and underexposure. Otherwise, it means that the segmentation region before the current segmentation region of the second image block has not been fully output. In order to output in sequence, the statistical information of the current segmentation region of the second image block is written into the SRAM space.
[0107] Case 2: When the incomplete image segmentation area is divided into blocks (i.e., there is a segmentation area divided into two image blocks), the data of each row of the segmentation block of the second image block will not be processed until the information of the last segmentation area of the current row of the first image block is loaded (to ensure that when the segmentation area divided into two image blocks is statistically operated, the statistical values of all pixels in the segmentation area are used). The amount of statistical information loaded from the DDR module for each row is proportional to the block index of the segmentation area, and the address written into the SRAM space is equal to the block index of the segmentation area block.
[0108] If the valid flag signal of the second image block is 0 (indicating that there is no data to be counted in the second image block), the data of the first image block in the DDR module is directly loaded for subsequent processing and output, such as averaging, weighting or filtering the statistical values of the image.
[0109] By performing the above steps on the first image block and the second image block, the first image block and the second image block have been fully input, and all statistical information has been loaded into the DDR module, and the next frame of the image to be processed can be processed at this time.
[0110] The acquired statistical information can be used for subsequent image analysis, comparison, screening and other operations. For example, by analyzing the statistical characteristics of the image, appropriate enhancement algorithms and parameters can be selected to enhance low-contrast, blurred or noisy images, thereby improving the clarity and quality of the image; by analyzing the statistical characteristics of the image, the shape, texture, color and other features of the object can be extracted to help detect and identify the target object; by analyzing the statistical characteristics of the image such as gray value distribution, texture, and edge, the image can be divided into different areas, thereby facilitating subsequent analysis and processing; by analyzing the statistical characteristics of the image, the compression algorithm and parameters can be optimized to achieve more efficient image compression and reduce the amount of data and transmission time during storage and transmission; by comparing the statistical characteristics of the image, the similarity measurement and matching of the image can be achieved, thereby achieving image retrieval and classification.
[0111] The image processing method of the present application is exemplarily described below with reference to a specific example.
[0112] like Figure 6As shown, the image to be processed is divided into a first image block and a second image block, the first image block includes segmented areas A, B, and C, and the second image block includes segmented areas D, E, and F. In the first image block, when the pixels in segmented areas A and B are taken as current pixel points, if the pixel points meet the preset writing conditions and the DDR module can receive data, the final statistical value of all pixels currently acquired in segmented areas A and segmented areas B is accumulated and directly written into the DDR module. When the pixels in segmented area C are taken as current pixel points, if only the current pixel points meet the preset writing conditions, the DDR module If data cannot be received, the statistical values currently obtained in the segmented area C will be written into the SRAM space; in the second image block, when the pixels in the segmented area D are used as the current pixel points, if the pixel points meet the preset storage conditions, the statistical values currently obtained in the segmented area D will be written into the SRAM space, and when the pixels in the segmented areas E and F are used as the current pixel points, if the pixel points do not meet the preset writing conditions and preset storage conditions, the statistical values currently obtained in the segmented areas E and F will be temporarily stored in the register, and when the DDR module is able to receive data, the statistical values in the SRAM are transmitted to the DDR module.
[0113] It can be seen that the image processing method provided by the present application divides the image to be processed, obtains the statistical value of each pixel point, and opens up an SRAM space in the image statistics module that can store statistical information of a preset amount of data, determines the path for writing to the DDR module according to the statistical value, coordinates the data flow between the modules, avoids the backlog of a large amount of data in the SRAM module, and can effectively control and reduce the occupied area of the SRAM when processing high-resolution images.
[0114] The image processing device provided by the present application is described below.
[0115] like Figure 7 As shown, the image processing device 700 includes:
[0116] An image division module 701 is used to divide the image to be processed into a first image block and a second image block, wherein the first image block and the second image block each include a plurality of rows of pixels, and each row of pixels in the plurality of rows of pixels includes a plurality of pixels;
[0117] The image processing module 702 is used to obtain the statistical value of each pixel point of the first image block and the second image block respectively, and determine the path for writing the statistical value into the DDR module; wherein the path includes writing into the DDR module through the SRAM space, and the SRAM space can store the statistical information of a preset data volume.
[0118] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.
[0119] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0120] The image processing chip provided by this application is described below.
[0121] like Figure 8 As shown, the image processing chip includes the above-mentioned image processing device, DDR module, register module, image statistics module and DMA module.
[0122] The image statistics module in the image processing chip also includes an SRAM space that can store statistical information of a preset data volume, which can effectively control and reduce the occupied area of the SRAM during image statistics.
[0123] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0124] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0125] In the embodiments provided in the present application, it should be understood that the disclosed methods can be implemented in other ways. For example, the embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0126] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0127] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. An image processing method, applied to an image processing system, wherein the image processing system comprises a DDR module, characterized in that: The image processing method comprises: Dividing the image to be processed into a first image block and a second image block; the first image block and the second image block each include a plurality of rows of pixels, and each row of pixels in the plurality of rows of pixels includes a plurality of pixels; The statistical values of each pixel point of the first image block and the second image block are respectively obtained, and a path for writing the statistical values into the DDR module is determined; wherein the path includes writing into the DDR module through an SRAM space, and the SRAM space can store statistical information of a preset data volume.
2. The image processing method according to claim 1, characterized in that: The image processing system further includes a DMA module and an image statistics module provided with the SRAM space; the first image block and the second image block each include a plurality of segmented areas, each of the segmented areas includes a plurality of rows of pixels; the steps of respectively obtaining the statistical value of each pixel of the first image block and the second image block and determining a path for writing the statistical value into the DDR module include: Inputting each pixel point of the first image block and the second image block into the image statistics module in sequence, and obtaining a statistical value of each input pixel point through the image statistics module; According to the arrangement order of the pixels, one pixel is taken as the current pixel in turn; The path for writing the statistical value of the segmented area to which the current pixel belongs into the DDR module is determined according to the back pressure information sent by the DMA module.
3. The image processing method according to claim 2, characterized in that: The image processing system further includes a register module, and the method of determining a path for writing the statistical value of the segmented area to which the current pixel belongs into the DDR module according to the back pressure information sent by the DMA module includes: S1, determining whether the current pixel point meets the preset writing condition, if the current pixel point meets the preset writing condition, executing S2, otherwise, executing S3; S2, judging whether the DDR module can receive data according to the back pressure information sent by the DMA module, if the DDR module can receive data, directly writing the statistical value of the segmented area to which the current pixel belongs to the DDR module; otherwise, when the amount of data in the SRAM space does not reach the preset amount of data, writing the statistical value of the segmented area to which the current pixel belongs to the SRAM space in sequence, until the DDR module can receive data, writing the statistical value to the DDR module through the SRAM space; S3, determine whether the current pixel point meets the preset storage condition. If the current pixel point meets the preset storage condition, write the statistical value into the SRAM space, and use the next pixel point as the current pixel point in S1, and return to execute S1; otherwise, write the statistical value into the register module, and use the next pixel point as the current pixel point to execute S1 until the current pixel point meets the preset storage condition.
4. The image processing method according to claim 3, characterized in that: The determining whether the current pixel point meets a preset writing condition includes: If the current pixel is the last pixel of a segmented area, or is the pixel in the last row of a segmented area and the last column of the current image block, it is determined that the current pixel satisfies the preset writing condition; otherwise, it is determined that the current pixel does not satisfy the preset writing condition.
5. The image processing method according to claim 3, characterized in that: The determining whether the current pixel satisfies a preset storage condition includes: If the pixel point corresponding to the current pixel point is the last column of pixel points in a segmented area, or the last column of pixel points in the current image block, it is determined that the current pixel point meets the preset storage condition; otherwise, it is determined that the current pixel point does not meet the preset storage condition.
6. The image processing method according to claim 1, characterized in that: The obtaining the statistical value of each pixel point of the first image block and the second image block respectively includes: Obtain the statistical value of each pixel point of the first image block. After obtaining the statistical value of each pixel point of the first image block and writing it into the DDR module, obtain the statistical value of each pixel point of the second image block, and read the statistical values of the pixel points in the first image block in a preset order and rewrite them into the DDR module.
7. The image processing method according to claim 6, characterized in that: The acquiring the statistical value of each pixel point of the second image block, and reading the statistical values of the pixel points in the first image block in a preset order and rewriting them into the DDR module, comprises: For each pixel of the second image block, perform the following steps: If the segmented area to which the pixel point belongs is completely divided into blocks, while obtaining the statistical values of the pixel points of the second image block row by row, the statistical values of the pixel points of the corresponding row in the first image block are read row by row from the DDR module in a preset order and rewritten into the DDR module; wherein the image to be processed includes a plurality of segmented areas; the complete block refers to the same segmented area in the first image block and the second image block that does not contain the image to be processed; If the segmented area to which the pixel point belongs is incompletely divided, when processing the pixel points of the second image block row by row, it is necessary to first read the statistical values of the last segmented area of the corresponding row of the first image block from the DDR module in a preset order, and then obtain the statistical values of the pixel points of the corresponding row of the second image block based on the read statistical values and rewrite the statistical values of the pixel points of the corresponding row of the first image block into the DDR module; wherein the incomplete block situation refers to different parts of the same segmented area of the image to be processed contained in the first image block and the second image block.
8. The image processing method according to claim 1, characterized in that: The preset data amount is smaller than the data amount corresponding to the pixels in the first image block.
9. An image processing device, characterized in that: include: An image division module, used for dividing the image to be processed into a first image block and a second image block; the first image block and the second image block each include a plurality of rows of pixel points, and each row of pixel points in the plurality of rows of pixel points includes a plurality of pixel points; An image processing module is used to obtain the statistical value of each pixel point of the first image block and the second image block respectively, and determine the path for writing the statistical value into the DDR module; wherein the path includes writing into the DDR module through the SRAM space, and the SRAM space can store the statistical information of a preset data volume.
10. An image processing chip, characterized in that: The invention comprises the image processing device as claimed in claim 9.