Image processing apparatus and control method of image processing apparatus
By using an image processing device in the image processing device, including a first buffer circuit and a second buffer circuit, and operating in two modes through a direct memory access circuit, the ping-pong buffering in the prior art is solved. The cost increase caused by the buffer circuit enables more economical image processing.
Patent Information
- Application Number
- CN202411034934.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2024-12-03
AI Technical Summary
In the existing image processing technology, ping-pong buffering requires two buffering circuits, resulting in an increase in image processing cost.
An image processing device is adopted, including a first buffer circuit and a second buffer circuit. The direct memory access circuit operates in two modes, respectively, and writes and reads the image data, reducing the dependence on the buffer circuit. , reduces costs.
It effectively reduces the cost of image processing, realizes image processing with the same effect as existing ping-pong buffers, and reduces hardware requirements.
Smart Images

Figure CN119067834A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to an image processing device and a control method for the image processing device. Background Art
[0002] See also Figure 1 , Figure 1 Schematic diagram of an existing ping-pong buffer for image processing. The height and width of the image frame 100 are Ht and Wd respectively (Ht and Wd are integers greater than 1). The image frame 100 includes Ht lines of pixels (LN_1, LN_2, ..., LN_i, ..., 1≤i≤Ht), and a line of pixels LN_i includes Wd pixels. The ping-pong buffer includes a buffer circuit 110 and a buffer circuit 120. The buffer circuit 110 and the buffer circuit 120 can each store Wd×M pixels. In other words, the buffer circuit 110 and the buffer circuit 120 can each store up to M lines of pixels (M <Ht)。
[0003] When the image processing circuit of the previous stage writes the image data to the buffer circuit 110 on a line-based basis, the image processing circuit of the subsequent stage reads the image data from the buffer circuit 120 on a tile-based basis. When the image data in the buffer circuit 110 and the image data in the buffer circuit 120 are both read, the roles of the buffer circuit 110 and the buffer circuit 120 are exchanged to form a ping-pong buffer.
[0004] However, since the ping-pong buffer requires two buffer circuits, the cost of image processing increases. Summary of the invention
[0005] One of the purposes of the present application is to provide an image processing device and a control method of the image processing device to improve the deficiencies of the prior art.
[0006] An embodiment of the present application provides an image processing device for processing a picture frame, comprising: a first buffer circuit, a second buffer circuit, a first direct memory access circuit, and a second direct memory access circuit. The first buffer circuit comprises M buffer blocks, where M is an integer greater than 1. The second buffer circuit comprises W coding blocks, where W is an integer greater than 1. The first direct memory access circuit is coupled to the first buffer circuit and operates in a first mode or a second mode. The second direct memory access circuit is coupled to the first buffer circuit and the second buffer circuit and operates in the first mode or the second mode. In the first mode, the first direct memory access circuit writes a first pixel of the picture frame into one of the M buffer blocks, and the second direct memory access circuit reads N pixels from each of the M buffer blocks, and writes the M×N pixels into one of the W coding blocks, where N is an integer greater than 1. In the second mode, the first direct memory access circuit writes a second strip of pixels of the image frame into the M buffer blocks, each buffer block including a portion of the second strip of pixels, and the second direct memory access circuit reads M×N pixels from one of the M buffer blocks and writes the M×N pixels into one of the W coding blocks. After the first direct memory access circuit writes the M strips of pixels into the M buffer blocks, the first direct memory access circuit switches modes.
[0007] Another embodiment of the present application provides a control method for an image processing device. The image processing device is used to process a frame and includes a first buffer circuit, a second buffer circuit, a first direct memory access circuit and a second direct memory access circuit. The first buffer circuit includes M buffer blocks, M is an integer greater than 1. The second buffer circuit includes W encoding blocks, W is an integer greater than 1. The first direct memory access circuit operates in a first mode or a second mode and generates a first current access progress. The second direct memory access circuit operates in the first mode or the second mode and generates a second current access progress. The control method includes: when the first current access progress meets a first condition, controlling the second direct memory access circuit to switch mode; or when the second current access progress meets a second condition, controlling the first direct memory access circuit to switch mode. In the first mode, the first direct memory access circuit writes a first strip of pixels of the image frame into one of the M buffer blocks, and the second direct memory access circuit reads N pixels from each of the M buffer blocks and writes the M×N pixels into one of the W coding blocks, where N is an integer greater than 1. In the second mode, the first direct memory access circuit writes a second strip of pixels of the image frame into the M buffer blocks, each buffer block including a portion of the second strip of pixels, and the second direct memory access circuit reads M×N pixels from one of the M buffer blocks and writes the M×N pixels into one of the W coding blocks.
[0008] The technical means embodied in the embodiments of the present application can improve at least one of the shortcomings of the prior art, so the present application can reduce costs compared to the prior art.
[0009] The features, implementation and effects of the present application are described in detail below with reference to the accompanying drawings for a preferred embodiment. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] Figure 1 is a schematic diagram of an existing ping-pong buffer for image processing;
[0011] Figure 2 is a functional block diagram of an embodiment of the image processing device of the present application;
[0012] Figure 3 is a schematic diagram of an embodiment of the buffer circuit 240 and the buffer circuit 250 of the present application;
[0013] Figure 4 is a flow chart of an embodiment of a control method of an image processing device of the present application;
[0014] Figure 5is a flow chart of DMA 222 operating in a first mode;
[0015] Figure 6 is a schematic diagram of the buffer circuit 240 after the DMA 222 writes M pixels into the buffer circuit 240;
[0016] Figure 7 is a flow chart of DMA 260 operating in a first mode;
[0017] Figure 8 is a schematic diagram of the DMA 260 filling up the buffer circuit 250 for the first time in the first mode;
[0018] Fig. 9 is a flow chart of DMA 222 operating in the second mode;
[0019] Fig.10 This application is completed by DMA 222 Fig. 9 A schematic diagram of the buffer circuit 240 after one round of the process;
[0020] Fig.11 This application is completed by DMA 222 Fig. 9 A schematic diagram of a buffer circuit 240 after the process;
[0021] Fig.12 is a flow chart of DMA 260 operating in the second mode; and
[0022] Fig.13 FIG. 2 is a schematic diagram of the contents of the buffer circuit 240 and the buffer circuit 250 of the DMA 260 of the present application in the second mode.
[0023] Reference numerals:
[0024] 100: picture frame;
[0025] 110, 120, 240, 250: buffer circuit;
[0026] Ht: height;
[0027] LN, LN_1, LN_2, LN_i, LN_q, LN_q+1, LN_q+j-1, LN_q+M-1, LN_q+M, LN_q+2M-1, LN_q+M+1: one pixel;
[0028] TL, TL_1, TL_2, TL_k, TL_W: coding block;
[0029] Wd: width;
[0030] 200: image processing device;
[0031] 210: processor;
[0032] 220, 230: image processing circuit;
[0033] 222, 232, 260: DMA (direct memory access circuit);
[0034] CP1, CP2: current access progress;
[0035] Ctrl1, Ctrl2: control signal;
[0036] LS_1, LS_2, LS_j, LS_M: buffer blocks;
[0037] R: number of columns;
[0038] : steps. S405, S410, S420, S430, S440, S450, S505, S510, S520, S530, S540, S705, S710, S720, S730, S740, S750, S760, S770, S780, S790, S795, S905, S910, S920, S930, S940, S950, S960, S1205, S1210, S1220, S1230, S1240, S1250, S1260, S1270, S1280, S1290: steps. DETAILED DESCRIPTION
[0039] The technical terms used in the following descriptions are based on customary terms in the technical field. If some of the terms are explained or defined in this specification, the interpretation of the terms shall be based on the explanation or definition in this specification.
[0040] The present application includes an image processing device and a control method for the image processing device. Since some of the components included in the image processing device of the present application may be known components individually, the details of the known components will be omitted in the following description without affecting the full disclosure and feasibility of the invention of the device. In addition, part or all of the process of the control method of the image processing device of the present application may be in the form of software and / or firmware, and can be executed by the image processing device of the present application or its equivalent device. Without affecting the full disclosure and feasibility of the invention of the method, the following description of the method invention will focus on the step content rather than the hardware.
[0041] See also Figure 2 , Figure 21 is a functional block diagram of an embodiment of an image processing device of the present application. The image processing device 200 includes a processor 210, an image processing circuit 220, an image processing circuit 230, a buffer circuit 240, a buffer circuit 250, and a direct memory access (DMA) 260. The image processing circuit 220 includes a DMA 222. The image processing circuit 230 includes a DMA 232. The buffer circuit 240 may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), and the buffer circuit 250 may be a static random access memory.
[0042] The processor 210 may be a circuit or electronic component with program execution capability, such as a central processing unit, a microprocessor, a microprocessing unit, a digital signal processor, an application specific integrated circuit (ASIC), or an equivalent circuit thereof.
[0043] The image processing circuit 220 writes the image data into the buffer circuit 240 through the DMA 222. The DMA 260 is used to read the image data from the buffer circuit 240 and write the image data into the buffer circuit 250. The image processing circuit 230 reads the image data from the buffer circuit 250 through the DMA 232.
[0044] DMA 222 and DMA 260 operate in the first mode or the second mode. Processor 210 controls DMA 222 with a control signal Ctrl1 and controls DMA 260 with a control signal Ctrl2. DMA 222 and DMA 260 transmit current access progress CP1 and current access progress CP2 to processor 210 respectively.
[0045] In some embodiments (for example only, not to limit the present application), the image processing circuit 220 may be an image scaler, and the image processing circuit 230 may be a Joint Photographic Experts Group (JPEG) encoder.
[0046] In the following description, it is assumed that the minimum coding unit of the image processing circuit 230 is M (columns)×N (pixels). That is, the image processing circuit 230 performs image coding in units of coding blocks TL, and the size of a coding block TL is M×N pixels. M and N are both integers greater than 1. In some embodiments, M is equal to N.
[0047] See also Figure 3 , Figure 3 : is a schematic diagram of an embodiment of the buffer circuit 240 and the buffer circuit 250 of the present application. The buffer circuit 240 includes M buffer blocks (LS_1, LS_2, ..., LS_j, ... and LS_M, 1≤j≤M). The size of each buffer block is R (column) × N (pixel). The buffer circuit 250 includes W coding blocks TL (TL_1, TL_2, ..., TL_k, ... and TL_W, W is an integer greater than 1, 1≤k≤W). The size of each coding block TL is M (column) × N (pixel). Therefore, the width of the buffer circuit 250 is W×N, that is, one column includes W×N pixels.
[0048] In the first mode, a buffer block stores a whole strip of pixels of the image frame 100. In the second mode, a buffer block stores W×N pixels in a strip of pixels, but stores M consecutive strips of the image frame 100 (i.e., a total of M×W×N pixels). The number of columns R of a buffer block is as shown in formula (1) (where Wd is the width of the image frame 100). For example, when Wd=1920, N=M=16, then R=128. For another example, when Wd=3840, N=M=16, then R=256.
[0049] R=(Wd+N×M-1) / (N×M) (1),
[0050] See also Figure 4 , Figure 4 It is a flow chart of one embodiment of a control method of an image processing device of the present application, comprising the following steps.
[0051] Step S405: The processor 210 initializes the variable i to 0. The variable i will be used in the operation flow of the DMA 222, which will be described below. Figure 5 and Fig. 9 Describe in detail.
[0052] Step S410: The processor 210 controls the DMA 222 to start operating in another mode (ie, switching mode) via the control signal Ctrl1. More specifically, if the DMA 222 previously operates in the first mode ( Figure 5 )(or the second mode( Fig. 9 )), then after step S410, DMA 222 changes to operate in the second mode (or the first mode). That is, DMA 222 operates alternately in the first mode and the second mode. In some embodiments, the first operating mode of DMA 222 after the image processing apparatus 200 is activated is the first mode.
[0053] Step S420: The processor 210 determines whether the progress of the DMA 222 satisfies the first condition. If yes, step S430 is executed; otherwise, the processor 210 continuously monitors the progress of the DMA 222 (step S420). The DMA 222 continuously notifies the processor 210 of the current access progress CP1 during the process of accessing the buffer circuit. In some embodiments, the current access progress CP1 may be a write index of the DMA 222 to the buffer circuit 240.
[0054] Step S430: The processor 210 controls the DMA 260 to start operating in another mode (ie, switching mode) via the control signal Ctrl2. More specifically, if the DMA 260 previously operates in the first mode ( Figure 7 )(or the second mode( Fig.12 )), then after step S430, DMA 260 changes to operate in the second mode (or the first mode). That is, DMA 260 operates alternately in the first mode and the second mode. In some embodiments, the first operating mode of DMA 260 after the image processing apparatus 200 is activated is the first mode.
[0055] Step S440: The processor 210 determines whether the progress of the DMA 260 satisfies the second condition. If yes, step S450 is executed; otherwise, the processor 210 continuously monitors the progress of the DMA 260 (step S440). The DMA 260 continuously notifies the processor 210 of the current access progress CP2 during the process of accessing the buffer circuit. In some embodiments, the current access progress CP2 may be a read index (read index) of the DMA 260 to the buffer circuit 240 and / or a write index to the buffer circuit 250.
[0056] Step S450: The processor 210 determines whether the writing and reading of a frame has been completed. If yes, the processor 210 executes step S405 (resetting the variable i to 0); otherwise, the processor 210 executes step S410.
[0057] The following are the first mode ( Figures 5 to 8 ) and the second mode ( Figures 9 to 13 ) illustrate the operation details of DMA 222 and DMA 260. The current access progress CP1, the current access progress CP2, the first condition and the second condition are illustrated below with examples.
[0058] First mode.
[0059] See also Figure 5 , Figure 5 is a flow chart of the DMA 222 operating in the first mode, including the following steps.
[0060] Step S505 : DMA 222 updates variable i to i+1, and sets variable j to 1.
[0061] Step S510: DMA 222 writes a strip of pixels LN_i into a buffer block LS_j of the buffer circuit 240. In this step, DMA 222 continuously writes the strip of pixels into the same buffer block.
[0062] Step S520: DMA 222 determines whether j is equal to M. If yes, execute step S540; otherwise, execute step S530.
[0063] Step S530 : DMA 222 updates i to i+1, and updates j to j+1.
[0064] Step S540: DMA 222 waits for the instruction of processor 210. More specifically, DMA 222 operates in the waiting state (suspended operation) until processor 210 controls it to leave the waiting state and operate in the second mode ( Fig. 9 ). That is, when the DMA 222 operates in the waiting state, it means that the DMA 222 is about to switch modes.
[0065] See also Figure 6 , Figure 6 FIG. 2 is a schematic diagram of the buffer circuit 240 after the DMA 222 writes M pixels into the buffer circuit 240. Figure 5 In the process of writing the qth pixel LN_q, the q+1th pixel LN_q+1, the q+j-1th pixel LN_q+j-1 and the q+M-1th pixel LN_q+M-1 of the image frame 100 into the buffer block LS_1, the buffer block LS_2, the buffer block LS_j and the buffer block LS_M in sequence. That is, the write stride of the DMA 222 is the size of a buffer block (i.e., W×M×N pixels). When j is equal to M (i.e., when the image processing circuit 220 completes the first mode) (the result of step S520 is yes), the buffer blocks LS_1 to LS_M store the qth pixel to the q+M-1th pixel of the image frame 100 respectively (stored in the 1st column to the Wd / Nth column of each buffer block, Wd / N≤R) (q=r×M+1, r=0, 1, 2, . . . , and q≤Ht).
[0066] See also Figure 7 , Figure 7 is a flow chart of the DMA 260 operating in the first mode, including the following steps.
[0067] Step S705 : DMA 260 sets variables j and k to 1, and sets variable p to 0.
[0068] Step S710 : DMA 260 reads N pixels from the p×W+k-th column of buffer block LS_j.
[0069] Step S720: DMA 260 writes the N pixels into the j-th column of the coding block TL_k.
[0070] Step S730: DMA 260 determines whether j is equal to M. If yes (which means DMA 260 has written the p×W+kth column of M buffer blocks into coding block TL_k, and also means coding block TL_k is full), execute step S740; otherwise, execute step S750.
[0071] Step S740 : DMA 260 resets the variable j to 1.
[0072] Step S750: DMA 260 updates the variable j to j+1.
[0073] Step S760: DMA 260 determines whether k is equal to W. If yes (indicating that the Wth coding block TL_W of the buffer circuit 250 has been fully written, that is, DMA 260 has fully written the entire buffer circuit 250), step S780 is executed; otherwise, step S770 is executed.
[0074] Step S770 : DMA 260 updates the variable k to k+1, so that DMA 260 reads the next column of buffer block LS_j in step S710 and writes the next encoding block in step S720 .
[0075] Step S780: DMA 260 determines whether p is equal to M-1. If so (meaning DMA 260 has read all buffer blocks (LS_1-LS_M) of buffer circuit 240), then execute step S795; otherwise, execute step S790.
[0076] Step S790 : DMA 260 resets the variable k to 1, and updates the variable p to p+1.
[0077] Step S795: DMA 260 waits for the instruction of processor 210. More specifically, DMA 260 operates in the waiting state (suspended operation) until processor 210 controls it to leave the waiting state and operate in the second mode ( Fig.12 ). That is, when the DMA 260 operates in the waiting state, it means that the DMA 260 is about to switch modes.
[0078] Depend on Figure 7It can be seen that since the variable j increases by 1 each time in step S750, the read offset of DMA 260 is the size of a buffer block (W×M×N pixels) (step S710), and the write step size of DMA 260 is a row of buffer circuit 250 (W×N pixels) (step S720).
[0079] See also Figure 8 , Figure 8 is a schematic diagram of the DMA 260 filling the buffer circuit 250 for the first time in the first mode (the result of step S760 is yes). For the buffer circuit 240, W columns of pixels (a total of W×N pixels) in each buffer block have been read, that is, there is a temporary space of W×N pixels in each buffer block for writing image data. For the buffer circuit 250, each encoding block stores M columns of pixels, and the M columns of pixels come from the M buffer blocks (LS_1~LS_M) of the buffer circuit 240 respectively. In other words, the multiple W×N pixels of the buffer block LS_j have been written to the jth column of the buffer circuit 250 (a total of W×N pixels, spanning W encoding blocks). It should be noted that the multiple temporary spaces of the M buffer blocks (a total of W×M×N pixels) are exactly equal to the storage space of one buffer block.
[0080] In some embodiments, when the coding block TL_1 is full, the DMA 232 can start reading the coding block TL_1 without waiting for all coding blocks in the buffer circuit 250 to be full. That is, the buffer circuit 240 together with the buffer circuit 250 can achieve the same effect as the existing ping-pong buffer.
[0081] Second mode.
[0082] See also Fig. 9 , Fig. 9 is a flow chart of the DMA 222 operating in the second mode, including the following steps.
[0083] Step S905: DMA 222 updates variable i to i+1, and sets variable j to 1. Figure 8 For example, since the variable i is equal to q+M-1 when the first mode ends, the DMA 222 updates the variable i to q+M in this step. That is, when step S905 ends, i=1 (for example, the image processing device 200 has just started to process a frame), or the value of the variable i in the first mode is continued. The same is true for step S505.
[0084] Step S910: DMA 222 determines whether j is equal to M. If yes (ie, DMA 222 is processing the Mth buffer block LS_M), step S940 is executed; otherwise, step S920 is executed.
[0085] Step S920 : DMA 222 writes the (W×N)×(j−1)+1th to (W×N)×jth (W×N in total) pixels in a line of pixels LN_i into the buffer block LS_j of the buffer circuit 240 , more specifically, into the temporary space of the buffer block LS_j.
[0086] Step S930: DMA 222 updates the variable j to j+1.
[0087] Step S940: DMA 222 writes the remaining pixels in a row of pixels LN_i into the buffer block LS_M of the buffer circuit 240. In some embodiments, the remaining pixels are exactly W×N pixels. The end of step S940 indicates that DMA 222 has written a row of pixels LN_i into the buffer circuit 240. Since a row of pixels LN_i is written into M buffer blocks, DMA 222 performs a scatter write operation in the second mode. The write step size of the scatter write is one buffer block (i.e., W×M×N pixels).
[0088] Step S950: DMA 222 determines whether i is an integer multiple of M (i.e., determines whether "i mod M" is equal to 0, where "mod" represents modulo operation). If yes (it means that the image processing circuit 220 has written the other M pixels LN_q+M to LN_q+2M-1 into the temporary space of the buffer circuit 240, i.e., the buffer blocks LS_1 to LS_M each include a part of the M pixels), execute step S960 (end the second mode); otherwise, execute step S905 (execute the next round).
[0089] Step S960: DMA 222 waits for the instruction of processor 210. More specifically, DMA 222 operates in the waiting state (suspended operation) until processor 210 controls it to leave the waiting state and operate in the first mode ( Figure 5 ).
[0090] It should be noted that in some cases, when the DMA 222 operates in the second mode, the DMA 260 is operating in the first mode. In other words, while the DMA 260 reads the image data from the buffer circuit 240 to generate more temporary spaces (step S710), the DMA 222 writes the image data into the plurality of temporary spaces of the buffer circuit 240 (step S920 or step S940).
[0091] See also Fig.10 , Fig.10 This application is completed by DMA 222 Fig. 9 Schematic diagram of the buffer circuit 240 after one round of the process. Fig.10 It is a continuation Figure 8 As shown in the figure, DMA 222 writes the q+Mth pixel LN_q+M to buffer blocks LS_1 to LS_M. In the second mode, the writing step size of DMA 222 is the size of a buffer block (ie, W×M×N pixels).
[0092] See also Fig.11 , Fig.11 This application is completed by DMA 222 Fig. 9 Schematic diagram of the buffer circuit 240 after the process (the result of step S950 is yes). As shown in the figure, each of the buffer blocks LS_1 to LS_M stores a part of M pixels (LN_q+M to LN_q+2M-1).
[0093] See also Fig.12 , Fig.12 is a flow chart of the DMA 260 operating in the second mode, including the following steps.
[0094] Step S1205: DMA 260 sets variables j, k, and p to 1.
[0095] Step S1210 : DMA 260 reads N pixels in the p-th column of buffer block LS_j.
[0096] Step S1220: DMA 260 writes the N pixels into the “p / W”th column of the coding block TL_k.
[0097] Step S1230: DMA 260 determines whether "p / W" is equal to M. If so (indicating that DMA 260 has fully written the coding block TL_k), step S1250 is executed; otherwise, step S1240 is executed.
[0098] Step S1240: DMA 260 updates the variable p to p+W. That is, the read offset of DMA 260 is W×N pixels.
[0099] Step S1250: DMA 260 determines whether k is equal to W. If yes (meaning DMA 260 has read an entire buffer block LS_j and has filled the buffer circuit 250), step S1270 is executed; otherwise, step S1260 is executed.
[0100] Step S1260: DMA 260 updates the variable k to k+1 and the variable p to k, and then executes step S1210.
[0101] Step S1270: DMA 260 determines whether j is equal to M. If so (it means that DMA 260 has read all buffer blocks (LS_1-LS_M) of buffer circuit 240), step S1290 is executed; otherwise, step S1280 is executed.
[0102] Step S1280 : DMA 260 updates variable j to j+1, and resets variable k and variable p to 1, and then executes step S1210 .
[0103] Step S1290: DMA 260 waits for the instruction of processor 210. More specifically, DMA 260 operates in a waiting state (suspended operation) until processor 210 controls it to leave the waiting state and operate in the first mode ( Figure 7 ).
[0104] See also Fig.13 , Fig.13 FIG. 2 is a schematic diagram of the contents of the buffer circuit 240 and the buffer circuit 250 of the DMA 260 of the present application in the second mode. Fig.13 It is shown that DMA 260 has written M×W×N image data of buffer block LS_1 into buffer circuit 250. After DMA 260 has written a full coding block (i.e., the result of step S1230 is yes), it starts to write the next coding block (k=k+1 or k=1). Therefore, the writing step size of DMA 260 is one column (W×N pixels) of buffer circuit 250. It should be noted that when DMA 260 has written the full coding block TL_1, DMA 232 can start to read buffer circuit 250.
[0105] See also Fig.13 When the image data in the buffer block LS_1 is completely written into the buffer circuit 250, the processor 210 controls the DMA 222 to switch to the first mode to re-execute Figure 5 process (first mode).
[0106] Note that in some cases, when DMA 222 operates in the first mode, DMA 260 is operating in the second mode. In other words, DMA 260 reads image data from buffer circuit 240 to release more buffer blocks (step S1210), while DMA 222 writes image data to the released buffer blocks (step S510).
[0107] The current access progress CP1, the current access progress CP2, the first condition and the second condition are respectively described below with reference to the first mode and the second mode.
[0108] For the first mode, the current access progress CP1 may be the variable j of the DMA 222, and the first condition may be j=M. For example, see Figure 6 When the DMA 222 has sequentially written the first to M-1th pixels of the continuous M pixels of the image frame 100 into the M-1 buffer blocks (e.g., buffer blocks LS_1 to LS_M-1 (not shown)) of the buffer circuit 240, and has written the Mth pixel into the remaining one of the M buffer blocks (e.g., buffer block LS_M), the processor 210 controls the DMA 260 to leave the waiting state and enter the first mode (i.e., continue to operate to perform the operation). Figure 7 The current access progress CP2 may be a variable k of the DMA 260, and the second condition may be k=W. For example, see Figure 8 When DMA 260 reads W×N pixels from M buffer blocks (LS_1 to LS_M) respectively (equivalent to when DMA 260 fills up buffer circuit 250, i.e., when k=W), processor 210 controls DMA 222 to leave the waiting state and enter the second mode (i.e., continue to operate to perform Fig. 9 process).
[0109] For the second mode, the current access progress CP1 may be the variable i and the variable j of the DMA 222, and the first condition may be i=c*M and j=1 (c=2, 4, 6, ...). For example, see Fig.11 When DMA 222 writes part of the data (W*N pixels) of the q+2M-1th pixel LN_q+2M-1 (q=r*M+1, r=0,2,4,6,...) into buffer block LS_1, processor 210 controls DMA 260 to leave the waiting state and enter the second mode (i.e., continue to operate to perform Fig.12 The current access progress CP2 may be the variable j and the variable p of the DMA 260, and the second condition may be j=1 and p=R. For example, see Fig.13 When DMA 260 has read all M*W*N pixels from buffer block LS_1, processor 210 controls DMA 222 to leave the waiting state and enter the first mode (ie, continue to operate to perform Figure 5 process).
[0110] See also Figure 1 , the total size of the buffer circuit 110 and the buffer circuit 120 is Wd×M×2 pixels. Figure 3, the sum of the sizes of the buffer circuit 240 and the buffer circuit 250 of the present application is (W×M×N)×M+W×M×N=(W×M×N)×(M+1) pixels. When W×M×N=Wd (i.e., the size of a buffer block LS_j is just equal to a pixel Wd), the sum of the sizes of the buffer circuit 240 and the buffer circuit 250 is (M+1)×Wd. In other words, the size of the buffer circuit of the present application is about half of the existing buffer circuit ((M+1) / 2M), so the cost can be greatly reduced.
[0111] In some embodiments, whether in the first mode or the second mode, the operating speed of DMA 260 is greater than or equal to the operating speed of DMA 222, and the operating speed of DMA 232 is greater than the operating speed of DMA 260. This can be achieved by designing the frequency of image processing circuit 230 to be greater than the frequency of image processing circuit 220.
[0112] Although the above embodiment takes the operation of converting pixel-by-pixel writing into coding block-by-coding reading as an example, this is not a limitation of the present application. Technicians in this technical field can appropriately apply the present application to the operation of converting coding block-by-coding writing into pixel-by-pixel reading based on the disclosure of the present application.
[0113] Although the embodiments of the present application are described above, these embodiments are not intended to limit the present application. Technicians in this technical field may change the technical features of the present application based on the explicit or implicit contents of the present application. All these changes may fall within the scope of patent protection sought by the present application. The scope of patent protection of the present application shall be based on the scope of the patent application of this specification.
Claims
1. An image processing device, characterized in that: Used to process a frame, including: a first buffer circuit, comprising M buffer blocks, where M is an integer greater than 1; a second buffer circuit, comprising W coding blocks, where W is an integer greater than 1; a first direct memory access circuit, coupled to the first buffer circuit, and operating in a first mode or a second mode; and a second direct memory access circuit coupled to the first buffer circuit and the second buffer circuit and operating in the first mode or the second mode; Wherein, in the first mode, the first direct memory access circuit writes a first pixel of the image frame into one of the M buffer blocks, and the second direct memory access circuit reads N pixels from each of the M buffer blocks and writes the M×N pixels into one of the W coding blocks, where N is an integer greater than 1; Wherein, in the second mode, the first direct memory access circuit writes a second strip of pixels of the image frame into the M buffer blocks, each buffer block including a portion of the second strip of pixels, and the second direct memory access circuit reads M×N pixels from one of the M buffer blocks and writes the M×N pixels into one of the W coding blocks; After the first direct memory access circuit writes the M pixels into the M buffer blocks, the first direct memory access circuit switches the mode.
2. The image processing device according to claim 1, wherein: The size of any one of the M buffer blocks is W×M×N pixels, and the size of each encoding block is N×M pixels.
3. The image processing device according to claim 2, wherein: M equals N.
4. The image processing device according to claim 1, wherein: After the second direct memory access circuit finishes reading the M buffer blocks, the second direct memory access circuit switches modes.
5. The image processing device according to claim 1, wherein: The portion of the second strip of pixels is W×N pixels.
6. The image processing device according to claim 1, wherein: In the first mode, a write step size of the first direct memory access circuit is a size of a buffer block.
7. The image processing device according to claim 1, wherein: In the second mode, a writing step size of the first direct memory access circuit is a size of a buffer block.
8. The image processing device according to claim 1, wherein: In the first mode, a read offset and a write step size of the second direct memory access circuit are respectively a size of a buffer block and a column of the second buffer circuit.
9. The image processing device according to claim 1, wherein: In the second mode, a read offset and a write step size of the second direct memory access circuit are W×N pixels and one column of the second buffer circuit, respectively.
10. The image processing device according to claim 1, wherein: The first direct memory access circuit operates in a waiting state before switching modes. The image processing device further includes: A processor is coupled to the first direct memory access circuit and the second direct memory access circuit, and is used to receive a current access progress from the second direct memory access circuit, and perform the following steps: When the current access progress satisfies a condition, controlling the first direct memory access circuit to leave the waiting state and operate in the second mode; The condition is that the second direct memory access circuit has read W×N pixels from the M buffer blocks respectively.
11. The image processing device according to claim 1, wherein: The first direct memory access circuit operates in a waiting state before switching modes. The image processing device further includes: A processor is coupled to the first direct memory access circuit and the second direct memory access circuit, and is used to receive a current access progress from the second direct memory access circuit, and perform the following steps: When the current access progress satisfies a condition, controlling the first direct memory access circuit to leave the waiting state and operate in the first mode; The condition is that the second direct memory access circuit has read M×W×N pixels from one of the M buffer blocks.
12. The image processing device according to claim 1, wherein: The second direct memory access circuit operates in a waiting state before switching modes. The image processing device further includes: A processor is coupled to the first direct memory access circuit and the second direct memory access circuit, and is used to receive a current access progress from the first direct memory access circuit, and perform the following steps: When the current access progress satisfies a condition, controlling the second direct memory access circuit to leave the waiting state and operate in the first mode; The condition is that the first direct memory access circuit has written the first to M-1th pixels of the M pixels of the image frame into M-1 of the M buffer blocks, and has written the portion of the Mth pixel of the M pixels into the remaining one of the M buffer blocks.
13. The image processing device according to claim 1, wherein: The second direct memory access circuit operates in a waiting state before switching modes. The image processing device further includes: A processor is coupled to the first direct memory access circuit and the second direct memory access circuit, and is used to receive a current access progress from the first direct memory access circuit, and perform the following steps: When the current access progress satisfies a condition, controlling the second direct memory access circuit to leave the waiting state and operate in the second mode; The condition is that the first direct memory access circuit has written the portion of the c×Mth pixel of the frame into one of the M buffer blocks, and c is an even number.
14. The image processing device according to claim 1, wherein: An operation speed of the second direct memory access circuit is greater than an operation speed of the first direct memory access circuit.
15. The image processing device according to claim 1, wherein: When the first direct memory access circuit is operating in the first mode, the second direct memory access circuit is operating in the second mode, and when the first direct memory access circuit is operating in the second mode, the second direct memory access circuit is operating in the first mode.
16. A method for controlling an image processing device, characterized in that: The image processing device is used to process a picture frame and includes a first buffer circuit, a second buffer circuit, a first direct memory access circuit and a second direct memory access circuit, the first buffer circuit includes M buffer blocks, M is an integer greater than 1, the second buffer circuit includes W encoding blocks, W is an integer greater than 1, the first direct memory access circuit operates in a first mode or a second mode and generates a first current access progress, the second direct memory access circuit operates in the first mode or the second mode and generates a second current access progress, and the control method includes: When the first current access progress satisfies a first condition, controlling the second direct memory access circuit to switch mode; or When the second current access progress satisfies a second condition, controlling the first direct memory access circuit to switch mode; Wherein, in the first mode, the first direct memory access circuit writes a first pixel of the image frame into one of the M buffer blocks, and the second direct memory access circuit reads N pixels from each of the M buffer blocks and writes the M×N pixels into one of the W coding blocks, where N is an integer greater than 1; In which, in the second mode, the first direct memory access circuit writes a second strip of pixels of the image frame into the M buffer blocks, each buffer block includes a portion of the second strip of pixels, and the second direct memory access circuit reads M×N pixels from one of the M buffer blocks and writes the M×N pixels into one of the W coding blocks.
17. The control method according to claim 16, characterized in that: The first condition is that the first direct memory access circuit has written the first to M-1th pixels of the M pixels of the image frame into M-1 of the M buffer blocks, and has written the portion of the Mth pixel of the M pixels into the remaining one of the M buffer blocks, and when the first current access progress satisfies the first condition, the second direct memory access circuit starts operating in the first mode.
18. The control method according to claim 16, characterized in that: The first condition is that the first direct memory access circuit has written the portion of the c×Mth pixel of the frame into one of the M buffer blocks, c is an even number, and when the first current access progress satisfies the first condition, the second direct memory access circuit starts operating in the second mode.
19. The control method according to claim 16, characterized in that: The second condition is that the second direct memory access circuit has read W×N pixels from the M buffer blocks respectively, and when the second current access progress satisfies the second condition, the first direct memory access circuit starts to operate in the second mode.
20. The control method according to claim 16, characterized in that: The second condition is that the second direct memory access circuit has read M×W×N pixels from one of the M buffer blocks, and when the second current access progress satisfies the second condition, the first direct memory access circuit starts to operate in the first mode.
