RGBG-to-RGB conversion system and method and storage medium
By using the first memory and the second memory to alternately perform read and write operations in the RGBG-to-RGB image processing system, the problems of large storage resources and long delay time in the prior art are solved, and more efficient image processing is achieved.
Patent Information
- Application Number
- CN202411228671.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-03
- Publication Date
- 2024-12-27
AI Technical Summary
In the image processing process of RGBG to RGB, existing hardware circuits require three SRAM for ping-pong storage, resulting in large consumption of storage resources and long delay time. How to reduce SRAM consumption and reduce latency has become an urgent problem.
Using an RGBG to RGB conversion system, the first memory and the second memory are alternately read and write operations, and the first two rows of pixel data are read and written separately in every two clock cycles, and the data flow is managed by a logic controller. Reduce memory storage resource consumption and latency.
It realizes the use of memory resources, reduces the output timing delay time, and improves the efficiency of image processing.
Smart Images
Figure CN119211596A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image data processing, and in particular to an RGBG to RGB conversion system, method and storage medium. Background Art
[0002] In the image processing process, RGBG, as a common image arrangement method, needs to be converted into RGB format before subsequent scaling, compression and other processing can be performed. The conversion process is performed in units of "rows" of image pixel arrangement, and one RGBG data is input in each clock cycle.
[0003] To achieve pipeline processing, a similar RGBG to RGB hardware circuit requires three SRAMs to ping-pong (a data transmission technology) the current row. The input and output timing of the conversion process is as follows Figure 1 As shown, when the Nth row of data is input, the data is immediately written into the first SRAM, and the first two rows of data are read out from the second and third SRAMs for calculation; when the N+1th row of data is input, the first two rows of data need to be read out from the first and third SRAMs for calculation, and the N+1th row of data is written into the second SRAM; when the N+2th row of data is input, the first two rows of data need to be read out from the first SRAM and the second SRAM for calculation, and the N+2th row of data is written into the third SRAM, and the entire data storage and reading is ping-pong stored in the third single-port SRAM. Therefore, the output timing delay includes at least the total delay of the continuous write and read operations, that is, 2×t wr +t rd , t wr is the write delay time, t rd is the read delay time. In the figure, vs / hs / data represent frame synchronization / line synchronization / data respectively, _i suffix represents input, _o suffix represents output, _wr suffix represents write, _rd suffix represents read, _wr_ptr represents write pointer, ram0 represents the first SRAM, ram1 represents the second SRAM, and ram2 represents the third SRAM.
[0004] For video processing chips, the consumption of storage resources is very huge. During the RGB conversion process, the first two rows of pixel data must be stored, which requires at least two SRAMs. Therefore, optimization can only be performed on the third SRAM to reduce SRAM resource consumption and reduce output timing delay. Therefore, how to reduce SRAM consumption and delay has become an urgent problem to be solved. Summary of the invention
[0005] The object of the present invention is to provide an RGBG to RGB conversion system, method and storage medium that can save storage resources of a memory and reduce delay time.
[0006] Technical solution: A RGBG to RGB conversion system, including a first memory, a second memory, a logic controller and a logic operation module.
[0007] The first memory and the second memory are interactively connected with the logic controller, and the first memory and the second memory perform read and write operations alternately every two clock cycles, wherein when the Nth row of data stream is input, the first memory reads the N-2th row of pixel data in the current clock cycle, and the second memory reads the N-1th row of pixel data in the next clock cycle, and the first memory writes the Nth row of pixel data; when the N+1th row of data stream is input, the first memory reads the Nth row of pixel data in the current clock cycle, and the second memory reads the N-1th row of pixel data in the next clock cycle, and the second memory writes the N+1th row of pixel data;
[0008] The logic controller is used to manage and coordinate the data flow read and write operations and control signals in the first memory and the second memory;
[0009] The logic operation module is connected to the logic controller and is used to realize the processing and conversion of pixel data.
[0010] Furthermore, the first memory and the second memory are any one of static random access memory, dynamic random access memory, synchronous dynamic random access memory, and video random access memory.
[0011] Furthermore, the first memory and the second memory are dual-port memories or single-port memories.
[0012] Furthermore, data transmission between the first memory and the second memory adopts direct memory access.
[0013] Furthermore, the logic controller is connected to the first memory and the second memory via a parallel interface, a serial interface or a differential interface.
[0014] Furthermore, the logic controller also includes one or more registers.
[0015] Furthermore, the logic controller is any one of a microcontroller, a digital signal processor, a field programmable gate array, and a programmable logic device.
[0016] Furthermore, it also includes a configuration register, and the configuration register is connected to the logic operation module and the logic controller respectively.
[0017] The configuration register is used to store parameters required by the logic controller or the logic operation module, and the parameters are weighting factor, data polarity, boundary filling mode, and row data length.
[0018] A RGBG to RGB conversion method, applied to an RGBG to RGB conversion system, comprises the following steps:
[0019] Input data stream and control signals;
[0020] According to the control signal, the first memory and the second memory are controlled to alternately perform read and write data stream operations in every two clock cycles, wherein when the Nth row of data stream is input, the first memory reads the N-2th row of pixel data in the current clock cycle, the second memory reads the N-1th row of pixel data in the next clock cycle, and the first memory writes the Nth row of pixel data; when the N+1th row of data stream is input, the first memory reads the Nth row of pixel data in the current clock cycle, the second memory reads the N-1th row of pixel data in the next clock cycle, and the second memory writes the N+1th row of pixel data;
[0021] Inputting the pixel data read from the first memory and the second memory and the pixel data of the current row into the logic operation module to perform color conversion operation;
[0022] Repeat the above steps until all data streams are processed.
[0023] Furthermore, the pixel data read from the first memory and the second memory are input into a logic operation module for color conversion operation, which also includes the steps of:
[0024] Data stream output timing adjustment and color conversion operations are performed according to the parameters output by the configuration register and the pixel data read by the first memory and the second memory, wherein the parameters are weighting factor, data polarity, boundary filling mode, and row data length.
[0025] A storage medium stores a computer program, which implements an RGBG to RGB conversion method when executed by a processor.
[0026] Technical effect: The present invention separates the operations of writing the current pixel data into the memory and reading the pixel data of the previous two rows, which are performed simultaneously in different memories in each cycle, into time-sharing operations in the same memory, forming a read-write-read-write cycle; the present invention changes the cycle of completing a read-write operation in every two cycles to reading or writing two pixel data in every two clock cycles, completing two read-write operation cycles. This not only saves the storage resources of the memory, but also reduces the delay time. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 This is the input and output timing diagram of the existing RGBG to RGB system;
[0028] Figure 2 is a schematic diagram of the system structure of RGBG to RGB conversion in an embodiment;
[0029] Figure 3 is a pixel unit matrix with pixel R as the origin;
[0030] Figure 4 is a pixel unit matrix with pixel B as the origin;
[0031] Figure 5 is a flow chart of a method for converting RGBG to RGB in an embodiment;
[0032] Figure 6 is an input and output timing diagram of the RGBG to RGB system of the embodiment;
[0033] Figure 7 is an RGBG pixel image;
[0034] Figure 8 is the RGB pixel image after conversion in the embodiment;
[0035] Fig. 9 It is a schematic diagram of the structure of a computer device in an embodiment;
[0036] Fig.10 It is another schematic diagram of the structure of a computer device according to an embodiment. DETAILED DESCRIPTION
[0037] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be understood by people with general skills in the field to which the present invention belongs. "Including" and similar words used in this article mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects.
[0038] like Figure 2 As shown, an embodiment of the present invention provides an RGBG to RGB conversion system, including a first memory 1, a second memory 2, a logic controller 3, a logic operation module 4 and a configuration register 5.
[0039] The first memory 1 and the second memory 2 are interactively connected with the logic controller 3, and the first memory 1 and the second memory 2 perform read and write operations alternately every two clock cycles, wherein, when the Nth row of data stream is input, the first memory 1 reads the N-2th row of pixel data in the current clock cycle, the second memory 2 reads the N-1th row of pixel data, and the first memory 1 writes the Nth row of pixel data in the next clock cycle; when the N+1th row of data stream is input, the first memory 1 reads the Nth row of pixel data in the current clock cycle, the second memory 2 reads the N-1th row of pixel data, and the second memory 2 writes the N+1th row of pixel data in the next clock cycle; the first memory 1 and the second memory 2 can perform read and write operations alternately to ensure the continuity and efficiency of data during processing.
[0040] In some embodiments, the first memory 1 and the second memory 2 are any one of a static random access memory, a dynamic random access memory, a synchronous dynamic random access memory, and a video random access memory.
[0041] In some embodiments, the first memory 1 and the second memory 2 are dual-port or single-port memories.
[0042] In some embodiments, data transmission between the first memory 1 and the second memory 2 adopts direct memory access, without the intervention of the central processing unit, allowing external devices to directly access the memory and efficiently and quickly transmit data.
[0043] In some embodiments, the logic controller 3 is used to manage and coordinate the data flow read and write operations and control signals in the first memory 1 and the second memory 2 .
[0044] In some embodiments, the logic controller 3 is connected to the first memory 1 and the second memory 2 using a parallel interface, a serial interface, or a differential interface.
[0045] In some embodiments, the logic controller 3 further includes one or more registers for buffering data streams.
[0046] In some embodiments, the logic controller 3 is any one of a microcontroller, a digital signal processor, a field programmable gate array, and a programmable logic device.
[0047] In some embodiments, the logic operation module 4 is used to implement pixel data processing and conversion;
[0048] In some embodiments, the configuration register 5 is connected to the logic operation module 4 and the logic controller 3 respectively.
[0049] The configuration register 5 is used to store parameters required by the logic controller 3 or the logic operation module 4, and the parameters are weighting factor, data polarity, boundary filling mode, and row data length.
[0050] In some embodiments, the logic controller 3 adjusts the timing according to the input data stream and the row data length to optimize the data processing efficiency. The configuration register 5 outputs the row data length to the logic controller 3 to ensure that each pixel data can be correctly accessed, read and written.
[0051] The logic operation module 4 performs color interpolation on the pixel data according to the weighting factor output by the configuration register 5 to estimate the missing color component value.
[0052] For example, if the input matrix of each color unit of RGBG is as follows Figure 3 and Figure 4 As shown, Figure 3 It is a matrix pixel unit with R as the coordinate origin. Figure 4 It is a matrix pixel unit with B as the coordinate origin.
[0053] The RGBG color units are weighted averaged to obtain the RGB output result. The weighted calculation formula is as follows:
[0054] RGB at R
[0055] R = R (0, 0);
[0056] G=[α 0 *G(-1,0)+α 1 *G(1,0)+α 2 *G(-1,1)+α 3 *G(1,1)];
[0057] B=[α 4 *B(0,-1)+α 5 *B(1,0)+α 6 *B(1,0)+α 7 *G(-1,0)];
[0058] RGB at B
[0059] R=[α 4 *R(0,-1)+α 5 *R(1,0)+α 6 *R(1,0)+α 7 *R(-1,0)];
[0060] G=[α 0 *G(-1,0)+α 1 *G(1,0)+α 2 *G(-1,1)+α3 *G(1,1)];
[0061] B=B(0,0);
[0062] In this design, the weighting factors α0 to α7 are all register-configurable, and their bit width and whether they are signed numbers (an 8-bit unsigned number is used in this embodiment) can be determined according to requirements such as algorithm accuracy.
[0063] In some embodiments, the weighting factors are generally all set to 1 / 4 by default. The weighting factors are weight values used for color conversion, that is, the weight of each color component is 1 / 4.
[0064] In some embodiments, the logic operation module 4 performs polarity conversion on the pixel data according to the data polarity output by the configuration register 5 to adapt to specific image processing requirements or achieve specific visual effects. The data polarity can be determined according to the specific arrangement of the input image. Suppose we have an input image with a resolution of 1024x768 pixels and each pixel consists of 4 bytes (RGBG). In this case, each pixel consists of 4 bytes (RGBG) because each pixel contains a red (R), a green (G), a blue (B) and an additional green (G). Data polarity refers to the arrangement of the input image in pixels, which can be RGBG, BGRG, GBGR, and GRGB.
[0065] In some embodiments, the logic operation module 4 processes the missing data at the edge of the image according to the border filling mode output by the configuration register 5 to avoid discontinuity or blurring of the image edge. The user can select the border filling mode according to the sensitivity of the perception of the image border.
[0066] For example, if the user is highly sensitive to the image boundary and wants to maintain the clarity of the image boundary, you can select all 0 as the boundary filling mode. If the user is less sensitive to the image boundary and wants the image boundary to be smoother, you can select the last data as the boundary filling mode. When processing the edge of the image, if the user selects all 0 as the boundary filling mode, the missing data at the edge will be filled with 0. If the user selects the last data as the boundary filling mode, the missing data at the edge will be filled with the last processed data.
[0067] The present invention changes the original operation of writing the current pixel data into the memory and reading the pixel data of the previous two rows, which are performed simultaneously in different memories in each cycle, into time-sharing operations in the same memory, forming a read-write-read-write cycle; the present invention changes the original cycle of completing a read-write operation in every two cycles into reading or writing two pixel data in every two clock cycles, completing two read-write operation cycles. This not only saves the storage resources of the memory, but also reduces the delay time.
[0068] like Figure 5 As shown, another embodiment provides a RGBG to RGB conversion method, which is applied to a conversion system and includes the following steps:
[0069] S1: input data stream and control signal;
[0070] S2: Control the first memory and the second memory to perform alternate read and write data stream operations every two clock cycles according to the control signal, wherein when the Nth row of data stream is input, the first memory reads the N-2th row of pixel data in the current clock cycle, the second memory reads the N-1th row of pixel data in the next clock cycle, and the first memory writes the Nth row of pixel data; when the N+1th row of data stream is input, the first memory reads the Nth row of pixel data in the current clock cycle, the second memory reads the N-1th row of pixel data in the next clock cycle, and the second memory writes the N+1th row of pixel data;
[0071] S3: Inputting the pixel data read from the first memory and the second memory and the pixel data of the current row into the logic operation module for color conversion operation; further comprising the steps of:
[0072] Data stream output timing adjustment and color conversion operations are performed according to the parameters output by the configuration register and the pixel data read by the first memory and the second memory, and the parameters are weighting factor, data polarity, boundary filling mode, and row data length.
[0073] S4: Repeat the above steps until all data streams are processed.
[0074] The timing diagram output by the above embodiment is as follows: Figure 6 As shown in the figure, hs is the line synchronization signal, vs is the frame synchronization signal, de is the input data valid signal, the _i suffix indicates input, the _o suffix indicates output, the _wr suffix indicates write, the _rd suffix indicates read, _rdata indicates read data, _wdata indicates write data, ram0 indicates the first memory, and ram1 indicates the second memory.
[0075] Before the de_i valid signal arrives, the parameters in the configuration register 5 are configured by the host computer or other means. The configuration parameters include weighting factors, data polarity, boundary filling mode, row data length, etc. These configuration parameters are configured before the frame synchronization signal is valid, cannot be changed in a frame of data, and can be freely configured between frames. After each row of data is processed by the logic control module, it is written into the first memory 1 or the second memory 2. At the same time, the data in the first memory 1 and the second memory 2 are used for calculation. Before each row of data starts, the first memory 1 and the second memory 2 read out the pixel data of the previous two rows, and perform calculations with the current input pixel data after one cycle. In order to achieve better output timing characteristics, the calculation results are output after the configuration register 5 is tapped. Therefore, the output delay is 2 cycles. The first two rows of data in each frame and the last data in each row need to be filled with data. The filling data can be selected as all 0 or the previous data. After the calculation is completed, it is output in order according to the input timing. Compared with the prior art, it is at least greater than 2 clock cycles (2×t wr +t rd ) delay time, the present invention reduces the clock delay time.
[0076] The RGB image converted through the above steps is as follows Figure 8 As shown, the RGBG image before conversion is as follows Figure 7 shown.
[0077] like Fig. 9 As shown, another embodiment further provides a computer device, including: a memory 12 and a processor 11, wherein the memory 12 stores a computer program, and when the computer program is executed by the processor 11, the conversion system in the above embodiment is implemented.
[0078] like Fig.10 As shown, in another embodiment, the computer device 20 further includes: an output interface 23 for outputting results; a communication interface 24 for communicating and transmitting signals; and an antenna 25 for transmitting or receiving signals.
[0079] It should be noted that the processor 21 in this embodiment can be an image processing chip or an integrated circuit chip, which has the ability to process image signals. In the implementation process, each step of the above method embodiment can be completed by the hardware integrated logic circuit in the processor or the instruction in the form of software. The above processor can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices. The methods, steps and logic block diagrams disclosed in this embodiment can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in this embodiment can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor can be combined to perform. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.
[0080] It can be understood that the memory 22 in this embodiment can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.
[0081] In another embodiment, a readable storage medium is provided, wherein a program is stored in the readable storage medium, and when the program is executed, the method described in any one of the above embodiments is implemented.
[0082] It is worth noting that if the method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling an electronic device to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk.
[0083] Although the embodiments of the present invention are described in detail above, it is obvious to those skilled in the art that various modifications and variations can be made to these embodiments. However, it should be understood that such modifications and variations are within the scope and spirit of the present invention as described in the claims. Moreover, the present invention described herein may have other embodiments and may be implemented or realized in a variety of ways.
Claims
1. A RGBG to RGB conversion system, characterized in that: It includes a first memory, a second memory, a logic controller and a logic operation module. The first memory and the second memory are interactively connected with the logic controller, and the first memory and the second memory perform read and write operations alternately every two clock cycles, wherein when the Nth row of data stream is input, the first memory reads the N-2th row of pixel data in the current clock cycle, and the second memory reads the N-1th row of pixel data in the next clock cycle, and the first memory writes the Nth row of pixel data; when the N+1th row of data stream is input, the first memory reads the Nth row of pixel data in the current clock cycle, and the second memory reads the N-1th row of pixel data in the next clock cycle, and the second memory writes the N+1th row of pixel data; The logic controller is used to manage and coordinate the data flow read and write operations and control signals in the first memory and the second memory; The logic operation module is connected to the logic controller and is used to realize the processing and conversion of pixel data.
2. The conversion system according to claim 1, characterized in that The first memory and the second memory are any one of static random access memory, dynamic random access memory, synchronous dynamic random access memory, and video random access memory.
3. The conversion system according to claim 2, characterized in that: The first memory and the second memory are dual-port memories or single-port memories.
4. The conversion system according to claim 3, characterized in that: The data transmission between the first memory and the second memory adopts a direct memory access method.
5. The conversion system according to claim 1, characterized in that: The logic controller is connected to the first memory and the second memory via a parallel interface, a serial interface or a differential interface.
6. The conversion system according to claim 5, characterized in that The logic controller also includes one or more registers.
7. The conversion system according to claim 6, characterized in that The logic controller is any one of a microcontroller, a digital signal processor, a field programmable gate array, and a programmable logic device.
8. The conversion system according to claim 1, characterized in that: It also includes a configuration register, which is connected to the logic operation module and the logic controller respectively. The configuration register is used to store parameters required by the logic controller or the logic operation module, and the parameters are weighting factor, data polarity, boundary filling mode, and row data length.
9. A method for converting RGBG to RGB, characterized in that: The conversion system according to any one of claims 1 to 8 comprises the following steps: Input data stream and control signals; According to the control signal, the first memory and the second memory are controlled to alternately perform read and write data stream operations in every two clock cycles, wherein when the Nth row of data stream is input, the first memory reads the N-2th row of pixel data in the current clock cycle, the second memory reads the N-1th row of pixel data in the next clock cycle, and the first memory writes the Nth row of pixel data; when the N+1th row of data stream is input, the first memory reads the Nth row of pixel data in the current clock cycle, the second memory reads the N-1th row of pixel data in the next clock cycle, and the second memory writes the N+1th row of pixel data; Inputting the pixel data read from the first memory and the second memory and the pixel data of the current row into the logic operation module to perform color conversion operation; Repeat the above steps until all data streams are processed.
10. The conversion method according to claim 9, characterized in that: Inputting the pixel data read from the first memory and the second memory into a logic operation module for color conversion operation, further comprising the steps of: Data stream output timing adjustment and color conversion operations are performed according to the parameters output by the configuration register and the pixel data read by the first memory and the second memory, wherein the parameters are weighting factor, data polarity, boundary filling mode, and row data length.
11. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the conversion method according to any one of claims 9 to 10 is implemented.