A method and device for video processing in a baseboard management control chip

Abstract

The present invention provides a method, system, device and storage medium for video processing in a baseboard management control chip. The method includes: creating a RISC-V extension instruction set and compiling it using a compiler; in response to receiving a start instruction in the RISC-V extension instruction set, calculating the address and data length for each read of the storage unit according to the resolution information, compression mode and RGB initial address information in the start instruction; generating a load instruction in the RISC-V extension instruction set according to the address and data length, and sending the RGB data obtained according to the load instruction to a multiply-accumulate array for parallel RGB-to-YUV conversion; calculating the UV data to be discarded in the calculation result of the multiply-accumulate array according to the current compression mode, calculating the storage address according to the discarded UV data, and storing the YUV data from the cache to the storage unit according to the storage address. The present invention performs the conversion of RGB2YUV and YUV2BLOCK at the CPU instruction set level, which speeds up the processing speed.

Description

Technical Field

[0001] The present invention relates to the field of video processing, and more specifically, particularly to a method, system, device, and storage medium for video processing in a baseboard management control chip. Background Art

[0002] With the increasingly fierce international competition, the requirements for domestic processors in various industries are becoming more urgent. RISC-V is pronounced as "RISC Five", which means the fifth generation of reduced instruction set processors. It is a brand-new instruction set architecture and is open source, which can be freely used by any academic institution or commercial organization, and has the advantage of being independently controllable. Moreover, the instruction set is open and can be extended and customized by users.

[0003] There are two modes of video processing in traditional baseboard management control chips based on RISC-V processors:

[0004] Mode 1: Pure software processing. That is, after the video data at the host end is transmitted to the baseboard management control chip through PCIe (Peripheral Component Interconnect express, a high-speed serial computer expansion bus standard), a raw RGB (Red, Green, Blue) video signal is generated, and then the CPU of the baseboard management control chip performs pure software processing (including RGB2YUV conversion, BLOCK conversion, JPEG (Joint Photographic Experts Group) compression, etc.). That is, the software compression program compresses the raw video data, and the compressed video data is written to DDR (Double Data Rate, double data rate synchronous dynamic random access memory) and sent to the remote end through EMAC (Ethernet MAC) for display.

[0005] Mode 2: Pure hardware processing. After the video data at the host end is transmitted to the baseboard management control chip through PCIe, through the color space conversion module (RGB2YUV), the raw RGB format video data is converted into YUV (Luminance, Chrominance, Saturation) format data, and then the Y, U, and V data are cached in the on-chip storage resources (FIFO). According to the requirements of BLOCK format conversion (supporting YUV444 / YUV422 / YUV420 compression formats), 16 Y_FIFOs, 16 U_FIFOs, and 16 V_FIFOs are required. At the same time, according to project practice experience, for the maximum resolution of (1920*1200), the depth of the FIFO is 16384 and the width is 8 bits to ensure that the FIFO will not be full and no data will be lost. Then, it is input to the video compression IP in the order of BLOCK. After compression, the data is written to DDR, and the EMAC (Ethernet MAC) driver reads the compressed data and transmits the video data to the remote through the network for remote display.

[0006] The drawbacks of the traditional solution 1 are obvious. That is, the processing speed is very slow, and it occupies the CPU for a long time, seriously affecting the operation of other programs / processes on the substrate management control chip.

[0007] The drawbacks of the traditional solution 2 are that it requires a large amount of on-chip resources. For chip projects, the on-chip RAM (Random Access Memory) resources are very precious (because it will increase the chip area). At the same time, a large number and large capacity of RAM pose great difficulties for the chip's timing constraints, backend design, and packaging manufacturing. And the processing speed is relatively slow. Only one Y / U / V component (8 bits) can be read in one clock cycle, which affects the video processing speed to a certain extent and thus affects the overall performance of the chip. Summary of the Invention

[0008] In view of this, the purpose of the embodiments of the present invention is to provide a method, system, computer device, and computer-readable storage medium for video processing in a substrate management control chip. The present invention performs the conversion of RGB2YUV and YUV2BLOCK from the CPU instruction set level, accelerating the video processing speed (the operating frequency of the embedded CPU is generally above 800 MHz). At the same time, it avoids the excessive occupation of FIFO (on-chip resources) in the traditional solution, reduces the project cost, and also reduces the risk of project failure caused by the increased timing constraints and packaging manufacturing difficulties due to the too large FIFO capacity in the traditional solution.

[0009] Based on the above purpose, one aspect of the embodiments of the present invention provides a method for video processing in a substrate management control chip, including the following steps: creating a RISC-V extended instruction set and compiling it using a compiler; in response to receiving a start instruction in the RISC-V extended instruction set, calculating the address and data length of each read storage unit according to the resolution information, compression mode, and RGB initial address information in the start instruction; generating a load instruction in the RISC-V extended instruction set according to the address and data length, and sending the RGB data obtained according to the load instruction to a multiply-accumulate array for parallel RGB-to-YUV conversion; calculating the UV data to be discarded in the calculation result of the multiply-accumulate array according to the current compression mode, calculating the storage address according to the discarded UV data, and storing the YUV data from the cache to the storage unit according to the storage address.

[0010] In some embodiments, generating the load instruction in the RISC-V extension instruction set according to the address and the data length includes: writing the address into a first target register, writing the data length into a second target register, and loading the values of the first target register and the second target register into a conventional load instruction.

[0011] In some embodiments, calculating the UV data to be discarded in the calculation result of the multiply-accumulate array according to the current compression mode includes: in response to the current compression mode being YUV444, retaining all the YUV data; in response to the current compression mode being YUV422, retaining the UV components of the even / odd columns of all rows and discarding the remaining UV components; and in response to the current compression mode being YUV420, retaining the UV components of the even columns of the even rows and discarding the remaining UV components.

[0012] In some embodiments, calculating the storage address according to the discarded UV data includes: calculating the memory address of the store instruction and the length of a single storage after discarding the UV data.

[0013] In some embodiments, storing the YUV data from the cache to the storage unit according to the storage address includes: writing the memory address into a third target register, writing the length of a single storage after discarding the UV data into a fourth target register, and storing the YUV data according to the values of the third target register and the fourth target register.

[0014] In some embodiments, the method further includes: reading YUV data from the multiply-accumulate array and caching the YUV data after discarding the UV data in the order of the Y component, the U component, and the V component.

[0015] In some embodiments, creating the RISC-V extension instruction set and compiling it using a compiler includes: setting, as needed, whether it is necessary to read a first source register and a second source register in each extension instruction, and setting whether it is necessary to write data to a target register.

[0016] On the other hand, an embodiment of the present invention provides a video processing system in a baseboard management control chip, including: an instruction module configured to create a RISC-V extended instruction set and compile it using a compiler; a calculation module configured to, in response to receiving a start instruction in the RISC-V extended instruction set, calculate the address and data length of each read memory unit according to the resolution information, compression mode, and RGB initial address information in the start instruction; a conversion module configured to generate a load instruction in the RISC-V extended instruction set according to the address and data length, and send the RGB data obtained according to the load instruction to a multiply-accumulate array for parallel RGB-to-YUV conversion; an execution module configured to calculate the UV data to be discarded in the calculation result of the multiply-accumulate array according to the current compression mode, calculate the storage address according to the discarded UV data, and store the YUV data from the cache to the memory unit according to the storage address.

[0017] In yet another aspect, an embodiment of the present invention further provides a computer device, including: at least one processor; and a memory storing computer instructions executable on the processor, and when the instructions are executed by the processor, the steps of the above method are implemented.

[0018] In still another aspect, an embodiment of the present invention further provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above method.

[0019] The present invention has the following beneficial technical effects: The conversion between RGB2YUV and YUV2BLOCK is performed at the CPU instruction set level, which speeds up the video processing speed. At the same time, it avoids the excessive occupation of FIFO (on-chip resources) in the traditional solution, reduces the project cost, and also reduces the risk of project failure caused by the increased timing constraints and packaging manufacturing difficulties due to the excessive FIFO capacity in the traditional solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other embodiments can be obtained based on these drawings.

[0021] Figure 1 It is a schematic diagram of an embodiment of the method for video processing in the baseboard management control chip provided by the present invention;

[0022] Figure 2 It is a schematic diagram of the architecture of the RISC-V coprocessor in the present invention;

[0023] Figure 3 Schematic diagram of an embodiment of a video processing system in a baseboard management control chip provided by the present invention;

[0024] Figure 4 Hardware structure schematic diagram of an embodiment of a computer device for video processing in a baseboard management control chip provided by the present invention;

[0025] Figure 5 Schematic diagram of an embodiment of a computer storage medium for video processing in a baseboard management control chip provided by the present invention. Detailed implementation manners

[0026] To make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the following further elaborates on the embodiments of the present invention in detail with reference to specific embodiments and the accompanying drawings.

[0027] It should be noted that all the expressions using "first" and "second" in the embodiments of the present invention are used to distinguish two entities or parameters with the same name but different identities. It can be seen that "first" and "second" are only for the convenience of expression and should not be construed as limitations on the embodiments of the present invention. This will not be elaborated one by one in the subsequent embodiments.

[0028] In the first aspect of the embodiments of the present invention, an embodiment of a method for video processing in a baseboard management control chip is proposed. Figure 1 The following shows a schematic diagram of an embodiment of a method for video processing in a baseboard management control chip provided by the present invention. As Figure 1 shown, the embodiments of the present invention include the following steps:

[0029] S1. Create a RISC-V extension instruction set and compile it using a compiler;

[0030] S2. In response to receiving a start instruction in the RISC-V extension instruction set, calculate the address and data length for each read of the storage unit according to the resolution information, compression mode, and RGB initial address information in the start instruction;

[0031] S3. Generate a load instruction in the RISC-V extension instruction set according to the address and data length, and send the RGB data obtained according to the load instruction to a multiply-accumulate array for parallel RGB-to-YUV conversion; and

[0032] S4. Calculate the UV data to be discarded in the calculation result of the multiply-accumulate array according to the current compression mode, calculate the storage address according to the discarded UV data, and store the YUV data from the cache to the storage unit according to the storage address.

[0033] Figure 2 This is a schematic diagram of the architecture of the RISC-V coprocessor in the present invention, which will be described in conjunction with Figure 2 the embodiments of the present invention.

[0034] Create a RISC-V extension instruction set and compile it using a compiler.

[0035] The following table shows the general format of RISC-V extension instructions:

[0036] Table 1 - General Format of RISC-V Extension Instructions

[0037] funct7 rs2 rs1 xd xs1 xs2 rd opcode

[0038] Among them, Opcode[6:0] indicates the instruction type determined according to the opcode encoding table; rd[4:0] indicates the destination register; xs1, xs2, rd (each 1 bit) respectively indicate whether the instruction needs to read source registers rs1 and rs2 and write back to the destination register rd. A value of 1 means it is needed, and a value of 0 means it is not needed; rs1[4:0] indicates source register 1, and rs2[4:0] indicates source register 2; funct7[6:0] indicates additional encoding space. Therefore, one custom instruction group (0 - 3) corresponds to a maximum of 128 extension instructions.

[0039] In some embodiments, the creating a RISC-V extension instruction set and compiling it using a compiler includes: setting whether to read the first source register and the second source register in each extension instruction as needed, and setting whether to write data to the destination register. Each extension instruction can be set according to Table 2.

[0040] Table 2 - RISC-V Extension Instructions

[0041]

[0042]

[0043] The processing pipeline order of the CPU is: fetch, decode, execute, memory access, write back. The embodiments of the present invention are located between execute, memory access, and write back. At the same time, the interfaces of the coprocessor with the EXU (CPU internal execution unit), LSU (storage unit), and WB (write-back unit) are ROCC (Rocket Custom Coprocesser) interfaces (taking the Rocket RISC-V core as an example).

[0044] In response to receiving a start instruction in the RISC-V extension instruction set, calculate the address and data length of each read storage unit according to the resolution information, compression mode, and RGB initial address information in the start instruction.

[0045] The embodiments of the present invention require additional compiler modifications to compile the corresponding instructions in Table 2 above. However, the embodiments of the present invention are only limited to describing the hardware implementation and do not describe the compiler modifications. After the software program is compiled, a BEGIN_PRC instruction is generated. This instruction writes 0 to target register 0. After being parsed by the instruction parsing unit (INS_ANA), the coprocessor is ready to start working. According to the value of operand rs1, the resolution information of the current frame to be processed is obtained. (rsl_l * rsl_w). For example, if the resolution is 1024 * 768, then the [15:0] of rs1 stores rsl_w, that is, 768, and the [31:16] of rs1 stores rsl_l, that is, 1024. According to the value of operand rs2, the compression mode of the current frame is obtained, which can be represented by [1:0] of rs2. rs2 = 0 represents the YUV444 compression mode, rs2 = 1 represents the YUV422 compression mode, and rs2 = 2 represents the YUV420 compression mode. Then the RGB_YUV_PRC instruction is run. According to the RGB initial address information source_data_addr stored in operand rs1, the resolution information (rsl_l * rsl_w) obtained from the BEGIN_PRC instruction, and the compression mode (yuv_mode), the loading unit (CUL_LOAD) is used to calculate the address and data length of each read of the LSU.

[0046] In the YUV444 mode, when the source video data is written to the DDR, it should be in groups of 8 rows.

[0047] The RGB data of the 0th / 8th / 16th / 24th... rows is written to source_data_addr;

[0048] The RGB data of the 1st / 9th / 17th / 25th... rows is written to source_data_addr + rsl_l;

[0049] The RGB data of the 2nd / 10th / 18th / 26th... rows is written to source_data_addr + rsl_l * 2;

[0050] ......

[0051] The RGB data of the 7th / 15th / 23rd / 31st... rows is written to source_data_addr + rsl_l * 7;

[0052] In the CUL_LOAD unit, the generated read address and the number of read data are

[0053] For the 0th time, the address is Source_data_addr, and the number is 8;

[0054] For the first time, the address is Source_data_addr + rsl, and the number is 8;

[0055] For the second time, the address is Source_data_addr + rsl * 2, and the number is 8;

[0056] For the third time, the address is Source_data_addr + rsl * 3, and the number is 8;

[0057] ……

[0058] For the seventh time, the address is Source_data_addr + rsl * 7, and the number is 8;

[0059] The data of the first BLOCK is read back above, and then the address of the second group of BLOCKs is calculated continuously.

[0060] For the 0th time, the address is Source_data_addr + 8, and the number is 8;

[0061] For the first time, the address is Source_data_addr + rsl + 8, and the number is 8;

[0062] For the second time, the address is Source_data_addr + rsl * 2 + 8, and the number is 8;

[0063] For the third time, the address is Source_data_addr + rsl * 3 + 8, and the number is 8;

[0064] ……

[0065] For the seventh time, the address is Source_data_addr + rsl * 7 + 8, and the number is 8;

[0066] Calculate the BLOCK address of the Nth group.

[0067] For the 0th time, the address is Source_data_addr + 8 * (N - 1), and the number is 8;

[0068] For the first time, the address is Source_data_addr + rsl + 8 * (N - 1), and the number is 8;

[0069] For the second time, the address is Source_data_addr + rsl * 2 + 8 * (N - 1), and the number is 8;

[0070] For the third time, the address is Source_data_addr + rsl * 3 + 8 * (N - 1), and the number is 8;

[0071] ……

[0072] The 7th time, the address is Source_data_addr + rsl * 7 + 8 * (N - 1), and the number is 8;

[0073] where N <= rsl / 8. After exceeding rsl / 8, N is reset to 0, indicating that 8 lines of data have been read and the address is reset.

[0074] Write the calculated address information and the number of read data to target register 1 and target register 2 in sequence. It should be noted that the modified compiler needs to read the values of these two registers to generate the LOAD_RGB load instruction described below.

[0075] In YUV422 mode, when the source video data is written to the DDR, it is grouped by 16 lines.

[0076] The RGB data of the 0 / 16 / 32...th lines is written to source_data_addr;

[0077] The RGB data of the 1 / 17 / 33...th lines is written to source_data_addr + rsl_l;

[0078] The RGB data of the 2 / 18 / 34...th lines is written to source_data_addr + rsl_l * 2;

[0079] ...

[0080] The RGB data of the 15 / 31 / 47...th lines is written to source_data_addr + rsl_l * 15;

[0081] In the CUL_LOAD unit, the generated read address and the number of read data are:

[0082] The 0th time, the address is Source_data_addr, and the number is 16;

[0083] The 1st time, the address is Source_data_addr + rsl, and the number is 16;

[0084] The 2nd time, the address is Source_data_addr + rsl * 2, and the number is 16;

[0085] The 3rd time, the address is Source_data_addr + rsl * 3, and the number is 16;

[0086] ...

[0087] The 15th time, the address is Source_data_addr + rsl * 15, and the number is 16;

[0088] The above has read the data of the first BLOCK, and then continues to calculate the address of the second group of BLOCKs.

[0089] For the 0th time, the address is Source_data_addr + 16, and the number is 16;

[0090] For the 1st time, the address is Source_data_addr + rsl + 16, and the number is 16;

[0091] For the 2nd time, the address is Source_data_addr + rsl * 2 + 16, and the number is 16;

[0092] For the 3rd time, the address is Source_data_addr + rsl * 3 + 16, and the number is 16;

[0093] ……

[0094] For the 15th time, the address is Source_data_addr + rsl * 15 + 16, and the number is 16;

[0095] Calculate the BLOCK address of the Nth group:

[0096] For the 0th time, the address is Source_data_addr + 16 * (N - 1), and the number is 16;

[0097] For the 1st time, the address is Source_data_addr + rsl + 16 * (N - 1), and the number is 16;

[0098] For the 2nd time, the address is Source_data_addr + rsl * 2 + 16 * (N - 1), and the number is 16;

[0099] For the 3rd time, the address is Source_data_addr + rsl * 3 + 16 * (N - 1), and the number is 16;

[0100] ……

[0101] For the 15th time, the address is Source_data_addr + rsl * 15 + 16 * (N - 1), and the number is 16;

[0102] Where N <= rsl / 16. After exceeding rsl / 16, N is reset to 0, indicating that 16 lines of data have been read and the address is reset.

[0103] Write the calculated address information and the number of read data to target register 1 and target register 2 in sequence. It should be noted that the modified compiler needs to read the values of these two registers to generate the instruction LOAD_RGB described below.

[0104] When in YUV420 mode, when writing the source video data to the DDR, it should be grouped by 16 lines.

[0105] The RGB data of the 0th / 16th / 32nd... lines is written to source_data_addr;

[0106] The RGB data of the 1st / 17th / 33rd... lines is written to source_data_addr + rsl_l;

[0107] The RGB data of the 2nd / 18th / 34th... lines is written to source_data_addr + rsl_l * 2;

[0108] ...

[0109] The RGB data of the 15th / 31st / 47th... lines is written to source_data_addr + rsl_l * 15;

[0110] In the CUL_LOAD unit, the generated read address and the number of read data are:

[0111] For the 0th time, the address is Source_data_addr, and the number is 16;

[0112] For the 1st time, the address is Source_data_addr + rsl, and the number is 16;

[0113] For the 2nd time, the address is Source_data_addr + rsl * 2, and the number is 16;

[0114] For the 3rd time, the address is Source_data_addr + rsl * 3, and the number is 16;

[0115] ...

[0116] For the 15th time, the address is Source_data_addr + rsl * 15, and the number is 16;

[0117] The data of the first BLOCK is read back as above, and then the address of the 2nd group of BLOCKs is calculated continuously.

[0118] For the 0th time, the address is Source_data_addr + 16, and the number is 16;

[0119] For the 1st time, the address is Source_data_addr + rsl + 16, and the number is 16;

[0120] For the 2nd time, the address is Source_data_addr + rsl * 2 + 16, and the number is 16;

[0121] The 3rd time, the address is Source_data_addr + rsl * 3 + 16, and the number is 16;

[0122] ……

[0123] The 15th time, the address is Source_data_addr + rsl * 15 + 16, and the number is 16;

[0124] Calculate the BLOCK address of the Nth group:

[0125] The 0th time, the address is Source_data_addr + 16 * (N - 1), and the number is 16;

[0126] The 1st time, the address is Source_data_addr + rsl + 16 * (N - 1), and the number is 16;

[0127] The 2nd time, the address is Source_data_addr + rsl * 2 + 16 * (N - 1), and the number is 16;

[0128] The 3rd time, the address is Source_data_addr + rsl * 3 + 16 * (N - 1), and the number is 16;

[0129] ……

[0130] The 15th time, the address is Source_data_addr + rsl * 15 + 16 * (N - 1), and the number is 16.

[0131] Where N <= rsl / 16. After exceeding rsl / 16, N is reset to 0, indicating that 16 lines of data have been read and the address is reset.

[0132] Write the calculated address information and the number of read data to target register 1 and target register 2 in sequence. It should be noted that the modified compiler needs to read the values of these two registers to generate the instruction LOAD_RGB described below.

[0133] Generate a load instruction in the RISC-V extension instruction set according to the address and data length, and send the RGB data obtained according to the load instruction to the multiply-accumulate array for parallel RGB-to-YUV conversion.

[0134] In some embodiments, generating the load instruction in the RISC-V extension instruction set according to the address and data length includes: writing the address to a first target register, writing the data length to a second target register, and loading the values of the first target register and the second target register into a traditional load instruction.

[0135] The LOAD_RGB instruction is similar to a traditional load instruction. After the traditional load instruction runs, the modified compiler runs to obtain the values of target register 1 and target register 2, which are then loaded into the traditional LOAD instruction. The LOAD instruction runs on the RISC-V processor. According to the CUL_LOAD calculation, the target register 1 (load address) and target register 2 (load length) are used to load the original RGB data into the RISC-V coprocessor. Then the LOAD_RGB instruction is run, which enables the RGB2YUV conversion and sends the RGB data loaded by the LOAD instruction to the MUL_ADD_ARRAY (multiplier-accumulator array) for parallel RGB-to-YUV conversion. That is, the 8 / 16 read-back RGB data are converted in parallel for the number of RGB2YUV conversions, greatly accelerating the conversion speed.

[0136] Calculate the UV data to be discarded in the calculation result of the multiplier-accumulator array according to the current compression mode, calculate the storage address based on the discarded UV data, and store the YUV data from the cache to the storage unit according to the storage address. Run the YUV_BLOCK_PRC instruction, which completes the calculation of discarding the YUV data and the store (storage) address.

[0137] In some embodiments, calculating the UV data to be discarded in the calculation result of the multiplier-accumulator array according to the current compression mode includes: in response to the current compression mode being YUV444, retaining all the YUV data; in response to the current compression mode being YUV422, retaining the UV components of the even / odd columns of all rows and discarding the remaining UV components; and in response to the current compression mode being YUV420, retaining the UV components of the even columns of the even rows and discarding the remaining UV components. After being parsed by INS_ANA, the YUV_BLOCK_PRC instruction enters the storage discard (DROP_STORE) unit to calculate the part of the UV data to be discarded according to the current compression mode (YUV444 / YUV422 / YUV420) of the calculation result of the MUL_ADD_ARRAY. In the YUV444 mode, all the YUV data are retained; in the YUV422 mode, the UV components of the even / odd columns of all rows are retained and the remaining UV components are discarded; in the YUV420 mode, the UV components of the even columns of the even rows are retained and the remaining UV components are discarded. Therefore, the DROP_STORE unit has the hardware logic to calculate the row / column number of the current pixel point according to the resolution information.

[0138] In some embodiments, calculating the storage address based on the discarded UV data includes: calculating the memory address of the store instruction and the length of a single storage after discarding the UV data.

[0139] In some embodiments, the method further includes: reading YUV data from the multiply-accumulate array, and caching the YUV data after discarding the UV data in the order of Y component, U component, and V component.

[0140] Before the traditional STORE operation, the STORE_YUV instruction reads YUV data from the MUL_ADD_ARRAY array, discards some of the UV components according to the calculation result of YUV_BLOCK_PRC, and buffers the YUV data to be STORED. The buffering order is to buffer the Y component first, then the U component, and then the V component. After the STORE instruction, the Video CMP (Video Compression IP) of the baseboard management control chip can directly read the DDR sequentially.

[0141] In some embodiments, storing the YUV data from the cache to the storage unit according to the storage address includes: writing the memory address to a third target register, writing the length of a single storage after discarding the UV data to a fourth target register, and storing the YUV data according to the values of the third target register and the fourth target register. Calculate the memory address used by the traditional STORE instruction. The calculation of this address is relatively simple and can be accumulated sequentially according to the operand rs1 (the starting address of the YUV data in the LSU), and set the address to the target register 3. The length of a single STORE is the length of the YUV data after being discarded by DROP_STORE, and set the length to the target register 4. Through the modified compiler, this instruction obtains the values of the target register 3 and the target register 4. The STORE instruction runs on the RISC-V coprocessor and stores the YUV BLOCK data from the cache (the YUV data is buffered in the order of BLOCK) to the LSU according to the target register 3 (STORE address) and the target register 4 (STORE length).

[0142] It should be noted that due to the automatic update mechanism of the cache in the RISC-V processor, it will default to update the addresses that the RISC-V processor frequently reads and writes recently. Therefore, most of the original RGB data will be in the cache (because the RISC-V coprocessor always reads RGB data from 8 / 16 initial addresses), and the reading speed of the RISC-V core will be extremely fast. After modifying the instruction set and the corresponding hardware design, the RGB2YUV and YUV2BLOCK steps in the traditional solution are completed at the CPU level, and the running speed of the instruction set is extremely fast. At the same time, it avoids the disadvantages of a large number of on-chip caches such as FIFOs in the traditional solution, greatly accelerating the video processing speed in the baseboard management control chip based on the RISC-V processor and improving the overall performance of the chip.

[0143] It should be particularly pointed out that each step in each embodiment of the above method for video processing in the baseboard management control chip can be mutually crossed, replaced, added, or deleted. Therefore, these reasonable permutation and combination transformations for the method of video processing in the baseboard management control chip should also fall within the protection scope of the present invention, and the protection scope of the present invention should not be limited to the embodiments.

[0144] Based on the above purposes, in the second aspect of the embodiments of the present invention, a system for video processing in a baseboard management control chip is proposed. As Figure 3 shown, the system 200 includes the following modules: an instruction module configured to create a RISC-V extended instruction set and compile it using a compiler; a calculation module configured to, in response to receiving a start instruction in the RISC-V extended instruction set, calculate the address and data length of each read storage unit according to the resolution information, compression mode, and RGB initial address information in the start instruction; a conversion module configured to generate a load instruction in the RISC-V extended instruction set according to the address and data length, and send the RGB data obtained according to the load instruction to a multiply-accumulate array for parallel RGB-to-YUV conversion; an execution module configured to calculate the UV data to be discarded in the calculation result of the multiply-accumulate array according to the current compression mode, calculate the storage address according to the discarded UV data, and store the YUV data from the cache to the storage unit according to the storage address.

[0145] In some embodiments, the conversion module is configured to: write the address into a first target register, write the data length into a second target register, and load the values of the first target register and the second target register into a traditional load instruction.

[0146] In some embodiments, the execution module is configured to: retain all the YUV data in response to the current compression mode being YUV444; retain the UV components of the even / odd columns of all rows and discard the remaining UV components in response to the current compression mode being YUV422; and retain the UV components of the even columns of the even rows and discard the remaining UV components in response to the current compression mode being YUV420.

[0147] In some embodiments, the execution module is configured to: calculate the memory address of the storage instruction and the length of a single storage after discarding the UV data.

[0148] In some embodiments, the execution module is configured to: write the memory address into a third target register, write the length of a single storage after discarding the UV data into a fourth target register, and store the YUV data according to the values of the third target register and the fourth target register.

[0149] In some embodiments, the system further includes a cache module configured to: read the YUV data from the multiply-accumulate array and cache the YUV data after discarding the UV data in the order of the Y component, the U component, and the V component.

[0150] In some embodiments, the instruction module is configured to: set whether it is necessary to read the first source register and the second source register and set whether it is necessary to write data to the target register in each extended instruction as needed.

[0151] For the above purposes, in a third aspect of the embodiments of the present invention, a computer device is proposed, including: at least one processor; and a memory storing computer instructions that can be run on the processor, and the instructions are executed by the processor to implement the following steps: S1, create a RISC-V extended instruction set and compile it using a compiler; S2, in response to receiving a start instruction in the RISC-V extended instruction set, calculate the address and data length of each read storage unit according to the resolution information, compression mode, and RGB initial address information in the start instruction; S3, generate a load instruction in the RISC-V extended instruction set according to the address and data length, and send the RGB data obtained according to the load instruction to the multiply-accumulate array for parallel RGB-to-YUV conversion; and S4, calculate the UV data to be discarded in the calculation result of the multiply-accumulate array according to the current compression mode, calculate the storage address according to the discarded UV data, and store the YUV data from the cache to the storage unit according to the storage address.

[0152] In some embodiments, generating the load instruction in the RISC-V extension instruction set according to the address and the data length includes: writing the address into a first target register, writing the data length into a second target register, and loading the values of the first target register and the second target register into a conventional load instruction.

[0153] In some embodiments, calculating the UV data to be discarded in the calculation result of the multiply-accumulate array according to the current compression mode includes: in response to the current compression mode being YUV444, retaining all YUV data; in response to the current compression mode being YUV422, retaining the UV components of the even / odd columns of all rows and discarding the remaining UV components; and in response to the current compression mode being YUV420, retaining the UV components of the even columns of the even rows and discarding the remaining UV components.

[0154] In some embodiments, calculating the storage address according to the discarded UV data includes: calculating the memory address of the store instruction and the length of a single storage after discarding the UV data.

[0155] In some embodiments, storing the YUV data from the cache to the storage unit according to the storage address includes: writing the memory address into a third target register, writing the length of a single storage after discarding the UV data into a fourth target register, and storing the YUV data according to the values of the third target register and the fourth target register.

[0156] In some embodiments, the step further includes: reading the YUV data from the multiply-accumulate array and caching the YUV data after discarding the UV data in the order of the Y component, the U component, and the V component.

[0157] In some embodiments, creating the RISC-V extension instruction set and compiling it using a compiler includes: setting whether it is necessary to read a first source register and a second source register in each extension instruction as needed, and setting whether it is necessary to write data to a target register.

[0158] As Figure 4 shown, it is a schematic diagram of the hardware structure of an embodiment of a computer device for video processing in the above substrate management control chip provided by the present invention.

[0159] Taking the device as Figure 4 shown as an example, the device includes a processor 301 and a memory 302.

[0160] The processor 301 and the memory 302 can be connected by a bus or other means, Figure 4 and taking the connection by bus as an example.

[0161] The memory 302 serves as a non-volatile computer-readable storage medium and can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the video processing method in the baseboard management control chip in the embodiments of the present application. The processor 301 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions, and modules stored in the memory 302, thereby implementing the video processing method in the baseboard management control chip.

[0162] The memory 302 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the video processing method in the baseboard management control chip, etc. In addition, the memory 302 may include high-speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 302 may optionally include a memory remotely set relative to the processor 301, and these remote memories can be connected to the local module through a network. Examples of the above networks include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0163] One or more computer instructions 303 corresponding to the video processing method in the baseboard management control chip are stored in the memory 302. When executed by the processor 301, they execute the video processing method in the baseboard management control chip in any of the above method embodiments.

[0164] Any embodiment of the computer device that executes the above video processing method in the baseboard management control chip can achieve the same or similar effects as those corresponding to any of the foregoing method embodiments.

[0165] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, executes the video processing method in the baseboard management control chip.

[0166] As Figure 5 shown, it is a schematic diagram of an embodiment of the computer storage medium for video processing in the baseboard management control chip provided by the present invention. Taking the computer storage medium as shown in Figure 5 as an example, the computer-readable storage medium 401 stores a computer program 402 that, when executed by a processor, executes the above method.

[0167] Finally, it should be noted that those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The program for the method of video processing in the baseboard management control chip can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. Among them, the storage medium of the program can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc. The embodiments of the above computer program can achieve the same or similar effects as any of the foregoing method embodiments corresponding thereto.

[0168] The above are exemplary embodiments disclosed by the present invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments disclosed by the present invention as defined by the claims. The functions, steps, and / or actions of the method claims according to the disclosed embodiments herein need not be performed in any particular order. In addition, although the elements disclosed in the embodiments of the present invention can be described or claimed in individual form, they can also be understood as plural unless explicitly limited to the singular.

[0169] It should be understood that, as used herein, unless the context clearly supports exceptions, the singular form "a" is also intended to include the plural form. It should also be understood that the "and / or" used herein refers to any and all possible combinations of one or more of the associated listed items.

[0170] The serial numbers of the disclosed embodiments of the present invention above are only for description and do not represent the superiority or inferiority of the embodiments.

[0171] Those of ordinary skill in the art can understand that all or part of the steps of the above embodiments can be completed by hardware, or can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk, an optical disk, etc.

[0172] Those of ordinary skill in the art should understand that: the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the embodiments disclosed by the present invention (including the claims) is limited to these examples; under the idea of the embodiments of the present invention, the technical features between the above embodiments or different embodiments can also be combined, and there are many other variations in different aspects of the above embodiments of the present invention, which are not provided in detail for the sake of brevity. Therefore, any omission, modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present invention shall be included in the protection scope of the embodiments of the present invention.

Claims

1. A method for video processing in a baseboard management control chip, characterized in that, It includes the following steps: Create a RISC-V extension instruction set and compile it using a compiler; In response to receiving a start instruction in the RISC-V extension instruction set, run the RGB_YUV_PRC instruction in the RISC-V extension instruction set, and calculate the address and data length of each read storage unit according to the resolution information, compression mode, and RGB initial address information in the start instruction; Write the address to a first target register, write the data length to a second target register, and load the values of the first target register and the second target register into a traditional load instruction; The traditional load instruction loads RGB data according to the values of the first target register and the second target register, and the load instruction in the RISC-V extension instruction set sends the loaded RGB data to a multiply-accumulate array for parallel RGB-to-YUV conversion; The YUV_BLOCK_PRC instruction in the RISC-V extension instruction set calculates the UV data to be discarded in the calculation result of the multiply-accumulate array according to the current compression mode, and calculates the storage address according to the discarded UV data; The STORE_YUV instruction in the RISC-V extension instruction set reads YUV data from the multiply-accumulate array and caches the YUV data after discarding the UV data in the order of Y component, U component, and V component; The traditional store instruction stores the YUV data from the cache to the storage unit according to the storage address.

2. The method according to claim 1, wherein The calculating the UV data to be discarded in the calculation result of the multiply-accumulate array according to the current compression mode includes: In response to the current compression mode being YUV444, retain all YUV data; In response to the current compression mode being YUV422, retain the UV components of the even / odd columns of all rows and discard the remaining UV components; and In response to the current compression mode being YUV420, retain the UV components of the even columns of the even rows and discard the remaining UV components.

3. The method according to claim 1, characterized in that The calculating the storage address according to the discarded UV data includes: Calculate the memory address of the traditional store instruction and the length of a single storage after discarding the UV data; Write the memory address to a third target register and write the length of a single storage after discarding the UV data to a fourth target register.

4. The method according to claim 3, characterized in that, The storing the YUV data from the cache to the storage unit according to the storage address includes: Store the YUV data according to the values of the third target register and the fourth target register.

5. The method according to claim 1, wherein The creating a RISC-V extension instruction set and compiling it using a compiler includes: Set whether it is necessary to read a source register and whether it is necessary to write data to a target register in each extension instruction.

6. A video processing system in a baseboard management control chip, characterized in that, It includes: An instruction module configured to create a RISC-V extension instruction set and compile it using a compiler; A computing module, configured to run the RGB_YUV_PRC instruction in the RISC-V extension instruction set in response to receiving a start instruction in the RISC-V extension instruction set, and calculate the address and data length of each read storage unit according to the resolution information, compression mode, and RGB initial address information in the start instruction; A conversion module, configured to write the address to a first target register, write the data length to a second target register, and load the values of the first target register and the second target register into a traditional load instruction; the traditional load instruction loads RGB data according to the values of the first target register and the second target register, and the load instruction in the RISC-V extension instruction set sends the loaded RGB data to a multiply-accumulate array for parallel RGB-to-YUV conversion; An execution module, configured to call the YUV_BLOCK_PRC instruction in the RISC-V extension instruction set to calculate the UV data to be discarded in the calculation result of the multiply-accumulate array according to the current compression mode, and calculate the storage address according to the discarded UV data; the STORE_YUV instruction in the RISC-V extension instruction set reads YUV data from the multiply-accumulate array, and caches the YUV data after discarding the UV data in the order of Y component, U component, and V component; A traditional store instruction stores the YUV data from the cache to the storage unit according to the storage address.

7. A computer device, characterized in that, Comprising: At least one processor; And A memory, which stores computer instructions that can run on the processor, and when the instructions are executed by the processor, the steps of the method according to any one of claims 1-5 are implemented.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1-5 are implemented.

Images