Video function system for storing application chips, storage application chips, and servers

Abstract

The present invention discloses a video function system for a storage application chip, a storage application chip and a server, which relates to the technical field of chip design. The system is applied to the video data output control module of the storage application chip and includes: a read cache prediction module, which is used to determine the subsequent read cache situation according to the first frequency detected by the system state detection module and the second frequency detected by the off-chip storage interface detection module when the on-chip cache is full; a dynamic frame compensation control module, which is used to predict the predicted video data block according to each target video data block stored in the on-chip cache when the subsequent read cache situation is in the read state. The present invention uses the read cache prediction module to decide whether to perform dynamic frame compensation processing on the video frame being written when the on-chip cache is full; and the dynamic frame compensation control module performs dynamic frame compensation on the video frame that cannot continue to be written due to the on-chip cache being full, reduces the frame loss rate of the video function, and improves the performance of the storage application chip.

Description

Technical Field

[0001] The present invention relates to the technical field of chip design, and particularly relates to a video function system for a storage application chip, a storage application chip, and a server. Background Art

[0002] Currently, in the traditional storage application scenario, the processing flow of the video function of a SoC (System on Chip) chip is to first convert the original RGB (Red, Green, Blue) format data through color space conversion to generate YUV (Luminance, Chrominance, and Alpha) format video data, compress the YUV format data (such as video compression formats like JPEG and AVS) or directly transmit the YUV format data; write the compressed data or YUV format data to an external memory (such as Double Data Rate Synchronous Dynamic Random Access Memory DDR) through the video data output control module of the SoC chip.

[0003] As Figure 1 shown, in a typical storage application chip with integrated video function (such as a Baseboard Management Controller BMC), the processing flow of the video function is: the original video data of the server host is transmitted to the VGA (Video Graphics Array) module inside the storage application chip through PCIe (a high-speed serial computer expansion bus standard), the VGA generates the original video data in RGB format, and then undergoes the above processing. However, the operating system and application programs running on the storage application chip share an external memory (such as DDR memory), an on-chip storage controller (such as the on-chip DDR controller), and a set of system buses. Therefore, there is often competition for the control right of the memory bus, which often causes the video function of the server management controller to be unable to write video data to the external memory in time, resulting in the on-chip cache (used to cache video data) being filled up, causing frames of the current video frame being written to the on-chip cache to be dropped. Even if the on-chip cache becomes unfilled immediately after being full (such as when the video output control module competes for the control right of the memory bus again), at this time, the current video frame being written has already been dropped, resulting in the video data written to the on-chip cache becoming invalid data, making the frame drop rate of the video function very high and there being a large amount of invalid data written.

[0004] Therefore, how to reduce the frame drop rate of the video function of the storage application chip, reduce the processing process of invalid data, and improve the performance of the storage application chip are problems that urgently need to be solved today. Summary of the Invention

[0005] The object of the present invention is to provide a video function system for a storage application chip, a storage application chip, and a server, so as to reduce the frame loss rate of the video function of the storage application chip, reduce the processing process of invalid data, and improve the performance of the storage application chip.

[0006] To solve the above technical problems, the present invention provides a video function system for a storage application chip, which is applied to a video data output control module of the storage application chip and includes: an on-chip cache, a system state detection module, an off-chip storage interface detection module, a read cache prediction module, and a dynamic frame compensation control module;

[0007] The read cache prediction module is configured to, when the on-chip cache is full, determine the subsequent read cache situation according to the first frequency detected by the system state detection module and the second frequency detected by the off-chip storage interface detection module; wherein, the first frequency is the frequency of read and write operation commands initiated by an interface of a central processing unit in the storage application chip connected to the system bus, and the second frequency is the read and write frequency of all interfaces on an on-chip storage controller in the storage application chip, and the subsequent read cache situation includes a read state and a non-read state;

[0008] The dynamic frame compensation control module is configured to, when the subsequent read cache situation is the read state, predict a predicted video data block according to each target video data block stored in the on-chip cache, and write the predicted video data block into the on-chip cache when the on-chip cache is not full; wherein, the target video data block is a video data block adjacent to the predicted video data block in the current video frame data.

[0009] On the other hand, the read cache prediction module is specifically configured to, when the on-chip cache is full, determine the subsequent read cache situation according to the first frequency and the second frequency corresponding to each first preset time period within a first detection time period; wherein, the number of the first preset time periods is greater than or equal to 2, and the sum of all the first preset time periods is the first detection time period.

[0010] On the other hand, determining the subsequent read cache situation according to the first frequency and the second frequency corresponding to each first preset time period within the first detection time period when the on-chip cache is full includes:

[0011] When the on-chip cache is full, if PARAM_0 * (P_0 * Freq_sys0 + P_1 * Freq_sys1 + P_2 * Freq_sys2 + …… P_N-1 * Freq_sysN-1) + PARAM_1 * (Q_0 * Freq_DDR0 + Q_1 * Freq_DDR1 + Q_2 * Freq_DDR2 + …… Q_N-1 * Freq_DDRN-1) < THREHOLD_0, it is determined that the subsequent read cache situation is the read state; if PARAM_0 * (P_0 * Freq_sys0 + P_1 * Freq_sys1 + P_2 * Freq_sys2 + …… P_N-1 * Freq_sysN-1) + PARAM_1 * (Q_0 * Freq_DDR0 + Q_1 * Freq_DDR1 + Q_2 * Freq_DDR2 + …… Q_N-1 * Freq_DDRN-1) ≥ THREHOLD_0, it is determined that the subsequent read cache situation is the non-read state to discard the current video frame data; where N is the number of the first preset time periods, PARAM_0 and P_0 to P_N-1 are the preset parameters corresponding to the first frequency respectively, PARAM_1 and Q_0 to Q_N-1 are the preset parameters corresponding to the second frequency respectively, Freq_sys0 to Freq_sysN-1 are the first frequencies corresponding to the 1st to Nth first preset time periods respectively, Freq_DDR0 to Freq_DDRN-1 are the second frequencies corresponding to the 1st to Nth first preset time periods respectively, and THREHOLD_0 is the first preset threshold; P_0 > P_1 > P_2 …… > P_N-1, Q_0 > Q_1 > Q_2 …… > Q_N-1.

[0012] On the other hand, PARAM_1 > PARAM_0, Q_i > P_i, where i is an integer greater than or equal to 0 and less than N.

[0013] On the other hand, the read cache prediction module is further configured to, after determining that the subsequent read cache situation is the non-read state, detect the data read-out situation of the on-chip cache, and determine whether the determination of the subsequent read cache situation is correct according to the data read-out situation; if not, then reduce PARAM_1, PARAM_0, P_0 to P_N-1 and Q_0 to Q_N-1.

[0014] On the other hand, during the process of reducing PARAM_1, PARAM_0, P_0 to P_N-1 and Q_0 to Q_N-1, the reduction amplitude of the preset parameter corresponding to the first frequency is less than the reduction amplitude of the preset parameter corresponding to the second frequency.

[0015] On the other hand, the subsequent read cache situation further includes an invalid frame state. Determining the subsequent read cache situation according to the first frequencies and second frequencies respectively corresponding to the first preset time periods within the first detection time period includes:

[0016] Determining the subsequent read cache situation according to the first frequencies and second frequencies respectively corresponding to the first preset time periods within the first detection time period and the first frequencies and second frequencies respectively corresponding to the second preset time periods within the second detection time period;

[0017] The video function system further includes:

[0018] A pre-drop frame control module, configured to control a pre-stage hardware module corresponding to a video data output control module in the storage application chip to stop processing current video frame data when the subsequent read cache situation is in the invalid frame state.

[0019] On the other hand, determining the subsequent read cache situation according to the first frequencies and second frequencies respectively corresponding to the first preset time periods within the first detection time period and the first frequencies and second frequencies respectively corresponding to the second preset time periods within the second detection time period includes:

[0020] If PARAM_2 * (R_0 * Freq_sys0 + R_1 * Freq_sys1 + R_2 * Freq_sys2 +... + R_M-1 * Freq_sysM-1) + PARAM_3 * (S_0 * Freq_DDR0 + S_1 * Freq_DDR1 + S_2 * Freq_DDR2 +... + S_M-1 * Freq_DDRM-1) > THREHOLD_1, then determine that the subsequent read cache situation is in the invalid frame state; where M is the number of the second preset time periods, PARAM_2 and R_0 to R_M-1 are preset parameters corresponding to the first frequencies respectively, PARAM_3 and S_0 to S_M-1 are preset parameters corresponding to the second frequencies respectively, Freq_sys0 to Freq_sysM-1 are the first frequencies respectively corresponding to the 1st first preset time period to the Mth first preset time period, Freq_DDR0 to Freq_DDRM-1 are the second frequencies respectively corresponding to the 1st first preset time period to the Mth first preset time period, and THREHOLD_1 is a second preset threshold; R_0 > R_1 > R_2... > R_M-1, S_0 > S_1 > S_2... > S_M-1.

[0021] On the other hand, PARAM_3 > PARAM_2, S_j > R_j, where j is an integer greater than or equal to 0 and less than N.

[0022] On the other hand, the read cache prediction module is further configured to detect the data read-out situation of the on-chip cache after determining that the subsequent read cache situation is in the invalid frame state, and determine whether the determination of the subsequent read cache situation is correct according to the data read-out situation; if it is incorrect, then decrease PARAM_2, PARAM_3, R_0 to R_M-1, and S_0 to S_M-1.

[0023] On the other hand, predicting the predicted video data block according to each target video data block stored in the on-chip cache includes:

[0024] Determining each of the target video data blocks stored in the on-chip cache according to the image change situation of the current video frame data;

[0025] Predicting the predicted video data block according to each of the target video data blocks.

[0026] On the other hand, determining each of the target video data blocks stored in the on-chip cache according to the image change situation of the current video frame data includes:

[0027] If the image change situation of the current video frame data is in a state where the change amount is not greater than the threshold, then use the video data blocks above and to the left of the predicted video data block in the current video frame data in the on-chip cache as the target video data blocks respectively;

[0028] If the image change situation of the current video frame data is in a state where the change amount is greater than the threshold, then use the video data blocks above, to the left, to the upper left, and to the upper right of the predicted video data block in the current video frame data in the on-chip cache as the target video data blocks respectively.

[0029] On the other hand, predicting the predicted video data block according to each of the target video data blocks includes:

[0030] When the number of the target video data blocks is 2, calculate the predicted video data block through BLOCK_PREDICT = A_0 * BLOCK_0 + A_1 * BLOCK_2; where BLOCK_PREDICT is the predicted video data block, BLOCK_0 and BLOCK_2 are the 2 target video data blocks, A_0 and A_1 are preset frame interpolation parameters, and A_0 + A_1 = 1;

[0031] When the number of the target video data blocks is 4, the predicted video data block is calculated through BLOCK_PREDICT = B_0 * BLOCK_0 + B_1 * BLOCK_1 + B_2 * BLOCK_2 + B_3 * BLOCK_3; where BLOCK_0 to BLOCK_3 are the 4 target video data blocks, B_0 to B_3 are preset frame interpolation parameters, and B_0 + B_1 + B_2 + B_3 = 1.

[0032] On the other hand, the dynamic frame interpolation control module is specifically configured to, when the subsequent read cache condition is the read state, predict a predicted video data block according to each of the target video data blocks and the preset frame interpolation parameters corresponding to each of the target video data blocks;

[0033] The video function system further includes:

[0034] A frame interpolation parameter adjustment and correction module, configured to, when the on-chip cache is not full, predict a predicted video data block corresponding to the currently to-be-written video data block stored in the on-chip cache according to each target video data block corresponding to the currently to-be-written video data block stored in the on-chip cache and the preset frame interpolation parameters at a preset time interval; and adjust the preset frame interpolation parameters according to the comparison between the currently to-be-written video data block and the corresponding predicted video data block.

[0035] On the other hand, the system further includes: a compressed block information cache control module and / or an uncompressed block information cache control module;

[0036] The compressed block information cache control module is configured to, when the current video frame data received by the video data output control module is data in a preset compressed format, transfer each target video data block stored in the on-chip cache to the dynamic frame interpolation control module;

[0037] The uncompressed block information cache control module is configured to, when the current video frame data received by the video data output control module is data in a preset video data format, transfer each target video data block stored in the on-chip cache to the dynamic frame interpolation control module;

[0038] Wherein, the preset frame interpolation parameters corresponding to the target video data blocks in the preset compressed format are different from the preset frame interpolation parameters corresponding to the target video data blocks in the preset video data format.

[0039] On the other hand, when the video data received by the video data output control module includes data in a preset compressed format, the video function system further includes: a frame information parsing module, a frame common information control module, a frame basic information control module, a compression algorithm table control module, a luminance information quantization table control module, a chrominance information quantization table control module, an on-chip cache write control module, and an on-chip cache read control module;

[0040] The frame information parsing module is configured to parse the received current video frame data in the preset compression format to obtain the frame common information, frame basic information, compression algorithm table information, luminance information quantization table information, chrominance information quantization table information, and compressed data blocks of the current video frame data; wherein, the compressed data blocks are the compressed data in the current video frame data in units of blocks, and the preset compression format is an intra-frame compression format.

[0041] The frame common information control module is configured to update and store the frame common information of the current video frame data.

[0042] The frame basic information control module is configured to update and store the frame basic information of the current video frame data.

[0043] The compression algorithm table control module is configured to update and store the compression algorithm table information of the current video frame data.

[0044] The luminance information quantization table control module is configured to update and store the luminance information quantization table information of the current video frame data.

[0045] The chrominance information quantization table control module is configured to update and store the luminance information quantization table information of the current video frame data.

[0046] The on-chip cache write control module is configured to sequentially store the compressed data blocks of the current video frame data into the on-chip cache in units of blocks.

[0047] The on-chip cache read control module is configured to restore the compressed data blocks read from the on-chip cache to the data in the preset compression format according to the data stored by the frame common information control module, the frame basic information control module, the compression algorithm table control module, the luminance information quantization table control module, and the chrominance information quantization table control module.

[0048] In another aspect, the preset compression format is a Joint Photographic Experts Group (JPEG) compression format, and the compression algorithm table control module is specifically a Huffman table control module, which is configured to store the Huffman table information of the current video frame data.

[0049] In another aspect, the storage application chip is specifically a Baseboard Management Controller (BMC).

[0050] The present invention also provides a storage application chip, including: the video function system of the storage application chip as described above.

[0051] In addition, the present invention also provides a server, including: the storage application chip as described above.

[0052] A video function system for a storage application chip provided by the present invention is applied to a video data output control module of the storage application chip, and includes: an on-chip cache, a system state detection module, an off-chip storage interface detection module, a read cache prediction module, and a dynamic frame compensation control module; the read cache prediction module is used to determine the subsequent read cache situation according to the first frequency detected by the system state detection module and the second frequency detected by the off-chip storage interface detection module when the on-chip cache is full; wherein, the first frequency is the frequency of read and write operation commands initiated by the interface where the central processing unit in the storage application chip is connected to the system bus, and the second frequency is the read and write frequency of all interfaces on the on-chip storage controller in the storage application chip, and the subsequent read cache situation includes a read state and a non-read state; the dynamic frame compensation control module is used to predict a predicted video data block according to each target video data block stored in the on-chip cache when the subsequent read cache situation is in the read state, so as to write the predicted video data block into the on-chip cache when the on-chip cache is not full; wherein, the target video data block is a video data block adjacent to the predicted video data block in the current video frame data.

[0053] It can be seen that by adding a read cache prediction module to the video data output control module, the present invention can, when the on-chip cache is full, determine whether to perform dynamic frame compensation processing on the video frame being written according to the detection results output by the system state detection module and the off-chip storage interface detection module; thus, when it is determined to perform dynamic frame compensation processing, the dynamic frame compensation control module performs dynamic frame compensation on the current video frame that cannot be written continuously due to the on-chip cache being full, improves the complete write rate of video frame data, reduces the frame loss rate of the video function of the storage application chip, reduces the processing process of invalid data, improves the utilization rate of the system bus resources and storage bandwidth of the video function, and improves the performance of the storage application chip. In addition, the present invention also provides a storage application chip and a server, which also have the above beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0055] Figure 1 It is a schematic diagram of the remote video transmission solution of the BMC in the server in the related art;

[0056] Figure 2 It is a structural block diagram of a video function system for a storage application chip provided by an embodiment of the present invention;

[0057] Figure 3 Schematic diagram of a method for dividing a first preset time period provided by an embodiment of the present invention;

[0058] Figure 4 Schematic diagram of the structure of a video function system of a storage application chip provided by an embodiment of the present invention;

[0059] Figure 5 Schematic diagram of a method for dividing a second preset time period provided by an embodiment of the present invention;

[0060] Figure 6 Schematic diagram of the frame compensation function of the current video frame data in a preset compression format provided by an embodiment of the present invention;

[0061] Figure 7 Schematic diagram of the frame compensation function of the current video frame data in a preset video data format provided by an embodiment of the present invention. Detailed implementation manners

[0062] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0063] Please refer to Figure 2 , Figure 2 Block diagram of the structure of a video function system of a storage application chip provided by an embodiment of the present invention. This system is applied to the video data output control module of the storage application chip and may include: a system status detection module 10, an off-chip storage interface detection module 20, a read cache prediction module 30, a dynamic frame compensation control module 40, and an on-chip cache 50;

[0064] The read cache prediction module 30 is configured to determine the subsequent read cache situation according to the first frequency detected by the system status detection module 10 and the second frequency detected by the off-chip storage interface detection module 20 when the on-chip cache 50 is full; wherein, the first frequency is the frequency of the read and write operation commands initiated by the interface of the central processing unit in the storage application chip connected to the system bus, the second frequency is the read and write frequency of all interfaces on the on-chip storage controller in the storage application chip, and the subsequent read cache situation includes a read state and a non-read state;

[0065] The dynamic frame compensation control module 40 is configured to, when the subsequent read cache condition is in the read state, predict predicted video data blocks based on each target video data block stored in the on-chip cache 50, and write the predicted video data blocks into the on-chip cache 50 when the on-chip cache 50 is not full; wherein, the target video data block is a video data block adjacent to the predicted video data block in the current video frame data.

[0066] It can be understood that the storage application chip in this embodiment can be an SoC chip in a storage application scenario, such as a server management controller (such as BMC) in a server of a storage system, or an SSD (Solid State Drive) controller. The SoC chip is a single chip integrated with one or more of a microprocessor, an analog IP (Intellectual Property) core, a digital IP core, and a memory (or an off-chip storage control interface). The video data output control module of the storage application chip in this embodiment can be a hardware module in the storage application chip for writing received video data (such as compressed data or YUV format data) into the on-chip cache 50, reading out the video data in the on-chip cache 50, and converting it into data of a corresponding interface protocol for output to a hardware module at the subsequent stage of the storage application chip or an external device (such as a memory DDR). For example, when the storage application chip is BMC, as Figure 1 shown, the video data output control module in BMC can write the input video data (such as compressed data or YUV format data) into an external video memory (such as DDR) via the on-chip cache 50.

[0067] It can be understood that in the related art, the on-chip cache write control module in the video data output control module of the BMC is responsible for receiving the video data input from the previous stage (such as video data in the Joint Photographic Experts Group JPEG compression format or YUV format), and then writing it into the on-chip cache 50. To ensure the correctness of the data written into the on-chip cache 50, this module needs to ensure that writing stops when the on-chip cache 50 is full. Therefore, for a frame of video data (i.e., video frame data), there are two writing states. One is that all the video data of a frame is written into the on-chip cache 50, denoted as the state frame_wr_state = 0. The other is that the video data of a frame cannot be completely written into the on-chip cache 50 because the on-chip cache 50 is full, denoted as the state frame_wr_state = 1. The on-chip cache read control module in the video data output control module can be responsible for reading out the video data in the on-chip cache 50 and converting the read video data into the AXI (Advanced eXtensible Interface, a bus protocol) protocol format through the interface protocol conversion module and writing it into the DDR. Because the writing state of the video frame data (frame_wr_state = 0 / 1) is obtained only at the last data of the current video frame data, and the data before this time has been written into the DDR. If frame_wr_state = 1, then these written data are invalid data because the software of the remote device cannot restore and display these data. At the same time, a large amount of DDR memory bandwidth is wasted and the content capacity is invalidly occupied. At the same time, there is often a situation where after the on-chip cache 50 is full, the video output control module competes for the memory bus control right again and can continue to write video data. At this time, the current video frame data being written has been dropped, that is, the subsequent video data of the current video frame has stopped being written into the on-chip cache 50, and at the same time, the video data of the current video frame that has been written into the on-chip cache 50 becomes invalid data. Because although it is written into the on-chip cache 50, these data are not a complete frame of data, and the remote device cannot restore them. But at the same time, these data still need to be read out from the on-chip cache 50 by the internal logic of the video data output control module and written into the external DDR, resulting in a very high frame drop rate in the video function of the storage application chip in the related art and a large amount of invalid data writing.

[0068] Correspondingly, in this embodiment, by adding the read cache prediction module 30 in the video data output control module, it can be based on the first frequency detected by the system state detection module 10 and the off-chip storage interface detection module 20 (such as Figure 4The second frequency detected by the DDR interface detection module (in []) is used to predict the subsequent state (i.e., the subsequent read cache situation) at the moment when the on-chip cache 50 (such as a storage type like a receive first-in-first-out queue FIFO or a random access memory RAM) is full, so as to determine whether to perform dynamic frame compensation processing on the video frame data being written. Among them, the first frequency detected by the system state detection module 10 can be the frequency (Freq_sys) of the read and write operation commands initiated by the interface of the CPU in the storage application chip connected to the system bus. This frequency can represent all the programs allowed on the storage application chip and the state reflected by the system. The higher this frequency, the more frequently the operating system and software programs on the chip process data, and the greater the demand for the bus and external memory. The second frequency detected by the off-chip storage interface detection module 20 can be the read and write frequency (Freq_DDR) of all the interfaces on the on-chip storage controller in the storage application chip. That is, since many modules in the storage application chip need to access external memory (such as DDR), there are multiple interfaces on the corresponding storage controller (i.e., the on-chip storage controller, such as the DDR controller) in the storage application chip. The off-chip storage interface detection module 20 can determine the usage rights given to each hardware module mounted on this storage controller during bus arbitration by detecting the read and write frequencies of these interfaces. For example, the higher the second frequency, the more frequently each hardware module accesses the external memory, and the smaller the usage rights of each hardware module.

[0069] Correspondingly, for the specific method of the read cache prediction module 30 to determine the subsequent read cache situation based on the first frequency detected by the system state detection module 10 and the second frequency detected by the off-chip storage interface detection module 20 when the on-chip cache 50 is full, it can be set by the designer according to the practical scenario and user requirements. For example, in order to improve the prediction accuracy of the subsequent read cache situation, as Figure 3 shown, the read cache prediction module 30 can divide the first detection time period before the current moment (i.e., when the on-chip cache 50 is full) into N first preset time periods, so that the read cache prediction module 30 can determine the subsequent read cache situation according to the first frequency and the second frequency corresponding to each first preset time period within the first detection time period when the on-chip cache 50 is full. Among them, the number of first preset time periods is greater than or equal to 2, that is, N is greater than or equal to 2; the sum of all the first preset time periods is the first detection time period. Correspondingly, the read cache prediction module 30 can also directly determine the subsequent read cache situation according to the first frequency and the second frequency within the first detection time period when the on-chip cache 50 is full, such as determining the subsequent read cache situation by calculating the sum of the products of the first frequency and the second frequency within the first detection time period and their corresponding weight parameters and comparing it with a threshold.

[0070] For example, when the on-chip cache 50 is full, the read cache prediction module 30 can determine the subsequent read cache situation according to the first frequency and the second frequency corresponding to each first preset time period within the first detection time period and their respective corresponding preset parameters and the first preset threshold; for example, when the on-chip cache 50 is full, the read cache prediction module 30 can determine the subsequent read cache situation by comparing the sum of the products of the first frequency and the second frequency corresponding to each first preset time period within the first detection time period and their respective corresponding preset parameters with the first preset threshold. For example, when the on-chip cache 50 is full, if PARAM_0*(P_0*Freq_sys0 + P_1*Freq_sys1 + P_2*Freq_sys2 + …… P_N-1*Freq_sysN-1) + PARAM_1*(Q_0*Freq_DDR0 + Q_1*Freq_DDR1 + Q_2*Freq_DDR2 + …… Q_N-1*Freq_DDRN-1) < THREHOLD_0, it is determined that the subsequent read cache situation is the read state (such as predict_result = 0); if PARAM_0*(P_0*Freq_sys0 + P_1*Freq_sys1 + P_2*Freq_sys2 + …… P_N-1*Freq_sysN-1) + PARAM_1*(Q_0*Freq_DDR0 + Q_1*Freq_DDR1 + Q_2*Freq_DDR2 + …… Q_N-1*Freq_DDRN-1) ≥ THREHOLD_0, it is determined that the subsequent read cache situation is the non-read state (such as predict_result = 1) to discard the current video frame data; where N is the number of the first preset time periods, PARAM_0 and P_0 to P_N-1 are the preset parameters corresponding to the first frequency respectively, PARAM_1 and Q_0 to Q_N-1 are the preset parameters corresponding to the second frequency respectively, Freq_sys0 to Freq_sysN-1 are the first frequencies corresponding to the 1st to the Nth first preset time periods respectively, Freq_DDR0 to Freq_DDRN-1 are the second frequencies corresponding to the 1st to the Nth first preset time periods respectively, THREHOLD_0 is the first preset threshold; PARAM_1 > PARAM_0, P_0 > P_1 > P_2 …… > P_N-1, Q_0 > Q_1 > Q_2 …… > Q_N-1.

[0071] That is to say, the first detection time period (T1) before the moment when the on-chip cache 50 is filled (i.e., the current moment) is divided into N segments, and N first preset time periods can be obtained. The closer a first preset time period is to the current moment, the greater the calculation proportion of the corresponding frequency (the first frequency or the second frequency), and the farther it is from the current moment, the lower the calculation proportion of the corresponding frequency (the first frequency or the second frequency). Because the bus and memory states closer to the current moment (i.e., the moment when the on-chip cache 50 becomes full) are more similar to the states after the current moment, while the farther away from the current moment, the greater the difference between the bus and memory states and the states after the current moment.

[0072] As Figure 3 shown, the first detection time period before the current moment can be evenly divided into N segments, that is, the times of N first preset time periods can be the same; among them, the values of the first detection time period and N can be configured by the user through registers or can also adopt default values. The preset parameter corresponding to the 1st first frequency (Freq_sys0) can be P_0, the preset parameter corresponding to the 2nd first frequency (Freq_sys1) can be P_1... the preset parameter corresponding to the Nth first frequency (Freq_sysN-1) can be P_N-1, and their magnitude relationship is P_0 > P_1 > P_2... > P_N-1; the preset parameter corresponding to the 1st second frequency (Freq_DDR0) is Q_0, the preset parameter corresponding to the 2nd second frequency (Freq_DDR1) is Q_1... the parameter corresponding to the Nth second frequency (Freq_DDRN-1) is Q_N-1, and their magnitude relationship is Q_0 > Q_1 > Q_2... > Q_N-1; at the same time, Q_0 > P_0, Q_1 > P_1... Q_N-1 > P_N-1, that is, Q_i > P_i, where i is an integer greater than or equal to 0 and less than N. Because the influence of the second frequency is greater, PARAM_1 > PARAM_0, that is, the preset parameter corresponding to the first frequency can be less than the preset parameter corresponding to the second frequency.

[0073] Correspondingly, THREHOLD_0 is the first preset threshold (i.e., the threshold for judgment), which can be configured by the user through the register or can adopt the default value. Moreover, in this embodiment, the prediction mechanism of the read cache prediction module 30 can be dynamically adjusted. When the predicted subsequent read cache situation is the non-read state, the current video frame data can be discarded as in the related art. Further, the read cache prediction module 30 can also be used to detect the data read-out situation of the on-chip cache 50 after determining that the subsequent read cache situation is the non-read state, and determine whether the determination of the subsequent read cache situation is correct according to the data read-out situation. If it is incorrect, then reduce PARAM_1, PARAM_0, P_0 to P_N-1 and Q_0 to Q_N-1. If it is correct, then do not adjust PARAM_1, PARAM_0, P_0 to P_N-1 and Q_0 to Q_N-1. For example, when the predicted subsequent read cache situation is the non-read state, the read cache prediction module 30 can synchronously detect the data read-out situation of the on-chip cache 50 after the current moment, such as Figure 4 the behavior of the on-chip cache read control module reading the data in the on-chip cache 50 in Figure 4 , and simulate the write-side behavior, such as simulating continuing to parse the current video frame data and judging whether the current video frame data can be completely written into the on-chip cache 50 at the subsequent time of the current moment. If it can be completely written into the on-chip cache 50, it proves that the determination of the subsequent read cache situation is incorrect. At this time, the preset parameters corresponding to the first frequency and the second frequency in the above formula can be lowered, that is, the parameters of PARAM_0, PARAM_1, P_0...P_N-1, Q_0...Q_N-1 are lowered to improve the prediction accuracy. Regarding the lowering amplitude, the lowering amplitude of the preset parameters corresponding to the second frequency (Freq_DDR) (such as PARAM_1 and Q_0...Q_N-1) can be greater than the lowering amplitude of the preset parameters corresponding to the first frequency (Freq_sys) (such as PARAM_0 and P_0...P_N-1). That is to say, in the process of reducing PARAM_1, PARAM_0, P_0 to P_N-1 and Q_0 to Q_N-1, the reduction amplitude of the preset parameters corresponding to the first frequency is less than the reduction amplitude of the preset parameters corresponding to the second frequency.

[0074] Further, in this embodiment, the subsequent read cache situation determined by the read cache prediction module 30 may further include an invalid frame state (such as predict_result = 2) to indicate whether the new frame of video data to be input into the on-chip cache 50 can be completely input; for example, the read cache prediction module 30 may be used to determine the subsequent read cache situation according to the first frequency and the second frequency corresponding to each first preset time period within the first detection time period and the first frequency and the second frequency corresponding to each second preset time period within the second detection time period; the system provided in this embodiment may further include a pre-drop frame control module, which is used to control the previous-stage hardware module corresponding to the video data output control module in the storage application chip (such as Figure 1 the data capture module, color space conversion module, and core compression module in

[0075] to stop processing the current video frame data. For example, when the on-chip cache 50 is full, the read cache prediction module 30 may determine the subsequent read cache situation by comparing the sum of the products of the first frequency and the second frequency corresponding to each second preset time period within the second detection time period and their respective preset parameters with the second preset threshold, and by comparing the sum of the products of the first frequency and the second frequency corresponding to each first preset time period within the first detection time period and their respective preset parameters with the first preset threshold. For example, when the on-chip cache 50 is full, if PARAM_2 * (R_0 * Freq_sys0 + R_1 * Freq_sys1 + R_2 * Freq_sys2 + …… R_M-1 * Freq_sysM-1) + PARAM_3 * (S_0 * Freq_DDR0 + S_1 * Freq_DDR1 + S_2 * Freq_DDR2 + …… S_M-1 * Freq_DDRM-1) > THREHOLD_1, it is determined that the subsequent read cache situation is an invalid frame state; where M is the number of second preset time periods, PARAM_2 and R_0 to R_M-1 are the preset parameters corresponding to the first frequency, PARAM_3 and S_0 to S_M-1 are the preset parameters corresponding to the second frequency, Freq_sys0 to Freq_sysM-1 are the first frequencies corresponding to the 1st to Mth first preset time periods respectively, Freq_DDR0 to Freq_DDRM-1 are the second frequencies corresponding to the 1st to Mth first preset time periods respectively, and THREHOLD_1 is the second preset threshold; PARAM_3 > PARAM_2, R_0 > R_1 > R_2 …… > R_M-1, S_0 > S_1 > S_2 …… > S_M-1.

[0076] That is to say, the second detection time period (T2) before the moment when the on-chip cache 50 is filled (i.e., the current moment) is divided into M segments, and M second preset time periods can be obtained; the second detection time period is used to predict whether a new frame of video data to be input into the on-chip cache 50 can be completely input into the on-chip cache 50, and the second detection time period can be greater than the first detection time period. The closer a second preset time period is to the current moment, the greater the calculation proportion of the corresponding frequency (the first frequency or the second frequency), and the farther it is from the current moment, the lower the calculation proportion of the corresponding frequency (the first frequency or the second frequency), because the bus and memory states closer to the current moment (i.e., the moment when the on-chip cache 50 becomes full) are more similar to the states after the current moment, while the farther away from the current moment, the greater the difference between the bus and memory states and the states after the current moment.

[0077] As Figure 5 shown, the second detection time period before the current moment can be evenly divided into M segments, that is, the time of M second preset time periods can be the same; among them, the values of the second detection time period and M can be configured by the user through the register, or the default values can be adopted. For example, the second detection time period (or the second preset time period) can adopt the average value of the time used for multiple input complete frames of images (i.e., video frame data). The preset parameter corresponding to the first frequency (Freq_sys0) in the first second detection time period can be R_0, the preset parameter corresponding to the second first frequency (Freq_sys1) can be R_1... the preset parameter corresponding to the Mth first frequency (Freq_sysM-1) can be R_M-1, and their magnitude relationship is R_0>R_1>R_2...>R_M-1; the preset parameter corresponding to the first second frequency (Freq_DDR0) in the second detection time period is S_0, the preset parameter corresponding to the second second frequency (Freq_DDR1) is S_1... the parameter corresponding to the Mth second frequency (Freq_DDRM-1) is S_M-1, and their magnitude relationship is S_0>S_1>S_2...>S_M-1; at the same time, S_0>R_0, S_1>R_1... S_M-1>R_M-1, that is, S_j>R_j, where j is an integer greater than or equal to 0 and less than M, because the influence of the second frequency is greater, PARAM_3>PARAM_2, that is, the preset parameter corresponding to the first frequency can be less than the preset parameter corresponding to the second frequency.

[0078] Correspondingly, THREHOLD_1 is the second preset threshold (i.e., the threshold for judgment), which can be configured by the user through the register or can adopt the default value. For example, when the on-chip cache 50 is full, the read cache prediction module 30 can compare the sum of the products of the first frequency and the second frequency corresponding to each second preset time period within the second detection time period and their respective preset parameters with the second preset threshold. When the subsequent read cache situation is not in the invalid frame state, then through the comparison of the sum of the products of the first frequency and the second frequency corresponding to each first preset time period within the first detection time period and their respective preset parameters with the first preset threshold, determine whether the subsequent read cache situation is in the read state or the non-read state.

[0079] Meanwhile, in this embodiment, the prediction mechanism of the read cache prediction module 30 is dynamically adjusted. When the predicted subsequent read cache situation is in the invalid frame state, the current video frame data can be discarded, that is, the function of pre-discarding frames is realized, and the current video frame data to be written is determined as an invalid frame. Further, the read cache prediction module 30 (or the pre-discarding frame control module) can also be used to detect the data read situation of the on-chip cache 50 after determining that the subsequent read cache situation is in the invalid frame state, and determine whether the determination of the subsequent read cache situation is correct according to the data read situation; if not, then reduce PARAM_2, PARAM_3, R_0 to R_M-1, and S_0 to S_M-1; if correct, then do not adjust PARAM_2, PARAM_3, R_0 to R_M-1, and S_0 to S_M-1. For example, when the predicted subsequent read cache situation is in the invalid frame state, the read cache prediction module 30 can synchronously detect the data read situation of the on-chip cache 50 after the current moment, such as Figure 4 the behavior of the on-chip cache read control module reading the data in the on-chip cache 50, and simulate the writing side behavior, such as simulating continuing to write the current frame, and judge whether the current video frame data can be completely written into the on-chip cache 50 at the subsequent time of the current moment; if it can be completely written into the on-chip cache 50, it proves that the determination of the subsequent read cache situation is incorrect. At this time, the preset parameters corresponding to the first frequency and the second frequency in the above formula can be lowered, that is, the parameters of PARAM_2, PARAM_3, R_0...R_M-1, S_0...S_M-1 are lowered to improve the prediction accuracy; regarding the lowering amplitude, the lowering amplitude of the preset parameters corresponding to the second frequency (Freq_DDR) (such as PARAM_3 and S_0...S_M-1) can be greater than the lowering amplitude of the preset parameters corresponding to the first frequency (Freq_sys) (such as PARAM_2 and R_0...R_M-1); that is to say, in the process of reducing PARAM_2, PARAM_3, R_0 to R_M-1, and S_0 to S_M-1, the reduction amplitude of the preset parameters corresponding to the first frequency is less than the reduction amplitude of the preset parameters corresponding to the second frequency.

[0080] Correspondingly, for the current video frame data determined to be an invalid frame, the capture of the invalid frame and subsequent processing such as color space conversion and video compression can be stopped starting from the data capture module in Figure 1 to avoid the invalid processing of video data to the greatest extent; that is, after determining that it is an invalid frame, the current video frame data is not written into the on-chip cache 50 or the external DDR. Therefore, the processing of the previous stage for the invalid frame is meaningless, thereby reducing the power consumption of the chip, increasing the service life of the chip, and reducing the occupation of the memory bandwidth and system bus by invalid data to the greatest extent.

[0081] In some embodiments, as Figure 4 shown, the system provided in this embodiment may further include a frame information parsing module, a frame common information control module, a frame basic information control module, a compression algorithm table control module, a luminance information quantization table control module, a chrominance information quantization table control module, an on-chip cache write control module, and an on-chip cache read control module; the frame information parsing module is configured to parse the received current video frame data in a preset compression format to obtain the frame common information, frame basic information, compression algorithm table information, luminance information quantization table information, chrominance information quantization table information, and compressed data blocks of the current video frame data; wherein, the compressed data blocks are the compressed data in the current video frame data in units of blocks, and the preset compression format is an intra-frame compression format; the frame common information control module is configured to update and store the frame common information of the current video frame data; the frame basic information control module is configured to update and store the frame basic information of the current video frame data; the compression algorithm table control module (such as Figure 4 the Huffman table control module in

[0082] That is to say, for video data in an intra-frame compression format such as the JPEG compression format, parsing the structure of a single-frame image means parsing the structure of consecutive frames that make up a video. Therefore, the frame information parsing module can parse the current video frame data in a preset compression format and parse out six parts: frame common information, frame basic information, compression algorithm table information (such as Huffman table information in the JPEG compression format), luminance information quantization table information, chrominance information quantization information table, and compressed data blocks.

[0083] Among them, the current video frame data (i.e., compressed image) in an intra-frame compression format such as the JPEG compression format can sequentially include SOI (file header), APP0 (image recognition information), DQT (define quantization table), SOF0 (image basic information), DHT (define compression algorithm table, such as Huffman table), DRI (define restart interval), SOS (start of scan line), and EOI (file tail).

[0084] Correspondingly, the frame common information parsed by the frame information parsing module can include SOI, APP0, DRI, and EOI. The parsing method is based on the specific indication information fields corresponding to the above parts. For example, the indication information field of SOI is FF D8, the indication information field of APP0 is FF E0, the indication information field of DRI is FF DD, and the indication information field of EOI is FF D9. This frame common information is common to the preset compression format (such as the JPEG compression format), and each compressed image in the preset compression format is the same. After the above common information is parsed, it can be input to Figure 4 the frame common information control module in it for storage and updated when it changes.

[0085] The information of the compression algorithm table (such as Huffman table) can also be parsed through the corresponding specific indication information. For example, when the preset compression format is the JPEG compression format, the indication information field of the Huffman table information is FF C4. Since when Figure 1 the core compression module performs video data compression in the JPEG format, different Huffman tables may be used. Therefore, the Huffman table information can be parsed separately, and the parsed information is input to Figure 4 the Huffman table control module in it.

[0086] Correspondingly, the frame basic information can include SOF0. The indication information field of SOF0 is FF C0, and it can include information such as resolution information, quantization table number, sample precision, and YUV format. These information may be different in different video frame data, such as when the resolution changes and the YUV format changes (such as common YUV formats such as YUV444 / YUV422 / YUV420). The parsed information is input to the frame basic information control module in 4.

[0087] Correspondingly, the luminance information quantization table information may include SOF0. The SOF0 indication information field is FF DB, which is the quantization table used for luminance component quantization during JPEG compression. Since Figure 1 when the core compression module in Figure 4 performs JPEG format compression, different luminance quantization tables may be used. Therefore, the luminance quantization table information needs to be parsed separately, and the parsed information is input into the

[0088] The chrominance information quantization table may include DQT. DQT can be divided into two parts. First is the luminance information quantization table information, and then is the chrominance information quantization table information. The indication information field of this part of information is also FF DB, which is the quantization table used for chrominance component quantization during JPEG compression. Since Figure 1 when the core compression module in Figure 4 performs JPEG format compression, different chrominance quantization tables may be used. Therefore, the chrominance quantization table information needs to be parsed separately, and the parsed information is input into the

[0089] In addition to detecting the above information indication fields and writing the corresponding information into different control modules, the function of the frame information parsing module also includes detecting whether the above information changes in units of frames. If it changes, the information data in the corresponding control module is updated; if it does not change, the data in the corresponding control module is not updated. For example, if the luminance quantization table used for the 10th frame compression is different from that used for the 11th frame compression, then when the 11th frame compression data arrives, the content of the luminance information quantization table control module is updated, and the content of other control modules remains unchanged.

[0090] After the frame information parsing module parses the SOS indication field (FF DA), it indicates that the subsequent data is compressed data blocks in units of BLOCKs. The compressed data blocks can be written into the on-chip cache 50 through the on-chip cache write control module. Therefore, in this embodiment, a mechanism is implemented to store the compressed data in units of BLOCKs (i.e., compressed data blocks) separately, and the common information of video frame data, luminance quantization table, chrominance quantization table, Huffman table, etc. are stored separately. Therefore, a mechanism is implemented to share the common information of video frame data, luminance quantization table, chrominance quantization table, and Huffman table information among different frame data. On the read side of the on-chip cache 50, the on-chip cache read control module can recombine the compressed data blocks into complete JPEG format data, that is, obtain the corresponding data from the frame common information control module, frame basic information control module, Huffman table control module, luminance information quantization table control module, chrominance information quantization table control module, and the on-chip cache respectively, and splice them into complete JPEG format video frame data for output. Thus, it avoids the drawback in the related art that the same data such as the common information of compressed video data (such as multi-frame JPEG compressed images), luminance quantization table, chrominance quantization table, and Huffman table are repeatedly written into the on-chip cache 50, reduces the time of compressed data in the on-chip cache 50, reduces the latency of compressed data, and reduces the video frame loss rate.

[0091] It should be noted that the dynamic frame compensation control module 40 can start the dynamic frame compensation function when the subsequent read cache situation is in the read state. For the specific method of predicting the predicted video data block by the dynamic frame compensation control module 40 according to the target video data blocks stored in the on-chip cache 50 when the subsequent read cache situation is in the read state, the predicted video data block can be directly predicted according to the preset number of target video data blocks corresponding to the predicted video data block; for example, directly use the 4 video data blocks (i.e., target video data blocks) above, to the left, top left, and top right of the predicted video data block in the current video frame data in the on-chip cache 50 to predict the predicted video data block.

[0092] Correspondingly, in order to reduce the calculation amount of dynamic frame compensation, the dynamic frame compensation control module 40 can determine the target video data blocks stored in the on-chip cache 50 according to the image change situation of the current video frame data; and predict the predicted video data block according to the target video data blocks. That is to say, the dynamic frame compensation control module 40 selects different target video data blocks for prediction according to the image change situation of the current video frame data (i.e., the image change size within one frame of image).

[0093] For example, the process of determining each target video data block stored in the on-chip cache 50 according to the image change condition of the current video frame data may include: if the image change condition of the current video frame data is a state where the change amount is not greater than the threshold, the video data blocks above and to the left of the predicted video data block in the current video frame data in the on-chip cache 50 are respectively used as target video data blocks; if the image change condition of the current video frame data is a state where the change amount is greater than the threshold, the video data blocks above, to the left, to the upper left, and to the upper right of the predicted video data block in the current video frame data in the on-chip cache 50 are respectively used as target video data blocks.

[0094] Correspondingly, the process of predicting the predicted video data block according to each target video data block may include: when the number of target video data blocks is 2 (such as the above state where the change amount is not greater than the threshold), the predicted video data block is calculated by BLOCK_PREDICT = A_0 * BLOCK_0 + A_1 * BLOCK_2; where BLOCK_PREDICT is the predicted video data block, BLOCK_0 and BLOCK_2 are 2 target video data blocks, A_0 and A_1 are preset frame compensation parameters, and A_0 + A_1 = 1; when the number of target video data blocks is 4 (such as the above state where the change amount is greater than the threshold), the predicted video data block is calculated by BLOCK_PREDICT = B_0 * BLOCK_0 + B_1 * BLOCK_1 + B_2 * BLOCK_2 + B_3 * BLOCK_3; where BLOCK_0 to BLOCK_3 are 4 target video data blocks, and B_0 to B_3 are preset frame compensation parameters, and B_0 + B_1 + B_2 + B_3 = 1.

[0095] For example, when the current video frame data is data in a preset compression format (such as JPEG compression format), as Figure 4 shown, the dynamic frame compensation control module 40 can, when the subsequent cache reading condition is the read state, use the compressed data blocks in units of BLOCK (block) in the on-chip cache 50 to predict the compressed data block that cannot be continuously written for the current video frame due to the full state of the on-chip cache 50; at the same time, the dynamic frame compensation control module 40 can select different compressed data blocks as target video data blocks for prediction according to the change frequency of the image in the original video frame data transmitted by the server host (that is, the image change condition of the current video frame data); as Figure 6As shown, BLOCK_PREDICT can be the compressed data block that is currently about to be written into the on-chip cache 50 but the on-chip cache 50 is in a full state. BLOCK_0, BLOCK_1, BLOCK_2, and BLOCK_3 are the compressed data blocks adjacent to BLOCK_PREDICT that have already been written into the on-chip cache 50. When the image change in the original video frame data is small (such as the change frequency is not greater than the change frequency threshold), it can be determined that the image change situation of the current video frame data is in a state not greater than the threshold, and BLOCK_0 and BLOCK_2 can be used to predict the data of BLOCK_PREDICT, that is, using BLOCK_0 on the left and BLOCK_2 above BLOCK_PREDICT in the current video frame data in the on-chip cache 50 as the target video data blocks for prediction, such as BLOCK_PREDICT = A_0 * BLOCK_0 + A_1 * BLOCK_2; where A_0 + A_1 = 1; that is to say, when the intra-frame image change is very small, good prediction can be completed with a small number of target video data blocks, reducing the computational amount of predicted frame compensation. When the image change in the original video frame data is large (such as the change frequency is greater than the change frequency threshold), it can be determined that the image change situation of the current video frame data is in a state greater than the threshold, and BLOCK_1 to BLOCK_3 can be used to predict the data of BLOCK_PREDICT, that is, using BLOCK_0 on the left, BLOCK_2 above, BLOCK_1 in the upper left, and BLOCK_3 in the upper right of BLOCK_PREDICT in the current video frame data in the on-chip cache 50 as the target video data blocks for prediction, such as BLOCK_PREDICT = B_0 * BLOCK_0 + B_1 * BLOCK_1 + B_2 * BLOCK_2 + B_3 * BLOCK_3; where B_0 + B_1 + B_2 + B_3 = 1; that is to say, when the intra-frame image change is large, more target video data blocks can be used for prediction to improve the accuracy of predicted frame compensation.

[0096] After the dynamic frame compensation control module 40 finishes obtaining the predicted data of BLOCK_PREDICT (i.e., the predicted video data block), at this time the on-chip cache 50 is in a full state. When it is in a non-full state next, the predicted data can be written into the on-chip cache 50; at the same time, it can be synchronously updated according to the positional relationship Figure 6The BLOCK_PREDICT data in []. Correspondingly, in this embodiment, the dynamic adjustment function of preset frame interpolation parameters (such as A_0, A_1, B_0 to B_3 in the above formula) can also be implemented; for example, when the subsequent read cache situation is in the read state, the dynamic frame interpolation control module 40 is specifically used to predict the predicted video data block according to each target video data block and the preset frame interpolation parameters corresponding to each target video data block. The video function system provided in this embodiment may further include: a frame interpolation parameter adjustment and correction module, which is used to predict the predicted video data block corresponding to the current video data block to be written according to each target video data block and the preset frame interpolation parameters corresponding to the current video data block stored in the on-chip cache 50 at a preset time interval when the on-chip cache 50 is not full; and adjust the preset frame interpolation parameters according to the comparison between the current video data block to be written and the corresponding predicted video data block.

[0097] As Figure 4 shown, the read cache prediction module 30 can start the frame interpolation function of the dynamic frame interpolation control module 40 when the on-chip cache 50 is not completely full. At this time, the on-chip cache 50 can continue to receive the real data of the current compressed data block (i.e., the current video data block to be written), and at the same time, the predicted data of the current compressed data block (i.e., the predicted video data block) can also be obtained. By comparing the real data and the predicted data, the corresponding preset frame interpolation parameters (such as A_0 and A_1 or B_0 to B_3) can be adjusted, so that the predicted data is equal to (or close to) the real data, realizing the dynamic adjustment and correction function of the preset frame interpolation parameters to improve the accuracy of frame interpolation. For example, a fixed time interval (i.e., the preset time) can be set to perform dynamic parameter adjustment on the frame interpolation function. The above dynamic adjustment and correction function of the preset frame interpolation parameters can be completed by Figure 4 the frame interpolation parameter adjustment and correction module in []. The transmission of the compressed data block as the target video data block in the on-chip cache 50 can be completed by Figure 4 the compressed block information cache control module in []. The calculation of BLOCK_PREDICT is completed by Figure 4 the dynamic frame interpolation control module 40 in [].

[0098] Correspondingly, the frame interpolation process when the current video frame data is in a preset video data format (such as the YUV format) is similar to the frame interpolation process of the data in the above preset compression format, except that the compressed data block is replaced with the data in the preset video data format. As Figure 7As shown, YUV_PREDICT is the YUV data (i.e., video data block) for which the frame is to be filled. Its length can be configured by the user register or can adopt a default value, such as 8 or 16, etc. YUV_0, YUV_1, YUV_2, and YUV_3 are the YUV data adjacent to YUV_PREDICT that have been written into the on-chip cache 50. Similarly, when the image change in the original video frame data is small, it can be determined that the image change situation of the current video frame data is in a state not greater than the threshold, and YUV_0 and YUV_2 can be used to predict the data of YUV_PREDICT, such as YUV_PREDICT = C_0 * YUV_0 + C_1 * YUV_2; where, C_0 + C_1 = 1; while when the image change in the original video frame data is large, it can be determined that the image change situation of the current video frame data is in a state greater than the threshold, and YUV_0 to YUV_3 can be used to predict the data of YUV_PREDICT, such as YUV_PREDICT = D_0 * YUV_0 + D_1 * YUV_1 + D_2 * YUV_2 + D_3 * YUV_3; where, D_0 + D_1 + D_2 + D_3 = 1. At the same time, the frame filling parameter adjustment and correction module can be used to complete the dynamic adjustment and correction of the preset frame filling parameters (such as C_0, C_1, D_0 to D_3 in the above formulas). The process is similar to the processing process under the preset compression format and will not be elaborated here. The dynamic adjustment and correction function of the preset frame filling parameters under the above preset video data format can be completed by Figure 4 the frame filling parameter adjustment and correction module in; the transmission of the YUV data in the on-chip cache 50 as the target video data block can be completed by Figure 4 the YUV information cache control module (i.e., the uncompressed block information cache control module) in, and the calculation of YUV_PREDICT can be completed by Figure 4 the dynamic frame filling control module 40 in.

[0099] That is to say, the system provided in this embodiment can also include a compressed block information cache control module and / or an uncompressed block information cache control module; the compressed block information cache control module is used to transfer each target video data block stored in the on-chip cache 50 to the dynamic frame filling control module 40 when the current video frame data received by the video data output control module is data in a preset compression format; the compressed block information cache control module is used to transfer each target video data block stored in the on-chip cache 50 to the dynamic frame filling control module 40 when the current video frame data received by the video data output control module is data in a preset video data format; where, the preset frame filling parameters (such as the above A_0 and A_1 or B_0 to B_3) corresponding to each target video data block in the preset compression format are different from the preset frame filling parameters (such as the above C_0 and C_1 or D_0 to D_3) corresponding to each target video data block in the preset video data format.

[0100] It should be noted that the specific application scenario of the storage application chip is not limited in this embodiment. For example, the storage application chip can be the BMC of a server. Correspondingly, the storage application chip can also include modules such as Figure 1 the VGA (Video Graphics Array) module, video data capture module, color space conversion module, core compression module, and network module as shown. For example, when the KVM (Keyboard, Video, and Mouse) function is started, the server host will transfer the original video data that needs to be remotely displayed (i.e., the remote display image of the operating system, such as the real-time interface of the operating system) to the VGA module of the BMC through PCIe (a high-speed serial computer expansion bus standard). The VGA module generates the original video data in RGB format, and this process needs to interact with the external DDR (i.e., video memory), such as Figure 1 the A area of the video memory (such as DDR) in ; The video data capture module obtains the original video data in RGB format from the VGA module and transmits it to the color space conversion module; The color space conversion module can perform color space conversion on the original video data in RGB format to obtain data in YUV format. The conversion process can use the following formulas: Y=(0.257*R)+(0.504*G)+(0.098*B)+16, U=0.148*R–0.291*G+0.439*B+128, and V=0.439*R-0.368*G-0.071*B+128.

[0101] Correspondingly, the core compression module can perform video compression on the received YUV-format data in the corresponding video compression format (such as H.264, JPEG, and AVS video compression formats) to obtain the compressed video data; The compressed video output control module can receive the compressed video data or the uncompressed YUV-format video data and store it in the external DDR through the on-chip cache 50, such as Figure 1 the B area of the video memory in, so that the network module can send the video data output by the compressed video output control module in the video memory (such as DDR) to the device at the remote end for display.

[0102] In this embodiment, the read cache prediction module 30 added to the video data output control module in the embodiment of the present invention can, when the on-chip cache 50 is full, determine whether to perform dynamic frame compensation processing on the video frame being written according to the detection results output by the system state detection module 10 and the off-chip storage interface detection module 20. Thus, when it is determined to perform dynamic frame compensation processing, the dynamic frame compensation control module 40 performs dynamic frame compensation on the video frame that cannot be written continuously due to the full write of the on-chip cache 50, improving the complete write rate of video frame data, reducing the frame loss rate of the video function of the storage application chip, reducing the processing process of invalid data, enhancing the utilization rate of the system bus resources and storage bandwidth by the video function, and improving the performance of the storage application chip.

[0103] Corresponding to the above system embodiment, the embodiment of the present invention also provides a storage application chip. A storage application chip described below can be correspondingly referred to with the video function system of a storage application chip described above.

[0104] A storage application chip includes: the video function system of the storage application chip provided in the above embodiment.

[0105] Among them, the storage application chip provided in this embodiment can specifically be a BMC.

[0106] Corresponding to the above system embodiment, the embodiment of the present invention also provides a server. A server described below can be correspondingly referred to with a storage application chip described above.

[0107] A server includes: the storage application chip provided in the above embodiment.

[0108] The embodiments in the specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For the storage application chip and server disclosed in the embodiments, since they correspond to the system disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the system part.

[0109] The above has introduced in detail a video function system, a storage application chip, and a server of a storage application chip provided by the present invention. Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the protection scope of the present invention.

Claims

1. A video function system for a storage application chip, characterized in that: A video data output control module applied to a storage application chip, including: an on-chip cache, a system status detection module, an off-chip storage interface detection module, a read cache prediction module, and a dynamic frame supplement control module; The read cache prediction module is used to determine the subsequent read cache status when the on-chip cache is full according to the first frequency detected by the system status detection module and the second frequency detected by the off-chip storage interface detection module; wherein the first frequency is the frequency of the read and write operation commands initiated by the interface connected to the system bus by the central processing unit in the storage application chip, the second frequency is the read and write frequency of all interfaces on the on-chip storage controller in the storage application chip, and the subsequent read cache status includes a read state and a non-read state; The dynamic frame supplement control module is used to determine each target video data block stored in the on-chip cache according to the image change of the current video frame data when the subsequent read cache status is the read state; predict the predicted video data block according to each target video data block, so as to write the predicted video data block to the on-chip cache when the on-chip cache is not full; wherein the target video data block is a video data block adjacent to the predicted video data block in the current video frame data.

2. The video function system of the storage application chip according to claim 1, characterized in that: The read cache prediction module is specifically used to determine the subsequent read cache status when the on-chip cache is full, according to the first frequency and the second frequency corresponding to each first preset time period within the first detection time period; wherein the number of the first preset time periods is greater than or equal to 2, and the sum of all the first preset time periods is the first detection time period.

3. The video function system of the storage application chip according to claim 2, characterized in that: When the on-chip cache is full, determining the subsequent read cache status according to the first frequency and the second frequency corresponding to each first preset time period within the first detection time period, includes: When the on-chip cache is full, if PARAM_0 * (P_0 * Freq_sys0 + P_1 * Freq_sys1 + P_2 * Freq_sys2 + …… P_N-1 * Freq_sysN-1) + PARAM_1 * (Q_0 * Freq_DDR0 + Q_1 * Freq_DDR1 + Q_2 * Freq_DDR2 + …… Q_N-1 * Freq_DDRN-1) < THREHOLD_0, it is determined that the subsequent read cache situation is the read state; if PARAM_0 * (P_0 * Freq_sys0 + P_1 * Freq_sys1 + P_2 * Freq_sys2 + …… P_N-1 * Freq_sysN-1) + PARAM_1 * (Q_0 * Freq_DDR0 + Q_1 * Freq_DDR1 + Q_2 * Freq_DDR2 + …… Q_N-1 * Freq_DDRN-1) ≥ THREHOLD_0, it is determined that the subsequent read cache situation is the non-read state to discard the current video frame data; where N is the number of the first preset time periods, PARAM_0 and P_0 to P_N-1 are the preset parameters corresponding to the first frequency respectively, PARAM_1 and Q_0 to Q_N-1 are the preset parameters corresponding to the second frequency respectively, Freq_sys0 to Freq_sysN-1 are the first frequencies corresponding to the 1st to the Nth first preset time periods respectively, Freq_DDR0 to Freq_DDRN-1 are the second frequencies corresponding to the 1st to the Nth first preset time periods respectively, and THREHOLD_0 is the first preset threshold; P_0 > P_1 > P_2 …… > P_N-1, Q_0 > Q_1 > Q_2 …… > Q_N-1.

4. The video function system of the storage application chip according to claim 3, characterized in that: PARAM_1 > PARAM_0, Q_i > P_i, where i is an integer greater than or equal to 0 and less than N.

5. The video function system of the storage application chip according to claim 3, characterized in that: The read cache prediction module is further configured to, after determining that the subsequent read cache situation is the non-read state, detect the data read-out situation of the on-chip cache, and determine whether the determination of the subsequent read cache situation is correct according to the data read-out situation; if not, PARAM_1, PARAM_0, P_0 to P_N-1, and Q_0 to Q_N-1 are decreased.

6. The video function system of the storage application chip according to claim 5, characterized in that: During the process of decreasing PARAM_1, PARAM_0, P_0 to P_N-1, and Q_0 to Q_N-1, the decreasing amplitude of the preset parameter corresponding to the first frequency is less than the decreasing amplitude of the preset parameter corresponding to the second frequency.

7. The video function system of the storage application chip according to claim 2, characterized in that: The subsequent read cache situation further includes an invalid frame state. Determining the subsequent read cache situation according to the first frequencies and the second frequencies corresponding to the respective first preset time periods within the first detection time period includes: Determine the subsequent read cache status according to the first frequency and the second frequency corresponding to each of the first preset time periods in the first detection time period and the first frequency and the second frequency corresponding to each of the second preset time periods in the second detection time period; The video function system also includes: The pre-frame loss control module is used to control the previous hardware module corresponding to the video data output control module in the storage application chip to stop processing the current video frame data when the subsequent read cache situation is the invalid frame state.

8. The video function system of the storage application chip according to claim 7, characterized in that: The determining the subsequent read cache status according to the first frequency and the second frequency corresponding to each of the first preset time periods within the first detection time period and the first frequency and the second frequency corresponding to each of the second preset time periods within the second detection time period includes: If PARAM_2*(R_0*Freq_sys0+R_1*Freq_sys1+R_2*Freq_sys2+…R_M-1*Freq_sysM-1)+PARAM_3*(S_0*Freq_DDR0+S_1*Freq_DDR1+S_2*Freq_DDR2+…S_M-1*Freq_DDRM-1)>THREHOLD_1, it is determined that the subsequent read cache situation is the invalid frame state; wherein M is the number of the second preset time period, PARAM_2 and R_0 to R_M-1 are respectively are respectively the preset parameters corresponding to the first frequency, PARAM_3 and S_0 to S_M-1 are respectively the preset parameters corresponding to the second frequency, Freq_sys0 to Freq_sysM-1 are the first frequencies corresponding to the 1st first preset time period to the Mth first preset time period, Freq_DDR0 to Freq_DDRM-1 are the second frequencies corresponding to the 1st first preset time period to the Mth first preset time period, THREHOLD_1 is the second preset threshold; R_0>R_1>R_2……>R_M-1, S_0>S_1>S_2……>S_M-1.

9. The video function system of the storage application chip according to claim 8, characterized in that: PARAM_3>PARAM_2, S_j>R_j, j is an integer greater than or equal to 0 and less than M.

10. The video function system of the storage application chip according to claim 8, characterized in that: The read cache prediction module is also used to detect the data read status of the on-chip cache after determining that the subsequent read cache status is the invalid frame state, and determine whether the determination of the subsequent read cache status is correct based on the data read status; if incorrect, reduce PARAM_2, PARAM_3, R_0 to R_M-1 and S_0 to S_M-1.

11. The video function system of the storage application chip according to claim 1, characterized in that: The determining, according to the image change of the current video frame data, each of the target video data blocks stored in the on-chip cache comprises: If the image change of the current video frame data is in a state where the change amount is not greater than a threshold value, the video data blocks above and to the left of the predicted video data block in the current video frame data in the on-chip buffer are respectively used as the target video data blocks; If the image change of the current video frame data is greater than a threshold value, the video data blocks above, to the left, above left and above right of the predicted video data block in the current video frame data in the on-chip cache are respectively used as the target video data blocks.

12. The video function system of the storage application chip according to claim 1, characterized in that: The step of predicting a predicted video data block according to each of the target video data blocks comprises: When the number of the target video data blocks is 2, the predicted video data block is calculated by BLOCK_PREDICT=A_0*BLOCK_0+A_1*BLOCK_2; wherein BLOCK_PREDICT is the predicted video data block, BLOCK_0 and BLOCK_2 are two target video data blocks, A_0 and A_1 are preset interpolation frame parameters, and A_0+A_1=1; When the number of the target video data blocks is 4, the predicted video data block is calculated by BLOCK_PREDICT=B_0*BLOCK_0+B_1*BLOCK_1+B_2*BLOCK_2+B_3*BLOCK_3; wherein BLOCK_0 to BLOCK_3 are the 4 target video data blocks, B_0 to B_3 are preset frame supplementation parameters, and B_0+B_1+B_2+B_3=1.

13. The video function system of the storage application chip according to claim 1, characterized in that: The dynamic frame interpolation control module is specifically used to predict and obtain a predicted video data block according to each of the target video data blocks and the preset frame interpolation parameters corresponding to each of the target video data blocks when the subsequent read cache status is the read state; The video function system also includes: A frame interpolation parameter adjustment and correction module is used to predict, when the on-chip cache is not full, the predicted video data block corresponding to the current video data block to be written according to each target video data block corresponding to the current video data block to be written stored in the on-chip cache and the preset frame interpolation parameter at a preset time interval; The preset frame interpolation parameters are adjusted according to a comparison between the current video data block to be written and the corresponding predicted video data block.

14. The video function system of the storage application chip according to claim 13, characterized in that: Also includes: A compressed block information cache control module and / or a non-compressed block information cache control module; A compression block information cache control module, configured to transfer each target video data block stored in the on-chip cache to the dynamic frame supplement control module when the current video frame data received by the video data output control module is data in a preset compression format; a non-compressed block information cache control module, configured to transfer each target video data block stored in the on-chip cache to the dynamic frame supplement control module when the current video frame data received by the video data output control module is data in a preset video data format; The preset frame interpolation parameters corresponding to each target video data block in the preset compression format are different from the preset frame interpolation parameters corresponding to each target video data block in the preset video data format.

15. The video function system of the storage application chip according to any one of claims 1 to 14, characterized in that: When the video data received by the video data output control module includes data in a preset compression format, the video function system further includes: a frame information parsing module, a frame common information control module, a frame basic information control module, a compression algorithm table control module, a brightness information quantization table control module, a chrominance information quantization table control module, an on-chip cache write control module, and an on-chip cache read control module; The frame information parsing module is used to parse the received current video frame data in the preset compression format to obtain frame commonality information, frame basic information, compression algorithm table information, brightness information quantization table information, chrominance information quantization table information and compressed data blocks of the current video frame data; wherein the compressed data block is compressed data in the current video frame data in blocks, and the preset compression format is an intra-frame compression format; The frame commonality information control module is used to update and store the frame commonality information of the current video frame data; The frame basic information control module is used to update and store the frame basic information of the current video frame data; The compression algorithm table control module is used to update the compression algorithm table information storing the current video frame data; The brightness information quantization table control module is used to update the brightness information quantization table information storing the current video frame data; The chrominance information quantization table control module is used to update the brightness information quantization table information storing the current video frame data; The on-chip cache write control module is used to store the compressed data blocks of the current video frame data in the on-chip cache in sequence in units of blocks; The on-chip cache read control module is used to restore the compressed data block read from the on-chip cache to the data in the preset compression format according to the data stored in the frame common information control module, the frame basic information control module, the compression algorithm table control module, the brightness information quantization table control module and the chrominance information quantization table control module.

16. The video function system of the storage application chip according to claim 15, characterized in that: The preset compression format is the Joint Photographic Experts Group compression format, and the compression algorithm table control module is specifically a Huffman table control module, which is used to store Huffman table information of current video frame data.

17. The video function system of the storage application chip according to claim 1, characterized in that: The storage application chip is specifically a baseboard management controller.

18. A storage application chip, characterized in that: include: A video function system for a storage application chip as claimed in any one of claims 1 to 17.

19. A server, characterized in that: include: The storage application chip as claimed in claim 18.

Images