Video function system of storage application chip, storage application chip and server

Abstract

The invention discloses a video function system of a storage application chip, the storage application chip and a server, and relates to the technical field of chip design, the system is applied to a video data output control module of the storage application chip, and the system comprises a read cache prediction module, a read cache prediction module and a read cache prediction module, according to the first frequency detected by the system state detection module and the second frequency detected by the off-chip storage interface detection module, the subsequent cache reading condition is determined; the dynamic frame compensation control module is used for predicting to obtain a predicted video data block according to each target video data block stored in the on-chip cache when the subsequent read cache condition is a read state; the method comprises the following steps of: when an on-chip cache is fully written, determining whether to carry out dynamic frame compensation processing on a video frame which is being written or not by utilizing a read cache prediction module; and the dynamic frame filling control module is used for carrying out dynamic frame filling on the video frames which cannot be continuously written because the on-chip cache is fully written, so that the frame loss rate of a video function is reduced, and the performance of the storage application chip is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of chip design, and in particular to a video function system of a storage application chip, a storage application chip and a server. Background Art

[0002] At present, the processing flow of the video function of the SoC (System on Chip) chip in the traditional storage application scenario is to first convert the original RGB (red, green, and blue) format data into a YUV (brightness, chroma, and density) format video data through color space conversion, compress the YUV format data (such as JPEG and AVS video compression formats) or directly transmit the YUV format data; write the compressed data or YUV format data to an external memory (such as a double data rate synchronous dynamic random access memory DDR) through the video data output control module of the SoC chip.

[0003] like Figure 1 As shown in the figure, the processing flow of the video function in a typical storage application chip with integrated video function (such as a baseboard management controller BMC) is as follows: the original video data of the server host is transmitted to the VGA (Video Graphics Array) module inside the storage application chip through PCIe (a high-speed serial computer expansion bus standard), and the VGA generates original video data in RGB format, which is then processed as described above. However, the operating system and application programs running on the storage application chip share an external memory (such as DDR memory) and an on-chip storage controller (such as an on-chip DDR controller) and a system bus. Therefore, there is often competition for control of the memory bus, which often causes the video function of the server management controller to be unable to write video data to the external memory in a timely manner, thereby causing the on-chip cache (used to cache video data) to be full, resulting in the video frame currently being written to the on-chip cache to be dropped. Even if the on-chip cache is full and then becomes unfull (such as the video output control module competes for control of the memory bus again), the video frame currently being written has already been dropped at this time, causing the video data already written to the on-chip cache to become invalid data, resulting in a high frame loss rate for the video function and a large amount of invalid data being written.

[0004] Therefore, how to reduce the frame loss rate of the video function of the storage application chip, reduce the processing of invalid data, and improve the performance of the storage application chip is an urgent problem to be solved today. Summary of the invention

[0005] The purpose of the present invention is to provide a video function system of a storage application chip, a storage application chip and a server to reduce the frame loss rate of the video function of the storage application chip, reduce the processing process of invalid data, and improve the performance of the storage application chip.

[0006] In order to solve the above technical problems, the present invention provides a video function system of a storage application chip, which is applied to a video data output control module of the storage application chip, including: an on-chip cache, a system status detection module, an off-chip storage interface detection module, a read cache prediction module and a dynamic frame supplement control module;

[0007] The read cache prediction module is used to determine the subsequent read cache status when the on-chip cache is full according to the first frequency detected by the system status detection module and the second frequency detected by the off-chip storage interface detection module; wherein the first frequency is the frequency of the read and write operation commands initiated by the interface connected to the system bus by the central processing unit in the storage application chip, the second frequency is the read and write frequency of all interfaces on the on-chip storage controller in the storage application chip, and the subsequent read cache status includes a read state and a non-read state;

[0008] The dynamic frame supplement control module is used to predict a predicted video data block based on each target video data block stored in the on-chip cache when the subsequent read cache situation is the read state, so as to write the predicted video data block into the on-chip cache when the on-chip cache is not full; wherein the target video data block is a video data block adjacent to the predicted video data block in the current video frame data.

[0009] On the other hand, the read cache prediction module is specifically used to determine the subsequent read cache situation when the on-chip cache is full, based on the first frequency and the second frequency corresponding to each first preset time period within the first detection time period; wherein the number of the first preset time periods is greater than or equal to 2, and the sum of all the first preset time periods is the first detection time period.

[0010] On the other hand, when the on-chip cache is full, determining the subsequent read cache status according to the first frequency and the second frequency corresponding to each first preset time period in the first detection time period includes:

[0011] When the on-chip cache is full, if PARAM_0 * (P_0 * Freq_sys0 + P_1 * Freq_sys1 + P_2 * Freq_sys2 + …… P_N-1 * Freq_sysN-1) + PARAM_1 * (Q_0 * Freq_DDR0 + Q_1 * Freq_DDR1 + Q_2 * Freq_DDR2 + …… Q_N-1 * Freq_DDRN-1) < THREHOLD_0, it is determined that the subsequent read cache situation is the read state; if PARAM_0 * (P_0 * Freq_sys0 + P_1 * Freq_sys1 + P_2 * Freq_sys2 + …… P_N-1 * Freq_sysN-1) + PARAM_1 * (Q_0 * Freq_DDR0 + Q_1 * Freq_DDR1 + Q_2 * Freq_DDR2 + …… Q_N-1 * Freq_DDRN-1) ≥ THREHOLD_0, it is determined that the subsequent read cache situation is the non-read state to discard the current video frame data; where N is the number of the first preset time periods, PARAM_0 and P_0 to P_N-1 are the preset parameters corresponding to the first frequency respectively, PARAM_1 and Q_0 to Q_N-1 are the preset parameters corresponding to the second frequency respectively, Freq_sys0 to Freq_sysN-1 are the first frequencies corresponding to the 1st to Nth first preset time periods respectively, Freq_DDR0 to Freq_DDRN-1 are the second frequencies corresponding to the 1st to Nth first preset time periods respectively, and THREHOLD_0 is the first preset threshold; P_0 > P_1 > P_2 …… > P_N-1, Q_0 > Q_1 > Q_2 …… > Q_N-1.

[0012] On the other hand, PARAM_1 > PARAM_0, Q_i > P_i, where i is an integer greater than or equal to 0 and less than N.

[0013] On the other hand, after determining that the subsequent read cache situation is the non-read state, the read cache prediction module is further configured to detect the data read-out situation of the on-chip cache, and determine whether the determination of the subsequent read cache situation is correct according to the data read-out situation; if not, PARAM_1, PARAM_0, P_0 to P_N-1, and Q_0 to Q_N-1 are decreased.

[0014] On the other hand, during the process of decreasing PARAM_1, PARAM_0, P_0 to P_N-1, and Q_0 to Q_N-1, the decreasing amplitude of the preset parameter corresponding to the first frequency is less than the decreasing amplitude of the preset parameter corresponding to the second frequency.

[0015] On the other hand, the subsequent read cache condition further includes an invalid frame state, and the determining of the subsequent read cache condition according to the first frequency and the second frequency corresponding to each first preset time period within the first detection time period includes:

[0016] determining the subsequent read cache status according to the first frequency and the second frequency corresponding to each of the first preset time periods within the first detection time period and the first frequency and the second frequency corresponding to each of the second preset time periods within the second detection time period;

[0017] The video function system also includes:

[0018] The pre-frame loss control module is used to control the previous hardware module corresponding to the video data output control module in the storage application chip to stop processing the current video frame data when the subsequent read cache situation is the invalid frame state.

[0019] On the other hand, determining the subsequent read cache situation according to the first frequency and the second frequency corresponding to each of the first preset time periods in the first detection time period and the first frequency and the second frequency corresponding to each of the second preset time periods in the second detection time period includes:

[0020] If PARAM_2*(R_0*Freq_sys0+R_1*Freq_sys1+R_2*Freq_sys2+…R_M-1*Freq_sysM-1)+PARAM_3*(S_0*Freq_DDR0+S_1*Freq_DDR1+S_2*Freq_DDR2+…S_M-1*Freq_DDRM-1)>THREHOLD_1, it is determined that the subsequent read cache situation is the invalid frame state; wherein M is the number of the second preset time period, PARAM_2 and R_0 to R_M-1 are respectively are respectively the preset parameters corresponding to the first frequency, PARAM_3 and S_0 to S_M-1 are respectively the preset parameters corresponding to the second frequency, Freq_sys0 to Freq_sysM-1 are the first frequencies corresponding to the 1st first preset time period to the Mth first preset time period, Freq_DDR0 to Freq_DDRM-1 are the second frequencies corresponding to the 1st first preset time period to the Mth first preset time period, THREHOLD_1 is the second preset threshold; R_0>R_1>R_2……>R_M-1, S_0>S_1>S_2……>S_M-1.

[0021] On the other hand, PARAM_3>PARAM_2, S_j>R_j, j is an integer greater than or equal to 0 and less than N.

[0022] On the other hand, the read cache prediction module is also used to detect the data read status of the on-chip cache after determining that the subsequent read cache status is the invalid frame state, and determine whether the subsequent read cache status is correct based on the data read status; if incorrect, reduce PARAM_2, PARAM_3, R_0 to R_M-1 and S_0 to S_M-1.

[0023] On the other hand, predicting the predicted video data blocks according to the target video data blocks stored in the on-chip cache includes:

[0024] Determining each of the target video data blocks stored in the on-chip cache according to image changes of the current video frame data;

[0025] According to each of the target video data blocks, a predicted video data block is predicted.

[0026] On the other hand, determining each of the target video data blocks stored in the on-chip cache according to the image change of the current video frame data includes:

[0027] If the image change of the current video frame data is in a state where the change amount is not greater than a threshold value, the video data blocks above and to the left of the predicted video data block in the current video frame data in the on-chip buffer are respectively used as the target video data blocks;

[0028] If the image change of the current video frame data is greater than a threshold value, the video data blocks above, to the left, above left and above right of the predicted video data block in the current video frame data in the on-chip cache are respectively used as the target video data blocks.

[0029] On the other hand, predicting a predicted video data block according to each of the target video data blocks includes:

[0030] When the number of the target video data blocks is 2, the predicted video data block is calculated by BLOCK_PREDICT=A_0*BLOCK_0+A_1*BLOCK_2; wherein BLOCK_PREDICT is the predicted video data block, BLOCK_0 and BLOCK_2 are two target video data blocks, A_0 and A_1 are preset interpolation frame parameters, and A_0+A_1=1;

[0031] When the number of the target video data blocks is 4, the predicted video data block is calculated by BLOCK_PREDICT=B_0*BLOCK_0+B_1*BLOCK_1+B_2*BLOCK_2+B_3*BLOCK_3; wherein BLOCK_0 to BLOCK_3 are the 4 target video data blocks, B_0 to B_3 are preset frame supplementation parameters, and B_0+B_1+B_2+B_3=1.

[0032] On the other hand, the dynamic frame interpolation control module is specifically used to predict and obtain a predicted video data block according to each of the target video data blocks and the preset frame interpolation parameters corresponding to each of the target video data blocks when the subsequent read cache status is the read state;

[0033] The video function system also includes:

[0034] The frame interpolation parameter adjustment and correction module is used to predict, when the on-chip cache is not full, the predicted video data block corresponding to the current video data block to be written according to the target video data blocks corresponding to the current video data block to be written stored in the on-chip cache and the preset frame interpolation parameters at preset time intervals; and adjust the preset frame interpolation parameters according to the comparison between the current video data block to be written and the corresponding predicted video data block.

[0035] In another aspect, the system further comprises: a compressed block information cache control module and / or a non-compressed block information cache control module;

[0036] A compression block information cache control module, configured to transfer each target video data block stored in the on-chip cache to the dynamic frame supplement control module when the current video frame data received by the video data output control module is data in a preset compression format;

[0037] a non-compressed block information cache control module, configured to transfer each target video data block stored in the on-chip cache to the dynamic frame supplement control module when the current video frame data received by the video data output control module is data in a preset video data format;

[0038] The preset frame interpolation parameters corresponding to each target video data block in the preset compression format are different from the preset frame interpolation parameters corresponding to each target video data block in the preset video data format.

[0039] On the other hand, when the video data received by the video data output control module includes data in a preset compression format, the video function system further includes: a frame information parsing module, a frame common information control module, a frame basic information control module, a compression algorithm table control module, a brightness information quantization table control module, a chrominance information quantization table control module, an on-chip cache write control module, and an on-chip cache read control module;

[0040] The frame information parsing module is used to parse the received current video frame data in the preset compression format to obtain frame commonality information, frame basic information, compression algorithm table information, brightness information quantization table information, chrominance information quantization table information and compressed data blocks of the current video frame data; wherein the compressed data block is compressed data in the current video frame data in blocks, and the preset compression format is an intra-frame compression format;

[0041] The frame commonality information control module is used to update and store the frame commonality information of the current video frame data;

[0042] The frame basic information control module is used to update and store the frame basic information of the current video frame data;

[0043] The compression algorithm table control module is used to update the compression algorithm table information storing the current video frame data;

[0044] The brightness information quantization table control module is used to update the brightness information quantization table information storing the current video frame data;

[0045] The chrominance information quantization table control module is used to update the brightness information quantization table information storing the current video frame data;

[0046] The on-chip cache write control module is used to store the compressed data blocks of the current video frame data in the on-chip cache in sequence in units of blocks;

[0047] The on-chip cache read control module is used to restore the compressed data block read from the on-chip cache to the data in the preset compression format according to the data stored in the frame common information control module, the frame basic information control module, the compression algorithm table control module, the brightness information quantization table control module and the chrominance information quantization table control module.

[0048] On the other hand, the preset compression format is a Joint Photographic Experts Group compression format, and the compression algorithm table control module is specifically a Huffman table control module, which is used to store Huffman table information of current video frame data.

[0049] On the other hand, the storage application chip is specifically a baseboard management controller.

[0050] The present invention also provides a storage application chip, including: the video function system of the storage application chip as described above.

[0051] In addition, the present invention also provides a server, including: the storage application chip as described above.

[0052] A video function system of a storage application chip provided by the present invention is applied to a video data output control module of the storage application chip, comprising: an on-chip cache, a system status detection module, an off-chip storage interface detection module, a read cache prediction module and a dynamic frame supplement control module; the read cache prediction module is used to determine the subsequent read cache situation according to the first frequency detected by the system status detection module and the second frequency detected by the off-chip storage interface detection module when the on-chip cache is full; wherein the first frequency is the frequency of the read and write operation commands initiated by the interface connected to the system bus by the central processing unit in the storage application chip, the second frequency is the read and write frequency of all interfaces on the on-chip storage controller in the storage application chip, and the subsequent read cache situation includes a read state and a non-read state; the dynamic frame supplement control module is used to predict the predicted video data block according to each target video data block stored in the on-chip cache when the subsequent read cache situation is a read state, so as to write the predicted video data block into the on-chip cache when the on-chip cache is not full; wherein the target video data block is a video data block adjacent to the predicted video data block in the current video frame data.

[0053] It can be seen that the present invention, through the read cache prediction module added to the video data output control module, can determine whether to perform dynamic frame interpolation processing on the video frame being written according to the detection results output by the system status detection module and the off-chip storage interface detection module when the on-chip cache is full; thus, when determining to perform dynamic frame interpolation processing, the dynamic frame interpolation control module dynamically interpolates the video frame that cannot be written due to the full on-chip cache, thereby improving the complete writing rate of the video frame data, reducing the frame loss rate of the video function of the storage application chip, reducing the processing process of invalid data, improving the utilization rate of the video function of the system bus resources and storage bandwidth, and improving the performance of the storage application chip. In addition, the present invention also provides a storage application chip and server, which also have the above-mentioned beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0055] Figure 1 A schematic diagram of a remote video transmission solution of a BMC in a server in the related art;

[0056] Figure 2 A structural block diagram of a video function system of a storage application chip provided by an embodiment of the present invention;

[0057] Figure 3 A schematic diagram of a method for dividing a first preset time period provided in an embodiment of the present invention;

[0058] Figure 4 A schematic diagram of the structure of a video function system of a storage application chip provided by an embodiment of the present invention;

[0059] Figure 5 A schematic diagram of a method for dividing a second preset time period provided in an embodiment of the present invention;

[0060] Figure 6 A schematic diagram of a frame interpolation function of current video frame data in a preset compression format provided by an embodiment of the present invention;

[0061] Figure 7 A schematic diagram of a frame interpolation function of current video frame data in a preset video data format provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] Please refer to Figure 2 , Figure 2 This is a structural block diagram of a video function system of a storage application chip provided by an embodiment of the present invention. The system is applied to a video data output control module of a storage application chip, and may include: a system status detection module 10, an off-chip storage interface detection module 20, a read cache prediction module 30, a dynamic frame supplement control module 40, and an on-chip cache 50;

[0064] The read cache prediction module 30 is used to determine the subsequent read cache status when the on-chip cache 50 is full, according to the first frequency detected by the system status detection module 10 and the second frequency detected by the off-chip storage interface detection module 20; wherein the first frequency is the frequency of the read and write operation commands initiated by the interface connected to the system bus by the central processing unit in the storage application chip, the second frequency is the read and write frequency of all interfaces on the on-chip storage controller in the storage application chip, and the subsequent read cache status includes a read status and a non-read status;

[0065] The dynamic frame supplement control module 40 is used to predict the predicted video data block according to each target video data block stored in the on-chip cache 50 when the subsequent read cache is in the read state, so as to write the predicted video data block into the on-chip cache 50 when the on-chip cache 50 is not full; wherein the target video data block is a video data block adjacent to the predicted video data block in the current video frame data.

[0066] It can be understood that the storage application chip in this embodiment can be a SoC chip in a storage application scenario, such as a server management controller (such as BMC) in a server of a storage system, or an SSD (solid state drive) controller. Among them, the SoC chip is a single chip that integrates one or more of a microprocessor, an analog IP (Intellectual Property) core, a digital IP core, and a memory (or an off-chip storage control interface). The video data output control module of the storage application chip in this embodiment can be a hardware module in the storage application chip for writing received video data (such as compressed data or YUV format data) to the on-chip cache 50, and reading the video data in the on-chip cache 50 and converting it into data of the corresponding interface protocol to output to the subsequent hardware module in the storage application chip or an external device (such as a memory DDR). For example, when the storage application chip is a BMC, Figure 1 As shown, the video data output control module in the BMC can write the input video data (such as compressed data or YUV format data) into the external video memory (such as DDR) via the on-chip cache 50 .

[0067] It can be understood that the on-chip cache write control module in the video data output control module of the BMC in the related art is responsible for receiving the video data input by the previous stage (such as video data in the Joint Photographic Experts Group JPEG compression format or video data in the YUV format), and then writing it into the on-chip cache 50. In order to ensure that the data written into the on-chip cache 50 is correct, the module needs to ensure that the writing is stopped when the on-chip cache 50 is full. Therefore, there are two write states for a frame of video data (i.e., video frame data). One is that all video data of a frame are written into the on-chip cache 50, recorded as the state frame_wr_state=0, and the other is that the video data of a frame cannot be completely written into the on-chip cache 50 because the on-chip cache 50 is full, recorded as the state frame_wr_state=1; the on-chip cache read control module in the video data output control module can be responsible for reading the video data in the on-chip cache 50 and converting the read video data into the AXI (Advanced eXtensible Interface, a bus protocol) protocol format through the interface protocol conversion module and writing it into the DDR. Because the write status of the video frame data (frame_wr_state=0 / 1) is only obtained through the frame write status statistics module when the last data of the current video frame data is written, and the previous data has been written to DDR at this time, if frame_wr_state=1, the written data is invalid data, because the software of the remote device cannot restore and display these data, and it has also caused a lot of waste of DDR memory bandwidth and invalid occupation of content capacity; at the same time, it often happens that after the on-chip cache 50 is full, the video output control module competes for the control of the memory bus, and can continue to The video data continues to be written, but at this time, the video frame data currently being written has been subjected to frame loss processing, that is, the subsequent video data of the current video frame has stopped being written into the on-chip cache 50, and at the same time, the video data of the current video frame that has been written into the on-chip cache 50 is invalid data, because although it is written into the on-chip cache 50, this data is not a complete frame of data, and the remote device cannot recover it. However, at the same time, this data needs to be read out from the on-chip cache 50 by the internal logic of the video data output control module and written into the external DDR, resulting in a high frame loss rate of the video function of the storage application chip in the related technology, and a large amount of invalid data is written.

[0068] Correspondingly, in this embodiment, by adding the read cache prediction module 30 as a hardware module in the video data output control module, the first frequency detected by the system status detection module 10 and the off-chip storage interface detection module 20 (such as Figure 4The second frequency detected by the DDR interface detection module in the memory is used to predict the subsequent state (i.e., the subsequent read cache status) of the on-chip cache 50 (such as a receiving first-in first-out queue FIFO or a random access memory RAM or other storage types) when it is full, so as to determine whether to perform dynamic frame supplement processing on the video frame data being written. Among them, the first frequency detected by the system status detection module 10 may be the frequency (Freq_sys) of the read and write operation commands initiated by the interface connected to the system bus by the CPU in the storage application chip. The frequency may represent the status of all programs and systems allowed on the storage application chip. The higher the frequency, the more frequently the operating system and software programs on the chip process data, and the greater the demand for the bus and external memory. The second frequency detected by the off-chip storage interface detection module 20 may be the read and write frequency (Freq_DDR) of all interfaces on the on-chip storage controller in the storage application chip. That is, since many modules in the storage application chip need to access external memory (such as DDR), there are multiple interfaces on the corresponding storage controller in the storage application chip (that is, the on-chip storage controller, such as the DDR controller). The off-chip storage interface detection module 20 may determine the use rights of each hardware module mounted on the storage controller during bus arbitration by detecting the read and write frequencies of these interfaces. For example, the higher the second frequency, the more frequently each hardware module accesses the external memory, and the smaller the use rights of each hardware module.

[0069] Correspondingly, when the on-chip cache 50 is full, the read cache prediction module 30 determines the specific method of the subsequent read cache situation according to the first frequency detected by the system status detection module 10 and the second frequency detected by the off-chip storage interface detection module 20. The specific method can be set by the designer according to the practical scenario and user needs. For example, in order to improve the prediction accuracy of the subsequent read cache situation, Figure 3 As shown, the read cache prediction module 30 can divide the first detection time period before the current moment (i.e., when the on-chip cache 50 is full), and divide the first detection time period into N first preset time periods, so that the read cache prediction module 30 can determine the subsequent read cache situation according to the first frequency and the second frequency corresponding to each first preset time period in the first detection time period when the on-chip cache 50 is full; wherein the number of first preset time periods is greater than or equal to 2, i.e., N is greater than or equal to 2; the sum of all first preset time periods is the first detection time period. Correspondingly, the read cache prediction module 30 can also determine the subsequent read cache situation directly according to the first frequency and the second frequency in the first detection time period when the on-chip cache 50 is full, such as by calculating the sum of the products of the first frequency and the second frequency in the first detection time period and the corresponding weight parameters and comparing it with the threshold value to determine the subsequent read cache situation.

[0070] For example, when the on-chip cache 50 is full, the read cache prediction module 30 can determine the subsequent read cache situation according to the first frequency and the second frequency corresponding to each first preset time period within the first detection time period, and their corresponding preset parameters and the first preset threshold. For example, when the on-chip cache 50 is full, the read cache prediction module 30 can determine the subsequent read cache situation by comparing the sum of the products of the first frequency and the second frequency corresponding to each first preset time period within the first detection time period and their corresponding preset parameters with the first preset threshold. For example, when the on-chip cache 50 is full, if PARAM_0 * (P_0 * Freq_sys0 + P_1 * Freq_sys1 + P_2 * Freq_sys2 +... + P_N-1 * Freq_sysN-1) + PARAM_1 * (Q_0 * Freq_DDR0 + Q_1 * Freq_DDR1 + Q_2 * Freq_DDR2 +... + Q_N-1 * Freq_DDRN-1) < THREHOLD_0, it is determined that the subsequent read cache situation is the read state (such as predict_result = 0); if PARAM_0 * (P_0 * Freq_sys0 + P_1 * Freq_sys1 + P_2 * Freq_sys2 +... + P_N-1 * Freq_sysN-1) + PARAM_1 * (Q_0 * Freq_DDR0 + Q_1 * Freq_DDR1 + Q_2 * Freq_DDR2 +... + Q_N-1 * Freq_DDRN-1) ≥ THREHOLD_0, it is determined that the subsequent read cache situation is the non-read state (such as predict_result = 1) to discard the current video frame data. Wherein, N is the number of the first preset time periods, PARAM_0 and P_0 to P_N-1 are the preset parameters corresponding to the first frequency respectively, PARAM_1 and Q_0 to Q_N-1 are the preset parameters corresponding to the second frequency respectively, Freq_sys0 to Freq_sysN-1 are the first frequencies corresponding to the 1st to Nth first preset time periods respectively, Freq_DDR0 to Freq_DDRN-1 are the second frequencies corresponding to the 1st to Nth first preset time periods respectively, and THREHOLD_0 is the first preset threshold; PARAM_1 > PARAM_0, P_0 > P_1 > P_2... > P_N-1, Q_0 > Q_1 > Q_2... > Q_N-1.

[0071] That is to say, the first detection time period (T1) before the moment when the on-chip cache 50 is full (i.e., the current moment) is divided into N segments, and N first preset time periods can be obtained. The closer a first preset time period is to the current moment, the greater the calculation proportion of its corresponding frequency (first frequency or second frequency), and the farther away from the current moment, the lower the calculation proportion of its corresponding frequency (first frequency or second frequency), because the closer the bus and memory states are to the current moment (i.e., the moment when the on-chip cache 50 is full), the closer they are to the states after the current moment, and the farther away from the current moment, the greater the difference between the bus and memory states and the states after the current moment.

[0072] like Figure 3 As shown, the first detection time period before the current moment can be evenly divided into N sections, that is, the time of the N first preset time periods can be the same; wherein, the value of the first detection time period and N can be configured by the user through registers, or the default value can be used. The preset parameter corresponding to the first first frequency (Freq_sys0) can be P_0, the preset parameter corresponding to the second first frequency (Freq_sys1) can be P_1... The preset parameter corresponding to the Nth first frequency (Freq_sysN-1) can be P_N-1, and the size relationship is P_0>P_1>P_2...>P_N-1; the preset parameter corresponding to the first second frequency (Freq_DDR0) is Q_0, and the preset parameter corresponding to the second second frequency (Freq_DDR1) is Q_1. It is Q_1...The parameter corresponding to the Nth second frequency (Freq_DDRN-1) is Q_N-1, and their size relationship is Q_0>Q_1>Q_2...>Q_N-1; at the same time, Q_0>P_0, Q_1>P_1...Q_N-1>P_N-1, that is, Q_i>P_i, i is an integer greater than or equal to 0 and less than N. Because the second frequency has a greater impact, PARAM_1>PARAM_0, that is, the preset parameter corresponding to the first frequency can be smaller than the preset parameter corresponding to the second frequency.

[0073] Correspondingly, THREHOLD_0 is the first preset threshold (i.e., the threshold used for judgment), which can be configured by the user through a register, or a default value can be used. Moreover, in the present embodiment, the prediction mechanism of the read cache prediction module 30 can be dynamically adjusted. When the predicted subsequent read cache situation is a no-read state, the current video frame data can be discarded as in the related art; further, the read cache prediction module 30 can also be used to detect the data read status of the on-chip cache 50 after determining that the subsequent read cache situation is a no-read state, and determine whether the subsequent read cache situation is correct based on the data read status; if not, PARAM_1, PARAM_0, P_0 to P_N-1 and Q_0 to Q_N-1 are reduced; if correct, PARAM_1, PARAM_0, P_0 to P_N-1 and Q_0 to Q_N-1 are not adjusted. For example, the read cache prediction module 30 can synchronously detect the data read status of the on-chip cache 50 after the current moment when the predicted subsequent read cache situation is a no-read state, such as Figure 4 The on-chip cache read control module reads the data in the on-chip cache 50 and simulates the write side behavior, such as simulating the continued parsing of the current video frame data to determine whether the current video frame data can be completely written into the on-chip cache 50 at a subsequent time of the current moment; if it can be completely written into the on-chip cache 50, it proves that the subsequent judgment of the cache read situation is wrong. At this time, the preset parameters corresponding to the first frequency and the second frequency in the above formula can be lowered, that is, the parameters of PARAM_0, PARAM_1, P_0...P_N-1, Q_0...Q_N-1 can be lowered to improve the preset Regarding the amplitude of the reduction, the amplitude of the reduction of the preset parameters (such as PARAM_1 and Q_0…Q_N-1) corresponding to the second frequency (Freq_DDR) can be greater than the amplitude of the reduction of the preset parameters (such as PARAM_0 and P_0…P_N-1) corresponding to the first frequency (Freq_sys); that is, in the process of reducing PARAM_1, PARAM_0, P_0 to P_N-1 and Q_0 to Q_N-1, the amplitude of the reduction of the preset parameters corresponding to the first frequency is less than the amplitude of the reduction of the preset parameters corresponding to the second frequency.

[0074] Further, in the present embodiment, the subsequent read cache situation determined by the read cache prediction module 30 may also include an invalid frame state (such as predict_result=2) to indicate whether a new frame of video data to be input into the on-chip cache 50 can be completely input; for example, the read cache prediction module 30 may be used to determine the subsequent read cache situation according to the first frequency and the second frequency corresponding to each first preset time period in the first detection time period and the first frequency and the second frequency corresponding to each second preset time period in the second detection time period; the system provided by the present embodiment may also include a pre-loss frame control module, which is used to control the previous hardware module (such as the pre-loss frame control module) corresponding to the video data output control module in the storage application chip when the subsequent read cache situation is an invalid frame state. Figure 1 The data capture module, color space conversion module and core compression module in the video processing module stop processing the current video frame data.

[0075] For example, when the on-chip cache 50 is full, the read cache prediction module 30 can determine the subsequent read cache status by comparing the sum of the products of the first frequency and the second frequency corresponding to each second preset time period in the second detection time period and the corresponding preset parameter with the second preset threshold, and by comparing the sum of the products of the first frequency and the second frequency corresponding to each first preset time period in the first detection time period and the corresponding preset parameter with the first preset threshold. For example, when the on-chip cache 50 is full, if PARAM_2*(R_0*Freq_sys0+R_1*Freq_sys1+R_2*Freq_sys2+…R_M-1*Freq_sysM-1)+PARAM_3*(S_0*Freq_DDR0+S_1*Freq_DDR1+S_2*Freq_DDR2+…S_M-1*Freq_DDRM-1)>THREHOLD_1, it is determined that the subsequent read cache situation is an invalid frame state; wherein M is the number of the second preset time period, and PARAM_2 and R_0 to R_M-1 are respectively The preset parameters corresponding to the first frequency, PARAM_3 and S_0 to S_M-1 are respectively the preset parameters corresponding to the second frequency, Freq_sys0 to Freq_sysM-1 are the first frequencies corresponding to the 1st first preset time period to the Mth first preset time period, Freq_DDR0 to Freq_DDRM-1 are the second frequencies corresponding to the 1st first preset time period to the Mth first preset time period, THREHOLD_1 is the second preset threshold; PARAM_3>PARAM_2, R_0>R_1>R_2……>R_M-1, S_0>S_1>S_2……>S_M-1.

[0076] That is to say, the second detection time period (T2) before the moment when the on-chip cache 50 is full (i.e., the current moment) is divided into M segments, and M second preset time periods can be obtained; the second detection time period is used to predict whether a new frame of video data to be input into the on-chip cache 50 can be completely input into the on-chip cache 50, and the second detection time period can be greater than the first detection time period. The closer a second preset time period is to the current moment, the greater the calculation proportion of its corresponding frequency (first frequency or second frequency), and the farther it is from the current moment, the lower the calculation proportion of its corresponding frequency (first frequency or second frequency), because the closer the bus and memory state is to the current moment (i.e., the moment when the on-chip cache 50 is full), the closer it is to the state after the current moment, and the farther it is from the current moment, the greater the difference between the bus and memory state and the state after the current moment.

[0077] like Figure 5 As shown, the second detection time period before the current moment can be evenly divided into M segments, that is, the time of the M second preset time periods can be the same; wherein, the second detection time period and the value of M can be configured by the user through registers, or the default values ​​can be used, such as the second detection time period (or the second preset time period) can use the average value of the time used to complete the input of multiple frames of images (i.e., video frame data). The preset parameter corresponding to the first first frequency (Freq_sys0) in the second detection time period can be R_0, the preset parameter corresponding to the second first frequency (Freq_sys1) can be R_1... The preset parameter corresponding to the Mth first frequency (Freq_sysM-1) can be R_M-1, and the size relationship is R_0>R_1>R_2...>R_M-1; the preset parameter corresponding to the first second frequency (Freq_DDR0) in the second detection time period is S_0, and the second second frequency (Freq_DDR 1) The corresponding preset parameter is S_1...The parameter corresponding to the Mth second frequency (Freq_DDRM-1) is S_M-1, and their size relationship is S_0>S_1>S_2...>S_M-1; at the same time, S_0>R_0, S_1>R_1...S_M-1>R_M-1, that is, S_j>R_j, j is an integer greater than or equal to 0 and less than M. Because the second frequency has a greater impact, PARAM_3>PARAM_2, that is, the preset parameter corresponding to the first frequency can be smaller than the preset parameter corresponding to the second frequency.

[0078] Correspondingly, THREHOLD_1 is a second preset threshold (i.e., a threshold for judgment), which can be configured by the user through a register, or a default value can be used. For example, when the on-chip cache 50 is full, the read cache prediction module 30 can compare the sum of the products of the first frequency and the second frequency corresponding to each second preset time period in the second detection time period and the corresponding preset parameter with the second preset threshold, and when the subsequent read cache situation is not an invalid frame state, the sum of the products of the first frequency and the second frequency corresponding to each first preset time period in the first detection time period and the corresponding preset parameter with the first preset threshold is compared to determine whether the subsequent read cache situation is a read state or a non-read state.

[0079] At the same time, in this embodiment, the prediction mechanism of the read cache prediction module 30 is dynamically adjusted. When the predicted subsequent read cache situation is an invalid frame state, the current video frame data can be discarded, that is, the pre-frame loss function is realized, and the current video frame data to be written is judged as an invalid frame. Furthermore, the read cache prediction module 30 (or pre-frame loss control module) can also be used to detect the data read status of the on-chip cache 50 after determining that the subsequent read cache situation is an invalid frame state, and determine whether the subsequent read cache situation is correct according to the data read status; if not, PARAM_2, PARAM_3, R_0 to R_M-1 and S_0 to S_M-1 are reduced; if correct, PARAM_2, PARAM_3, R_0 to R_M-1 and S_0 to S_M-1 are not adjusted. For example, when the predicted subsequent read cache situation is an invalid frame state, the read cache prediction module 30 can synchronously detect the data read status of the on-chip cache 50 after the current moment, such as Figure 4 The on-chip cache read control module reads the data in the on-chip cache 50 and simulates the write side behavior, such as simulating the continued writing of the current frame to determine whether the current video frame data can be completely written into the on-chip cache 50 at the subsequent time of the current moment; if it can be completely written into the on-chip cache 50, it proves that the subsequent judgment of the cache read situation is wrong. At this time, the preset parameters corresponding to the first frequency and the second frequency in the above formula can be lowered, that is, the parameters of PARAM_2, PARAM_3, R_0...R_M-1, S_0...S_M-1 can be lowered to improve the predicted Correctness; Regarding the amplitude of the reduction, the amplitude of the reduction of the preset parameters corresponding to the second frequency (Freq_DDR) (such as PARAM_3 and S_0...S_M-1) can be greater than the amplitude of the reduction of the preset parameters corresponding to the first frequency (Freq_sys) (such as PARAM_2 and R_0...R_M-1); that is, in the process of reducing PARAM_2, PARAM_3, R_0 to R_M-1 and S_0 to S_M-1, the amplitude of the reduction of the preset parameters corresponding to the first frequency is smaller than the amplitude of the reduction of the preset parameters corresponding to the second frequency.

[0080] Correspondingly, for the current video frame data judged as an invalid frame, Figure 1 The data capture module in the processor stops capturing invalid frames and subsequent color space conversion, video compression and other processing at the beginning, so as to avoid invalid processing of video data to the greatest extent; that is to say, after determining that the frame is invalid, the current video frame data is not written into the on-chip cache 50, nor is it written into the external DDR, so the processing of the invalid frame by the previous stage is meaningless, thereby reducing the power consumption of the chip, increasing the service life of the chip, and minimizing the occupancy of the memory bandwidth and the system bus by invalid data.

[0081] In some embodiments, Figure 4 As shown, the system provided by this embodiment may also include a frame information parsing module, a frame commonality information control module, a frame basic information control module, a compression algorithm table control module, a brightness information quantization table control module, a chrominance information quantization table control module, an on-chip cache write control module and an on-chip cache read control module; the frame information parsing module is used to parse the received current video frame data in a preset compression format to obtain the frame commonality information, frame basic information, compression algorithm table information, brightness information quantization table information, chrominance information quantization table information and compressed data blocks of the current video frame data; wherein the compressed data block is compressed data in the current video frame data in blocks, and the preset compression format is an intra-frame compression format; the frame commonality information control module is used to update the frame commonality information stored in the current video frame data; the frame basic information control module is used to update the frame basic information stored in the current video frame data; the compression algorithm table control module (such as Figure 4 a Huffman table control module in the video frame), used to update the compression algorithm table information storing the current video frame data; a brightness information quantization table control module, used to update the brightness information quantization table information storing the current video frame data; a chrominance information quantization table control module, used to update the brightness information quantization table information storing the current video frame data; an on-chip cache write control module, used to store the compressed data blocks of the current video frame data in the on-chip cache 50 in sequence in units of blocks; an on-chip cache read control module, used to restore the compressed data blocks read from the on-chip cache 50 to data in a preset compression format according to the data stored in the frame common information control module, the frame basic information control module, the compression algorithm table control module, the brightness information quantization table control module and the chrominance information quantization table control module.

[0082] That is to say, for video data in intra-frame compression format such as JPEG compression format, parsing the structure of a single frame image means parsing the structure of a video composed of continuous frames of images. Therefore, the frame information parsing module can parse the current video frame data in a preset compression format and parse out six parts: frame commonality information, frame basic information, compression algorithm table information (such as Huffman table information in JPEG compression format), brightness information quantization table information, chrominance information quantization information table and compressed data block.

[0083] Among them, the current video frame data (i.e., compressed image) in an intra-frame compression format such as JPEG compression format may include SOI (file header), APP0 (image identification information), DQT (defined quantization table), SOF0 (basic image information), DHT (defined compression algorithm table, such as Huffman table), DRI (defined restart interval), SOS (start of scan line) and EOI (end of file) in sequence.

[0084] Correspondingly, the frame common information parsed by the frame information parsing module may include SOI, APP0, DRI and EOI. The parsing method is based on the specific indication information fields corresponding to the above parts, such as the indication information field of SOI is FF D8, the indication information field of APP0 is FF E0, the indication information field of DRI is FF DD, and the indication information field of EOI is FF D9. This frame common information is common to the preset compression format (such as JPEG compression format), and each frame of the compressed image in the preset compression format is the same. After the above common information is parsed, it can be input into Figure 4 The frame common information control module in the control module is saved and updated when changes occur.

[0085] The compression algorithm table (such as the Huffman table) information can also be obtained by parsing the corresponding specific indication information. For example, when the preset compression format is the JPEG compression format, the indication information field of the Huffman table information is FF C4. Figure 1 When the core compression module of the JPEG format video data is compressed, it is possible to use different Huffman tables. Because the Huffman table information can be parsed separately, the parsed information can be input into Figure 4 The Huffman table control module in .

[0086] Correspondingly, the basic frame information may include SOF0, the indication information field of SOF0 is FF C0, which may include resolution information, quantization table number, sample accuracy and YUV format and other information. This information may be different in different video frame data. For example, if the resolution changes and the YUV format changes (such as common YUV formats such as YUV444 / YUV422 / YUV420), the parsed information is input into the basic frame information control module in 4.

[0087] Correspondingly, the brightness information quantization table information may include SOF0, and the SOF0 indication information field is FF DB, which is the quantization table used when the brightness component is quantized during JPEG compression, because Figure 1 When the core compression module in the JPEG format is compressed, different brightness quantization tables may be used. Therefore, the brightness quantization table information needs to be parsed separately and the parsed information is input into Figure 4 The brightness information quantization table control module in.

[0088] The chrominance information quantization table may include DQT, which can be divided into two parts, first the brightness information quantization table information, and then the chrominance information quantization table information. The indication information field of this part of the information is also FF DB, which is the quantization table used when quantizing the chrominance component during JPEG compression, because Figure 1 When the core compression module in the JPEG format is compressed, different colorimetric quantization tables may be used. Therefore, the colorimetric quantization table information needs to be parsed separately and the parsed information is input into Figure 4 The chrominance information quantization table control module in.

[0089] In addition to detecting the above-mentioned information indication field and writing the corresponding information into different control modules, the function of the frame information parsing module also includes detecting whether the above-mentioned information has changed in units of frames. If it has changed, the information data in the corresponding control module is updated; if it has not changed, the data in the corresponding control module is not updated. For example, if the brightness quantization table used for compressing the 10th frame is different from the brightness quantization table used for compressing the 11th frame, when the compressed data of the 11th frame arrives, the content of the brightness information quantization table control module is updated, and the contents of other control modules remain unchanged.

[0090] When the frame information parsing module parses the SOS indication field (FF DA), it indicates that the following data is a compressed data block in units of BLOCK (block), and the compressed data block can be written into the on-chip cache 50 through the on-chip cache write control module. Therefore, in this embodiment, compressed data (i.e., compressed data blocks) in units of BLOCK are stored separately, while common information of video frame data, brightness quantization table, chrominance quantization table, Huffman table and other information are stored separately. Therefore, a mechanism is implemented in which different frame data share common information of video frame data, brightness quantization table, chrominance quantization table and Huffman table information. On the read side of the on-chip cache 50, the on-chip cache read control module can recombine the compressed data blocks and restore them into complete JPEG format data, that is, obtain corresponding data from the frame common information control module, the frame basic information control module, the Huffman table control module, the brightness information quantization table control module, the chrominance information quantization table control module and the upper cache respectively, splice them into complete JPEG format video frame data for output, thereby avoiding the disadvantage of repeated writing of the same data such as common information, brightness quantization table, chrominance quantization table and Huffman table of compressed video data (such as multi-frame JPEG compressed images) into the on-chip cache 50 in the related art, reducing the time of compressed data in the on-chip cache 50, reducing the delay of compressed data, and reducing the frame loss rate of the video.

[0091] It should be noted that the dynamic interpolation control module 40 can start the dynamic interpolation function when the subsequent read cache status is in the read state. When the dynamic interpolation control module 40 is in the read state, the specific method of predicting the predicted video data block according to each target video data block stored in the on-chip cache 50 can be directly predicted according to a preset number of target video data blocks corresponding to the predicted video data block; for example, the predicted video data block can be predicted by directly using the four video data blocks (i.e., target video data blocks) above, to the left, to the upper left, and to the upper right of the predicted video data block in the current video frame data in the on-chip cache 50.

[0092] Accordingly, in order to reduce the amount of calculation for dynamic frame interpolation, the dynamic frame interpolation control module 40 can determine the target video data blocks stored in the on-chip cache 50 according to the image change of the current video frame data; and predict the predicted video data blocks according to the target video data blocks. In other words, the dynamic frame interpolation control module 40 selects different target video data blocks for prediction according to the image change of the current video frame data (i.e., the size of the image change within one frame of image).

[0093] For example, the above process of determining each target video data block stored in the on-chip cache 50 according to the image change of the current video frame data may include: if the image change of the current video frame data is a change amount not greater than a threshold state, then the video data blocks above and to the left of the predicted video data block in the current video frame data in the on-chip cache 50 are respectively used as target video data blocks; if the image change of the current video frame data is a change amount greater than a threshold state, then the video data blocks above, to the left, upper left and upper right of the predicted video data block in the current video frame data in the on-chip cache 50 are respectively used as target video data blocks.

[0094] Correspondingly, the process of predicting the predicted video data block according to each target video data block may include: when the number of target video data blocks is 2 (such as the above-mentioned change amount is not greater than the threshold state), the predicted video data block is calculated by BLOCK_PREDICT=A_0*BLOCK_0+A_1*BLOCK_2; wherein BLOCK_PREDICT is the predicted video data block, BLOCK_0 and BLOCK_2 are two target video data blocks, A_0 and A_1 are preset complementary blocks, and BLOCK_PREDICT is the predicted video data block. Frame parameters, A_0+A_1=1; when the number of target video data blocks is 4 (such as the above-mentioned change is greater than the threshold state), the predicted video data block is calculated by BLOCK_PREDICT=B_0*BLOCK_0+B_1*BLOCK_1+B_2*BLOCK_2+B_3*BLOCK_3; wherein, BLOCK_0 to BLOCK_3 are 4 target video data blocks, B_0 to B_3 are preset frame supplementation parameters, B_0+B_1+B_2+B_3=1.

[0095] For example, when the current video frame data is in a preset compression format (such as JPEG compression format), Figure 4 As shown, the dynamic frame interpolation control module 40 can use the compressed data blocks in the on-chip cache 50 in units of BLOCKs to predict the compressed data blocks that cannot be written to the current video frame due to the full state of the on-chip cache 50 when the subsequent read cache state is in the read state; at the same time, the dynamic frame interpolation control module 40 can select different compressed data blocks as target video data blocks for prediction according to the image change frequency in the original video frame data transmitted by the server host (i.e., the image change state of the current video frame data); Figure 6As shown, BLOCK_PREDICT may be a compressed data block that is currently being written into the on-chip cache 50 but the on-chip cache 50 is in a full state, and BLOCK_0, BLOCK_1, BLOCK_2 and BLOCK_3 are compressed data blocks adjacent to BLOCK_PREDICT that have been written into the on-chip cache 50. When the image change in the original video frame data is small (such as the change frequency is not greater than the change frequency threshold), it can be determined that the image change of the current video frame data is not greater than the threshold state, and BLOCK_0 and BLOCK_2 can be used to predict the data of BLOCK_PREDICT, that is, BLOCK_0 on the left side of BLOCK_PREDICT and BLOCK_2 above BLOCK_PREDICT in the current video frame data in the on-chip cache 50 are used as target video data blocks for prediction, such as BLOCK_PREDICT=A_0*BLOCK_0+A_1*BLOCK_2; wherein, A_0+A_1=1; that is, when the image change in the frame is very small , a good prediction can be completed with a small number of target video data blocks, reducing the calculation amount of prediction frame interpolation; when the image changes greatly in the original video frame data (such as the change frequency is greater than the change frequency threshold), it can be determined that the image change of the current video frame data is greater than the threshold state, and BLOCK_1 to BLOCK_3 can be used to predict the data of BLOCK_PREDICT, that is, BLOCK_0 on the left side of BLOCK_PREDICT in the current video frame data in the on-chip cache 50, BLOCK_2 above, BLOCK_1 on the upper left, and BLOCK_3 on the upper right are used as target video data blocks for prediction, such as BLOCK_PREDICT=B_0*BLOCK_0+B_1*BLOCK_1+B_2*BLOCK_2+B_3*BLOCK_3; wherein, B_0+B_1+B_2+B_3=1; that is, when the image changes greatly in the frame, more target video data blocks can be used for prediction to improve the accuracy of prediction frame interpolation.

[0096] After the dynamic frame interpolation control module 40 completes obtaining the predicted data of BLOCK_PREDICT (i.e., the predicted video data block), the on-chip cache 50 is in a full state. When it is not full next, the predicted data can be written into the on-chip cache 50; at the same time, it can be updated synchronously according to the position relationship Figure 6BLOCK_PREDICT data in. Correspondingly, in this embodiment, the dynamic adjustment function (such as A_0, A_1, B_0 to B_3 in the above formula) of the preset interpolation frame parameters (such as A_0, A_1, B_0 to B_3 in the above formula) can also be realized; for example, the dynamic interpolation frame control module 40 is specifically used to predict the predicted video data block according to each target video data block and the preset interpolation frame parameters corresponding to each target video data block when the subsequent read cache is in the read state. The video function system provided by this embodiment can also include: an interpolation frame parameter adjustment and correction module, which is used to predict the predicted video data block corresponding to the current video data block to be written according to each target video data block and the preset interpolation frame parameters corresponding to the current video data block to be written stored in the on-chip cache 50 at a preset time interval when the on-chip cache 50 is not full; and adjust the preset interpolation frame parameters according to the comparison between the current video data block to be written and the corresponding predicted video data block.

[0097] like Figure 4 As shown, the read cache prediction module 30 can start the frame interpolation function of the dynamic frame interpolation control module 40 when the on-chip cache 50 is not completely full. At this time, the on-chip cache 50 can continue to receive the real data of the current compressed data block (i.e., the current video data block to be written), and the predicted data of the current compressed data block (i.e., the predicted video data block) can also be obtained at this time. The real data and the predicted data are compared, and then the corresponding preset frame interpolation parameters (such as A_0 and A_1 or B_0 to B_3) can be adjusted to make the predicted data equal to (or close to) the real data, thereby realizing the dynamic adjustment and correction function of the preset frame interpolation parameters to improve the accuracy of frame interpolation. For example, a fixed interval time (i.e., preset time) can be set to dynamically adjust the parameters of the frame interpolation function. The above-mentioned dynamic adjustment and correction function of the preset frame interpolation parameters can be performed by Figure 4 The frame supplement parameter adjustment correction module in the on-chip buffer 50 is completed; the transmission of the compressed data block as the target video data block can be performed by Figure 4 The compressed block information cache control module in is completed, and the calculation of BLOCK_PREDICT is done by Figure 4 The dynamic frame interpolation control module 40 is completed.

[0098] Accordingly, the frame interpolation process when the current video frame data is data in a preset video data format (such as YUV format) is similar to the frame interpolation process of the data in the preset compression format, except that the compressed data block is replaced with data in the preset video data format. Figure 7As shown, YUV_PREDICT is the YUV data (i.e., video data block) to be supplemented, and its length can be configured by the user register or can adopt the default value, such as 8 or 16, etc. YUV_0, YUV_1, YUV_2 and YUV_3 are the YUV data adjacent to YUV_PREDICT that have been written to the on-chip cache 50. Similarly, when the image change in the original video frame data is small, it can be determined that the image change of the current video frame data is not greater than the threshold state, and YUV_0 and YUV_2 can be used to predict the YUV_PREDICT data, such as YUV_PREDICT=C_0*YUV_0+C_1*YUV_2; where C_0+C_1=1; and when the image change in the original video frame data is large, it can be determined that the image change of the current video frame data is greater than the threshold state, and YUV_0 to YUV_3 can be used to predict the YUV_PREDICT data, such as YUV_PREDICT=D_0*YUV_0+D_1*YUV_1+D_2*YUV_2+D_3*YUV_3; where D_0+D_1+D_2+D_3=1. At the same time, the frame interpolation parameter adjustment and correction module can be used to complete the dynamic adjustment and correction of the preset frame interpolation parameters (such as C_0, C_1, D_0 to D_3 in the above formula), and the process is similar to the processing process under the preset compression format, which will not be repeated here. The dynamic adjustment and correction function of the preset frame interpolation parameters under the above preset video data format can be performed by Figure 4 The frame interpolation parameter adjustment and correction module in the chip is completed; the transmission of the YUV data as the target video data block in the on-chip buffer 50 can be performed by Figure 4 The YUV information cache control module (i.e., non-compressed block information cache control module) is used to complete the calculation of YUV_PREDICT. Figure 4 The dynamic frame interpolation control module 40 is completed.

[0099] That is to say, the system provided by this embodiment may further include a compressed block information cache control module and / or a non-compressed block information cache control module; the compressed block information cache control module is used to transfer each target video data block stored in the on-chip cache 50 to the dynamic frame interpolation control module 40 when the current video frame data received by the video data output control module is data in a preset compression format; the compressed block information cache control module is used to transfer each target video data block stored in the on-chip cache 50 to the dynamic frame interpolation control module 40 when the current video frame data received by the video data output control module is data in a preset video data format; wherein the preset interpolation parameters corresponding to each target video data block in the preset compression format (such as A_0 and A_1 or B_0 to B_3 mentioned above) are different from the preset interpolation parameters corresponding to each target video data block in the preset video data format (such as C_0 and C_1 or D_0 to D_3 mentioned above).

[0100] It should be noted that the present embodiment does not limit the specific application scenario of the storage application chip. For example, the storage application chip may be a BMC of a server. Accordingly, the storage application chip may also include Figure 1 The VGA (Video Graphics Array) module, video data capture module, color space conversion module, core compression module and network module are shown. For example, when the KVM (abbreviation of keyboard, video and mouse) function is started, the server host will need to transmit the original video data displayed remotely (i.e., the operating system remote display image, such as the real-time interface of the operating system) to the VGA module of the BMC through PCIe (a high-speed serial computer expansion bus standard). The VGA module generates the original video data in RGB format. This process needs to interact with the external DDR (i.e., video memory), such as Figure 1 The A area of ​​the video memory (such as DDR) in the video data capture module; the video data capture module obtains the original video data in RGB format from the VGA module and transmits it to the color space conversion module; the color space conversion module can perform color space conversion on the original video data in RGB format to obtain data in YUV format, and the conversion process can be adopted by the following formulas: Y=(0.257*R)+(0.504*G)+(0.098*B)+16, U=0.148*R–0.291*G+0.439*B+128 and V=0.439*R-0.368*G-0.071*B+128.

[0101] Correspondingly, the core compression module can perform video compression of the received YUV format data in a corresponding video compression format (such as H.264, JPEG, AVS and other video compression formats) to obtain compressed video data; the compressed video output control module can receive the compressed video data or the uncompressed YUV format video data and store it in an external DDR through the on-chip cache 50, such as Figure 1 The B area of ​​the video memory in the network module can send the data video output by the compressed video output control module in the video memory (such as DDR) to the remote device for display.

[0102] In this embodiment, the embodiment of the present invention can, by adding a read cache prediction module 30 to the video data output control module, determine whether to perform dynamic frame interpolation processing on the video frames being written when the on-chip cache 50 is full, based on the detection results output by the system status detection module 10 and the off-chip storage interface detection module 20; thereby, when it is determined that dynamic frame interpolation processing is to be performed, the dynamic frame interpolation control module 40 is used to dynamically interpolate the video frames that cannot be written further due to the on-chip cache 50 being full, thereby improving the complete write rate of the video frame data, reducing the frame loss rate of the video function of the storage application chip, reducing the processing process of invalid data, improving the utilization rate of the system bus resources and storage bandwidth of the video function, and improving the performance of the storage application chip.

[0103] Corresponding to the above system embodiment, an embodiment of the present invention further provides a storage application chip. The storage application chip described below and the video function system of the storage application chip described above can refer to each other.

[0104] A storage application chip includes: the video function system of the storage application chip provided by the above embodiment.

[0105] The storage application chip provided in this embodiment may specifically be a BMC.

[0106] Corresponding to the above system embodiment, the embodiment of the present invention further provides a server. The server described below and the storage application chip described above can refer to each other.

[0107] A server includes: the storage application chip provided in the above embodiment.

[0108] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the storage application chip and server disclosed in the embodiment, since they correspond to the system disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the system part description.

[0109] The video function system, storage application chip and server of a storage application chip provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principle and implementation mode of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A video function system for a storage application chip, characterized in that: A video data output control module applied to a storage application chip, including: an on-chip cache, a system status detection module, an off-chip storage interface detection module, a read cache prediction module, and a dynamic frame supplement control module; The read cache prediction module is used to determine the subsequent read cache status when the on-chip cache is full according to the first frequency detected by the system status detection module and the second frequency detected by the off-chip storage interface detection module; wherein the first frequency is the frequency of the read and write operation commands initiated by the interface connected to the system bus by the central processing unit in the storage application chip, the second frequency is the read and write frequency of all interfaces on the on-chip storage controller in the storage application chip, and the subsequent read cache status includes a read state and a non-read state; The dynamic frame supplement control module is used to predict a predicted video data block based on each target video data block stored in the on-chip cache when the subsequent read cache situation is the read state, so as to write the predicted video data block into the on-chip cache when the on-chip cache is not full; wherein the target video data block is a video data block adjacent to the predicted video data block in the current video frame data.

2. The video function system of the storage application chip according to claim 1, characterized in that: The read cache prediction module is specifically used to determine the subsequent read cache status when the on-chip cache is full, according to the first frequency and the second frequency corresponding to each first preset time period within the first detection time period; wherein the number of the first preset time periods is greater than or equal to 2, and the sum of all the first preset time periods is the first detection time period.

3. The video function system of the storage application chip according to claim 2, characterized in that: When the on-chip cache is full, determining the subsequent read cache status according to the first frequency and the second frequency corresponding to each first preset time period within the first detection time period, includes: When the on-chip cache is full, if PARAM_0 * (P_0 * Freq_sys0 + P_1 * Freq_sys1 + P_2 * Freq_sys2 + …… P_N-1 * Freq_sysN-1) + PARAM_1 * (Q_0 * Freq_DDR0 + Q_1 * Freq_DDR1 + Q_2 * Freq_DDR2 + …… Q_N-1 * Freq_DDRN-1) < THREHOLD_0, it is determined that the subsequent read cache situation is the read state; if PARAM_0 * (P_0 * Freq_sys0 + P_1 * Freq_sys1 + P_2 * Freq_sys2 + …… P_N-1 * Freq_sysN-1) + PARAM_1 * (Q_0 * Freq_DDR0 + Q_1 * Freq_DDR1 + Q_2 * Freq_DDR2 + …… Q_N-1 * Freq_DDRN-1) ≥ THREHOLD_0, it is determined that the subsequent read cache situation is the non-read state to discard the current video frame data; where N is the number of the first preset time periods, PARAM_0 and P_0 to P_N-1 are the preset parameters corresponding to the first frequency respectively, PARAM_1 and Q_0 to Q_N-1 are the preset parameters corresponding to the second frequency respectively, Freq_sys0 to Freq_sysN-1 are the first frequencies corresponding to the 1st to Nth first preset time periods respectively, Freq_DDR0 to Freq_DDRN-1 are the second frequencies corresponding to the 1st to Nth first preset time periods respectively, and THREHOLD_0 is the first preset threshold; P_0 > P_1 > P_2 …… > P_N-1, Q_0 > Q_1 > Q_2 …… > Q_N-1.

4. The video function system of the storage application chip according to claim 3, characterized in that: PARAM_1 > PARAM_0, Q_i > P_i, where i is an integer greater than or equal to 0 and less than N.

5. The video function system of the storage application chip according to claim 3, characterized in that: The read cache prediction module is further configured to, after determining that the subsequent read cache situation is the non-read state, detect the data read-out situation of the on-chip cache, and determine whether the determination of the subsequent read cache situation is correct according to the data read-out situation; if it is incorrect, then reduce PARAM_1, PARAM_0, P_0 to P_N-1, and Q_0 to Q_N-1.

6. The video function system of the storage application chip according to claim 5, characterized in that: During the process of reducing PARAM_1, PARAM_0, P_0 to P_N-1, and Q_0 to Q_N-1, the reduction amplitude of the preset parameter corresponding to the first frequency is less than the reduction amplitude of the preset parameter corresponding to the second frequency.

7. The video function system of the storage application chip according to claim 2, characterized in that: The subsequent read cache situation further includes an invalid frame state, and determining the subsequent read cache situation according to the first frequencies and second frequencies corresponding to the respective first preset time periods within the first detection time period includes: Determine the subsequent read cache status according to the first frequency and the second frequency corresponding to each of the first preset time periods in the first detection time period and the first frequency and the second frequency corresponding to each of the second preset time periods in the second detection time period; The video function system also includes: The pre-frame loss control module is used to control the previous hardware module corresponding to the video data output control module in the storage application chip to stop processing the current video frame data when the subsequent read cache situation is the invalid frame state.

8. The video function system of the storage application chip according to claim 7, characterized in that: The determining the subsequent read cache status according to the first frequency and the second frequency corresponding to each of the first preset time periods within the first detection time period and the first frequency and the second frequency corresponding to each of the second preset time periods within the second detection time period includes: If PARAM_2*(R_0*Freq_sys0+R_1*Freq_sys1+R_2*Freq_sys2+…R_M-1*Freq_sysM-1)+PARAM_3*(S_0*Freq_DDR0+S_1*Freq_DDR1+S_2*Freq_DDR2+…S_M-1*Freq_DDRM-1)>THREHOLD_1, it is determined that the subsequent read cache situation is the invalid frame state; wherein M is the number of the second preset time period, PARAM_2 and R_0 to R_M-1 are respectively are respectively the preset parameters corresponding to the first frequency, PARAM_3 and S_0 to S_M-1 are respectively the preset parameters corresponding to the second frequency, Freq_sys0 to Freq_sysM-1 are the first frequencies corresponding to the 1st first preset time period to the Mth first preset time period, Freq_DDR0 to Freq_DDRM-1 are the second frequencies corresponding to the 1st first preset time period to the Mth first preset time period, THREHOLD_1 is the second preset threshold; R_0>R_1>R_2……>R_M-1, S_0>S_1>S_2……>S_M-1.

9. The video function system of the storage application chip according to claim 8, characterized in that: PARAM_3>PARAM_2, S_j>R_j, j is an integer greater than or equal to 0 and less than M.

10. The video function system of the storage application chip according to claim 8, characterized in that: The read cache prediction module is also used to detect the data read status of the on-chip cache after determining that the subsequent read cache status is the invalid frame state, and determine whether the determination of the subsequent read cache status is correct based on the data read status; if incorrect, reduce PARAM_2, PARAM_3, R_0 to R_M-1 and S_0 to S_M-1.

11. The video function system of the storage application chip according to claim 1, characterized in that: The step of predicting the predicted video data blocks according to the target video data blocks stored in the on-chip cache comprises: Determining each of the target video data blocks stored in the on-chip cache according to image changes of the current video frame data; According to each of the target video data blocks, a predicted video data block is predicted.

12. The video function system of the storage application chip according to claim 11, characterized in that: The determining, according to the image change of the current video frame data, each of the target video data blocks stored in the on-chip cache comprises: If the image change of the current video frame data is in a state where the change amount is not greater than a threshold value, the video data blocks above and to the left of the predicted video data block in the current video frame data in the on-chip buffer are respectively used as the target video data blocks; If the image change of the current video frame data is greater than a threshold value, the video data blocks above, to the left, above left and above right of the predicted video data block in the current video frame data in the on-chip cache are respectively used as the target video data blocks.

13. The video function system of the storage application chip according to claim 11, characterized in that: The step of predicting a predicted video data block according to each of the target video data blocks comprises: When the number of the target video data blocks is 2, the predicted video data block is calculated by BLOCK_PREDICT=A_0*BLOCK_0+A_1*BLOCK_2; wherein BLOCK_PREDICT is the predicted video data block, BLOCK_0 and BLOCK_2 are two target video data blocks, A_0 and A_1 are preset interpolation frame parameters, and A_0+A_1=1; When the number of the target video data blocks is 4, the predicted video data block is calculated by BLOCK_PREDICT=B_0*BLOCK_0+B_1*BLOCK_1+B_2*BLOCK_2+B_3*BLOCK_3; wherein BLOCK_0 to BLOCK_3 are the 4 target video data blocks, B_0 to B_3 are preset frame supplementation parameters, and B_0+B_1+B_2+B_3=1.

14. The video function system of the storage application chip according to claim 1, characterized in that: The dynamic frame interpolation control module is specifically used to predict and obtain a predicted video data block according to each of the target video data blocks and the preset frame interpolation parameters corresponding to each of the target video data blocks when the subsequent read cache status is the read state; The video function system also includes: A frame interpolation parameter adjustment and correction module is used to predict, when the on-chip cache is not full, the predicted video data block corresponding to the current video data block to be written according to each target video data block corresponding to the current video data block to be written stored in the on-chip cache and the preset frame interpolation parameter at a preset time interval; The preset frame interpolation parameters are adjusted according to a comparison between the current video data block to be written and the corresponding predicted video data block.

15. The video function system of the storage application chip according to claim 14, characterized in that: Also includes: A compressed block information cache control module and / or a non-compressed block information cache control module; A compression block information cache control module, configured to transfer each target video data block stored in the on-chip cache to the dynamic frame supplement control module when the current video frame data received by the video data output control module is data in a preset compression format; a non-compressed block information cache control module, configured to transfer each target video data block stored in the on-chip cache to the dynamic frame supplement control module when the current video frame data received by the video data output control module is data in a preset video data format; The preset frame interpolation parameters corresponding to each target video data block in the preset compression format are different from the preset frame interpolation parameters corresponding to each target video data block in the preset video data format.

16. The video function system of the storage application chip according to any one of claims 1 to 15, characterized in that: When the video data received by the video data output control module includes data in a preset compression format, the video function system further includes: a frame information parsing module, a frame common information control module, a frame basic information control module, a compression algorithm table control module, a brightness information quantization table control module, a chrominance information quantization table control module, an on-chip cache write control module, and an on-chip cache read control module; The frame information parsing module is used to parse the received current video frame data in the preset compression format to obtain frame commonality information, frame basic information, compression algorithm table information, brightness information quantization table information, chrominance information quantization table information and compressed data blocks of the current video frame data; wherein the compressed data block is compressed data in the current video frame data in blocks, and the preset compression format is an intra-frame compression format; The frame commonality information control module is used to update and store the frame commonality information of the current video frame data; The frame basic information control module is used to update and store the frame basic information of the current video frame data; The compression algorithm table control module is used to update the compression algorithm table information storing the current video frame data; The brightness information quantization table control module is used to update the brightness information quantization table information storing the current video frame data; The chrominance information quantization table control module is used to update the brightness information quantization table information storing the current video frame data; The on-chip cache write control module is used to store the compressed data blocks of the current video frame data in the on-chip cache in sequence in units of blocks; The on-chip cache read control module is used to restore the compressed data block read from the on-chip cache to the data in the preset compression format according to the data stored in the frame common information control module, the frame basic information control module, the compression algorithm table control module, the brightness information quantization table control module and the chrominance information quantization table control module.

17. The video function system of the storage application chip according to claim 16, characterized in that: The preset compression format is the Joint Photographic Experts Group compression format, and the compression algorithm table control module is specifically a Huffman table control module, which is used to store Huffman table information of current video frame data.

18. The video function system of the storage application chip according to claim 1, characterized in that: The storage application chip is specifically a baseboard management controller.

19. A storage application chip, characterized in that: include: A video function system for a storage application chip as claimed in any one of claims 1 to 18.

20. A server, characterized in that: include: The storage application chip as claimed in claim 19.

Images