A video data processing method, product, device and storage medium

Abstract

The embodiments of the present application relate to the field of data processing technology, and specifically to a video data processing method, product, device and storage medium, which are intended to optimize the video data processing process. The method includes: receiving first video data in a compressed format; obtaining an estimated storage space occupancy of a target video frame in the first video data according to the actual storage space occupancy of the cached video frame and the compression time of the video frame; writing the target video frame into the parallel memory when the estimated storage space occupancy does not exceed the remaining capacity of the parallel memory; reading the video frame corresponding to the first video data stored in the parallel memory from the parallel memory when the number of frames of the video frame corresponding to the first video data stored in the parallel memory meets a preset transmission frame number threshold; and writing the video frame corresponding to the first video data into an external memory.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and specifically, to a video data processing method, product, device and storage medium. Background Art

[0002] When processing video data, it is generally processed by the video module integrated on the management control chip. First, the video data is cached frame by frame in the internal cache, and then the data is written to the external memory frame by frame. The video data is then obtained from the external memory through the corresponding driver and displayed.

[0003] In the related art, since the video module on the management and control chip shares an external memory with the operating system and application running on the chip, there is competition for bus control, resulting in the inability to write the video data of the management and control chip into the external memory in a timely manner. The data in the internal cache is retained, and the data written subsequently is not a complete frame of data. The written part will become invalid, which will lead to video frame loss and waste of memory resources. Summary of the invention

[0004] The embodiments of the present application provide a video data processing method, product, device and storage medium, aiming to optimize the video data processing flow.

[0005] A first aspect of an embodiment of the present application provides a video data processing method, the method comprising:

[0006] receiving first video data in a compressed format;

[0007] Obtaining an estimated storage space occupancy of a target video frame in the first video data according to an actual storage space occupancy of the video frame that has been cached and a compression time of the video frame;

[0008] When the estimated storage space occupancy does not exceed the remaining capacity of the parallel memory, writing the target video frame into the parallel memory;

[0009] When the number of the video frames corresponding to the first video data stored in the parallel memory meets a preset transmission frame number threshold, reading the video frames corresponding to the first video data stored in the parallel memory from the parallel memory;

[0010] The video frame corresponding to the first video data is written into an external memory.

[0011] Optionally, the method further comprises:

[0012] When the estimated storage space occupied by the target video frame exceeds the remaining capacity of the parallel memory, determining the capacity gap of the parallel memory;

[0013] When the capacity gap is smaller than a preset capacity gap threshold, accelerating reading of the parallel memory;

[0014] The target video frame is written into the parallel memory while the parallel memory is read at an accelerated speed.

[0015] Optionally, the accelerating reading of the parallel memory comprises:

[0016] The number of addresses read from the parallel memory in each clock cycle is increased.

[0017] Optionally, obtaining the estimated storage space occupancy of the target video frame in the first video data according to the actual storage space occupancy of the cached video frame and the compression time of the video frame includes:

[0018] Determine the actual storage space occupied by each of the video frames that have completed caching and the compression time of the video frame;

[0019] Obtaining a corresponding compression coefficient according to the actual storage space occupied by the video frame and the compression time;

[0020] The estimated storage space occupancy of the target video frame is obtained through the compression coefficient, the compression time of the target video frame and the preset dynamic parameters.

[0021] Optionally, the method further comprises:

[0022] When the target video frame is written, determining the actual storage space occupied by the target video frame;

[0023] The dynamic parameter is adjusted according to the actual storage space occupancy of the target video frame and the estimated storage space occupancy of the target video frame.

[0024] Optionally, writing the target video frame into the parallel memory in parallel includes:

[0025] Determining a minimum writing unit of the target video frame according to a bit width of an interface for sending the first video data;

[0026] The target video frame is sequentially written into a plurality of sub-memories in the parallel memory in the minimum writing unit.

[0027] Optionally, the method further comprises:

[0028] Determine a storage address of the target video frame in each of the sub-memory;

[0029] The storage address is recorded in the frame information table.

[0030] Optionally, the reading the video frame corresponding to the first video data stored in the parallel memory includes:

[0031] Determine in the frame information table a storage address of the video frame corresponding to the first video data;

[0032] The video frame corresponding to the first video data is read from a plurality of sub-memories in the parallel memory according to the storage address.

[0033] Optionally, writing the video frame corresponding to the first video data into an external memory includes:

[0034] Writing the video frames of the corresponding frame number into the first storage area of ​​the external memory;

[0035] When the external driver reads the video frames in the first storage area, writing the video frames of a corresponding number in the second storage area of ​​the external memory;

[0036] When the video frames in the first storage area are completely read, writing the video frames of a corresponding number of frames into the first storage area;

[0037] When writing to the first storage area is completed, waiting for the video frame in the second storage area to be read;

[0038] When the reading of the video frames in the second storage area is completed, the video frames of the corresponding number of frames are written into the second storage area.

[0039] Optionally, the method further comprises:

[0040] receiving uncompressed second video data;

[0041] Writing a first video frame in the second video data into the parallel memory, and using the first video frame as a reference video frame;

[0042] Comparing each subsequent video frame in the second video data with the reference video frame to obtain different pixels between each subsequent video frame and the reference video frame;

[0043] The pixel value of the distinguishing pixel corresponding to each of the subsequent video frames is written into the parallel memory.

[0044] Optionally, the method further comprises:

[0045] Reading the reference video frame and the pixel values ​​of the distinguishing pixels from the parallel memory;

[0046] Writing the reference video frame and the pixel values ​​of the distinguishing pixels into the external memory;

[0047] Sending the reference video frame and the pixel values ​​of the distinguishing pixels in the external memory to a remote end;

[0048] The remote end obtains each of the subsequent video frames in the second video data according to the reference video frame and the pixel values ​​of the distinguishing pixels.

[0049] Optionally, the method further comprises:

[0050] After writing each of the subsequent video frames into the parallel memory, determining the number of the distinguishing pixels corresponding to the subsequent video frames;

[0051] When the number of the distinguishing pixels exceeds a preset pixel number threshold, the next stored video frame is used as the reference video frame.

[0052] A second aspect of an embodiment of the present application provides a video data processing device, the device comprising:

[0053] A first data receiving module, used for receiving first video data in a compressed format;

[0054] A size estimation module, configured to obtain an estimated storage space occupancy of a target video frame in the first video data according to an actual storage space occupancy of a cached video frame and a compression time of the video frame;

[0055] A parallel storage module, configured to write the target video frame into the parallel memory when the estimated storage space occupancy does not exceed the remaining capacity of the parallel memory;

[0056] A first data reading module, configured to read the video frames corresponding to the first video data stored in the parallel memory from the parallel memory when the number of the video frames corresponding to the first video data stored in the parallel memory meets a preset transmission frame number threshold;

[0057] The first data writing module is used to write the video frame corresponding to the first video data into an external memory.

[0058] Optionally, the device further comprises:

[0059] A capacity gap determination module, configured to determine the capacity gap of the parallel memory when the estimated storage space occupied by the target video frame exceeds the remaining capacity of the parallel memory;

[0060] An accelerated reading module, used for accelerating reading of the parallel memory when the capacity gap is less than a preset capacity gap threshold;

[0061] The accelerated writing module is used to write the target video frame into the parallel memory while accelerating the reading of the parallel memory.

[0062] Optionally, the accelerated reading module includes:

[0063] The address quantity increasing submodule is used to increase the number of addresses read from the parallel memory in each clock cycle.

[0064] Optionally, the size estimation module includes:

[0065] An information confirmation submodule, used to determine the actual storage space occupied by each of the video frames that have completed caching and the compression time of the video frame;

[0066] A compression coefficient calculation submodule, used to obtain a corresponding compression coefficient according to the actual storage space occupied by the video frame and the compression time;

[0067] The size calculation submodule is used to obtain the estimated storage space occupied by the target video frame through the compression coefficient, the compression time of the target video frame and the preset dynamic parameters.

[0068] Optionally, the device further comprises:

[0069] An actual storage amount determination submodule, used to determine the actual storage space occupation of the target video frame when the target video frame has completed writing;

[0070] The parameter adjustment submodule is used to adjust the dynamic parameter according to the actual storage space occupied by the target video frame and the estimated storage space occupied by the target video frame.

[0071] Optionally, the first data writing module includes:

[0072] A minimum writing unit determination submodule, used to determine the minimum writing unit of the target video frame according to the bit width of the interface for sending the first video data;

[0073] The sequential writing submodule is used to sequentially write the target video frame into the multiple sub-memory in the parallel memory in the minimum writing unit.

[0074] Optionally, the device further comprises:

[0075] A storage address determination submodule, used to determine the storage address of the target video frame in each of the sub-memory;

[0076] The address recording submodule is used to record the storage address in a frame information table.

[0077] Optionally, the first data reading module includes:

[0078] A storage address search submodule, used to determine the storage address of the video frame corresponding to the first video data in the frame information table;

[0079] The data reading submodule is used to read the video frame corresponding to the first video data from a plurality of sub-memories in the parallel memory according to the storage address.

[0080] Optionally, the first data writing module includes:

[0081] A first writing submodule, used for writing the video frames of a corresponding number of frames into a first storage area of ​​the external memory;

[0082] A second writing submodule is used for writing the video frames of a corresponding number of frames into the second storage area of ​​the external memory when the external driver reads the video frames in the first storage area;

[0083] The third writing submodule writes the video frames of a corresponding number of frames into the first storage area when the video frames in the first storage area are read completely;

[0084] A waiting writing submodule, used for waiting for the video frame in the second storage area to be read after the first storage area is written;

[0085] The fourth writing submodule is used to write the video frames of a corresponding number of frames into the second storage area when the video frames in the second storage area have been read.

[0086] Optionally, the device further comprises:

[0087] A second data receiving module, used for receiving uncompressed second video data;

[0088] A reference frame determination module, used for writing the first video frame in the second video data into the parallel memory, and using the first video frame as a reference video frame;

[0089] A difference acquisition module, used for comparing each subsequent video frame in the second video data with the reference video frame to obtain difference pixels between each subsequent video frame and the reference video frame;

[0090] The pixel value writing module is used to write the pixel value of the distinguishing pixel corresponding to each subsequent video frame into the parallel memory.

[0091] Optionally, the device further comprises:

[0092] The pixel value reading module reads the reference video frame and the pixel values ​​of the distinguishing pixels from the parallel memory;

[0093] A second data writing module, used for writing the reference video frame and the pixel value of the distinguishing pixel into the external memory;

[0094] A data sending submodule, used for sending the reference video frame and the pixel value of the distinguishing pixel in the external memory to a remote end;

[0095] A frame restoration module is used for the remote end to obtain each of the subsequent video frames in the second video data based on the reference video frame and the pixel value of the distinguishing pixel point.

[0096] Optionally, the device further comprises:

[0097] A quantity determination module, configured to determine the quantity of the distinguishing pixels corresponding to each subsequent video frame after writing the subsequent video frame into the parallel memory;

[0098] The reference frame updating module is used to use the next stored video frame as the reference video frame when the number of the distinguishing pixels exceeds a preset pixel number threshold.

[0099] A third aspect of an embodiment of the present application provides a product, including a computer program / instruction, which, when executed by a processor, implements the steps in any of the methods provided in the first aspect of the present application.

[0100] A fourth aspect of an embodiment of the present application provides a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps in the method described in the first aspect of the present application are implemented.

[0101] A fifth aspect of an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method described in the first aspect of the present application are implemented.

[0102] Using the video data processing method provided by the present application, receiving first video data in a compressed format;

[0103] According to the actual storage space occupancy of the video frames that have been cached and the compression time of the video frames, the estimated storage space occupancy of the target video frames in the first video data is obtained; when the estimated storage space occupancy does not exceed the remaining capacity of the parallel memory, the target video frames are written into the parallel memory; when the number of video frames corresponding to the first video data stored in the parallel memory meets a preset transmission frame number threshold, the video frames corresponding to the first video data stored in the parallel memory are read from the parallel memory; and the video frames corresponding to the first video data are written into an external memory.

[0104] In the method, when receiving video data in a compressed format, first, based on the size of the storage space occupied by the video frames that have been cached and the compression time of the video frames, the storage space occupied by the currently written video frames is estimated, and compared with the remaining storage space size in the parallel memory. When the remaining storage space is sufficient, the video frames are written to the parallel memory, and the writing is done in parallel, which improves the speed of data writing. After writing a certain number of video frames, the written video frames are read and written to the external memory, which ensures that the video frames written to the parallel memory are complete, avoids the writing of invalid data, reduces the frame loss rate of the video, and by setting the parallel memory, the speed of reading and writing video data is improved, the occupation of system bus resources, memory space and memory bandwidth by video functions is reduced, the overall performance of the server management and control chip is improved, and the optimization of the video data processing flow is completed. BRIEF DESCRIPTION OF THE DRAWINGS

[0105] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0106] Figure 1 It is a schematic diagram of a traditional video function implementation scheme of a server management and control chip;

[0107] Figure 2 It is a structural diagram of the data output control module in the traditional solution;

[0108] Figure 3 is an architecture diagram of a video data output control module proposed in an embodiment of the present application;

[0109] Figure 4 is a flow chart of a video data processing method proposed in one embodiment of the present application;

[0110] Figure 5 It is a schematic diagram of the traditional scheme;

[0111] Figure 6 is a schematic diagram of a writing scheme proposed in an embodiment of the present application;

[0112] Figure 7 It is a schematic diagram of the processing flow of the data output control module proposed in one embodiment of the present application;

[0113] Figure 8 is a schematic diagram of a video data processing device proposed in an embodiment of the present application;

[0114] Fig. 9 It is a schematic diagram of an electronic device proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0115] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0116] refer to Figure 1 , Figure 1 This is a schematic diagram of the traditional server management and control chip video function implementation solution, such as Figure 1 As shown in the figure, the process of video data processing in the traditional solution is: the original video data (operating system interface) on the host side is transmitted to the VGA (Video Graphics Array) module inside the baseboard management controller through the bus, and the VGA generates the original video data. The post-stage data capture module obtains the original data in two ways: Figure 1 The 1 in the code indicates that the DVI (Digital Video Interface) method is used to obtain the Figure 1 The 2 in the figure indicates that the video data is obtained through the AXI (Advanced eXtensible Interface, a bus protocol) bus. The original RGB data is converted into a color space by the following matrix conversion formula, and the video data in the YUV format (uncompressed format) is obtained from the RGB data.

[0117] The conversion process is completed using the matrix conversion formula.

[0118] Y = (0.257*R) + (0.504*G) + (0.098*B) +16 (1)

[0119] U = 0.148*R – 0.291*G + 0.439*B +128 (2)

[0120] V = 0.439*R-0.368*G-0.071*B+128 (3)

[0121] In compression mode, the core compression module receives data in YUV format and then performs video compression. The YUV format data obtained by color space conversion is directly transmitted to the data output control module. The data output control module receives compressed video data or video data in YUV format, and controls the video data to be written into the external memory (external DDR). After the driver of the network module reads the video data cached in DDR, it is sent to the remote end through the network for display.

[0122] refer to Figure 2 , Figure 2 This is the structure diagram of the data output control module in the traditional solution, such as Figure 2 As shown, the function of this submodule is to receive compressed video data in formats such as JPEG / H.264 or YUV format input from the previous stage, and then write it into the receiving FIFO. In order to avoid the data written into the FIFO being correct, this submodule needs to ensure that the writing is stopped when the FIFO (First Input First Output) is full. Therefore, there are two writing states for a frame of video data (including compressed format and YUV format), one is that all video data of a frame are written into the receiving FIFO, recorded as state 0, and the other is that the video data of a frame cannot be completely written into the receiving FIFO because the receiving FIFO is full, recorded as state 1. Figure 2The frame information statistics module in the above completes the statistics and status cache of the process of writing video frame data to DDR. The AXI interface control module converts the data in the receiving FIFO into the AXI protocol format and writes it to DDR. At the same time, because the writing status of the video frame (status 0 / status 1) can only be obtained from the frame information statistics module when the last data of the current frame data is received, and the previous data has been written to DDR at this time, and these written data are invalid data, because the software on the remote end cannot restore and display these data, and it has also caused a lot of waste of memory bandwidth and invalid occupation of content capacity, especially for YUV mode, because the data volume of a frame of video in YUV mode is huge. Calculated at a resolution of 1920*1200, the data volume of a frame of video is 1920*1200*3=6.59MB. It takes a long time to write video data to the receiving FIFO, so it is often easier to cause the receiving FIFO to be full and cause frame loss (the above phenomenon also occurs in compression mode).

[0123] Because the video function of the server management control chip shares an off-chip DDR and on-chip DDR controller with the operating system (usually Linux) and application running on the chip, there is competition for control of the memory bus, which often results in the video data of the server management control chip being unable to be written to the external DDR in a timely manner, which in turn results in the output control module of the video function of the server management control chip often being unable to receive a complete frame of data. It often happens that only 1 / 3, 1 / 2, or even 4 / 5 of a frame is received, and the internal cache of the output control module is full (because the data in the internal cache cannot be written to the DDR in time), which results in the output control module having to drop the frame being received, and the video data that has been written to the internal cache is invalid data, because although it is written to the server management The data is stored in the internal cache of the control chip, but the data is not a complete frame of data and the remote end cannot recover it. At the same time, the data must be read from the cache by the internal logic of the output control module and written to the external DDR. Because in the traditional solution, the output control module will not know that the data is invalid until the last data of the above data is read. However, the data has been written to the external DDR, and the writing process occupies a large amount of system bus resources, memory bandwidth and memory space. Therefore, the efficiency of the traditional solution in effectively writing to the external DDR is low, the frame loss rate is high, and the overall performance of the chip is low (because a large amount of invalid data is written to the external DDR, which occupies a large amount of memory bandwidth, resulting in the chip's operating system and allowed software being unable to occupy the memory bandwidth in time).

[0124] The above-mentioned drawbacks exist not only in the video processing function of the server management and control chip, but also in most SoC (System on Chip) chips with integrated video functions (because the video function shares DDR and DDR control rights with the chip's operating system and applications).

[0125] The embodiment of the present application is implemented based on the video data output control module, refer to Figure 3 , Figure 3 is an architecture diagram of a video data output control module proposed in an embodiment of the present application, such as Figure 3 As shown, the video output control module includes an uncompressed format data write control module, a memory write control module, a first sub-memory (RAM0), a second sub-memory (RAM1), a memory read control module, a frame information table creation module, an external memory write address control module, an interface control module, a frame size estimation module, and a memory resource comprehensive scheduling management control module. The video data output control module is connected to the external memory (DDR).

[0126] Among them, the uncompressed format data write control module is used to control the writing of uncompressed video data, such as (YUV) format data in the parallel memory, the memory write control module is used to control the writing of compressed format video data in the parallel memory, the first sub-memory and the second sub-memory constitute a parallel memory, the memory read control module is used to control the reading of data, the frame information table creation module is used to create a frame information table to store the storage address of each frame in the memory, the external memory write address control module is used to write data to the external memory, the interface control module is used to convert the data into a protocol format supported by the external memory for transmission, the size estimation module is used to estimate the size of the current frame, and the memory resource comprehensive scheduling management module is used to control the data reading speed and perform comprehensive scheduling of resources.

[0127] In this embodiment, the video data output control module implements a multi-memory parallel mechanism, realizes the sequential writing and parallel output of video data, and designs a memory resource comprehensive management scheduling module. Based on the frame size estimation function of the compression time, it realizes the function of accurately receiving video data and avoids the invalid data writing in the traditional solution. A real-time synchronization verification mechanism based on the comparison frame is implemented at the video output end to realize the YUV value of only the changed pixel points being written and transmitted, which greatly reduces the memory bandwidth and memory capacity occupation in the YUV mode. At the same time, a video frame data writing DDR strategy based on the second storage area mechanism is designed and implemented to ensure that there will be no scenario in which the written DDR data overwrites the unread data in the DDR. The real writing efficiency of video data is improved, the frame loss rate of the video is reduced, the occupation of system bus resources, memory bandwidth and memory capacity by the video function is reduced, and the overall performance of the baseboard management controller is improved.

[0128] refer to Figure 4 , Figure 4 1 is a flow chart of a video data processing method proposed in one embodiment of the present application. Figure 4 As shown, the method comprises the following steps:

[0129] S11: Receive first video data in a compressed format.

[0130] In this embodiment, the first video data is data in a compressed format that needs to be written into an on-chip cache.

[0131] In this embodiment, in the video data processing flow, firstly, the first video data in a compressed format is received by the memory write control module.

[0132] S12: Obtain an estimated storage space occupancy of a target video frame in the first video data according to the actual storage space occupancy of the cached video frame and the compression time of the video frame.

[0133] In this embodiment, one video data includes multiple video frames, each video data is cached in units of frames, the cached video frame is a video frame in the video that is stored before the first video data in the parallel memory, and the video frame compression time refers to the time used to compress one frame of video. The target video frame is the video frame currently being written.

[0134] In this embodiment, the frame size estimation module estimates the estimated storage space occupancy of the target video frame in the first video data according to the actual storage space occupancy of the stored video frames and the compression time of each video frame.

[0135] In this embodiment, under the compressed format, the compressed size of each frame of original video data is different, and the size of the current frame to be written cannot be accurately known before starting to write to the memory. There is no need to estimate the size of uncompressed data. For example, for YUV data, the frame size can be directly calculated based on the resolution.

[0136] In this embodiment, obtaining the estimated storage space occupancy of the target video frame in the first video data according to the actual storage space occupancy of the cached video frame and the compression time of the video frame includes:

[0137] S12-1: Determine the actual storage space occupied by each of the video frames that have completed caching and the compression time of the video frame.

[0138] In this embodiment, the actual storage space occupied by each cached video frame can be directly obtained from the parallel memory inside the module, and the compression time of the video frame is recorded when the video frame is compressed and obtained from the corresponding storage location.

[0139] In this embodiment, when estimating the storage space occupied by the target video frame, the actual storage space occupied by each cached video frame and the compression time of the video frame are first determined.

[0140] For example, the compression time of the frame that has completed on-chip writing (ie, RAM_0 and RAM_1) in the core compression module is recorded as T_0 / T_1 / T_2 . . . T_N.

[0141] S12-2: Obtain a corresponding compression coefficient according to the actual storage space occupied by the video frame and the compression time.

[0142] In this embodiment, the actual storage space occupied by each video frame (frame_real_length) is divided by the compression time of the frame, and the obtained results are summed to obtain the compression coefficient.

[0143] For example, the actual storage space occupied by each video frame is obtained as frame_real_length_0 / frame_real_length_1 / frame_real_length_2 / frame_real_length_N. The compression factor is (frame_real_length_0 / T0+frame_real_length_1 / T1+frame_real_length_2 / T2+...frame_real_length_N / T_N).

[0144] S12-3: Obtain an estimated storage space occupancy of the target video frame through the compression coefficient, the compression time of the target video frame and the preset dynamic parameters.

[0145] In this embodiment, the dynamic parameter is a parameter added to adjust the gap between the estimated video frame size and the actual video frame size.

[0146] In this embodiment, the estimated storage space occupied by the target video frame is obtained by the compression coefficient, the compression time of the target video frame and the preset dynamic parameters. The specific expression is:

[0147] frame_length=(frame_real_length_0 / T0 +frame_real_length_1 / T1 +frame_real_length_2 / T2+……frame_real_length_N / T_N)*T_cur*PARAM / (N+1) (4)

[0148] Among them, T_cur is the compression time of the target video frame, PARAM is a dynamic parameter, and the default value is 1.

[0149] In this embodiment, the method further includes:

[0150] S12-4: When the target video frame has been written, determine the actual storage space occupied by the target video frame.

[0151] In this embodiment, when the writing of the target video frame is completed, the actual storage space occupied by the target video frame is determined.

[0152] S12-5: Adjust the dynamic parameter according to the actual storage space occupancy of the target video frame and the estimated storage space occupancy of the target video frame.

[0153] In this embodiment, the memory write control module obtains the current actual frame size frame_real_length, and adjusts PARAM according to the ratio of frame_real_length to frame_length. The adjusted PARAM_mdy makes

[0154] frame_real_length_0 / T0+frame_real_length_1 / T1+frame_real_length_2 / T2+……+frame_real_length_N / T_N)*T_cur*PARAM_mdy / (N+1)=frame_real_length (5)

[0155] Through such adjustments, the size of the next video frame can be estimated more accurately.

[0156] In this embodiment, after estimating the current video frame, when estimating the size of the next video frame, the calculation parameters in formula (5) are replaced, and the actual space occupied by the frame that has been written to the on-chip memory and the compression time are used to replace the earliest frame in formula (5), that is, frame_real_length_0 / and 0. The next frame is estimated based on the information of the latest N+1 frame, because adjacent images are most similar and the estimation result is most accurate. N can use the default value, such as 10, or it can be configured by the user according to actual conditions.

[0157] S13: When the estimated storage space occupancy does not exceed the remaining capacity of the parallel memory, the target video frame is written into the parallel memory.

[0158] In this embodiment, the parallel memory is composed of a plurality of memories and can store data in parallel.

[0159] In this embodiment, the remaining capacity of the parallel memory is determined, and when the estimated storage space occupancy does not exceed the remaining capacity of the parallel memory, the target video frame is written in parallel into the parallel memory.

[0160] In this embodiment, a multi-memory parallel mechanism is proposed, and a memory carrying address information is used to store video frames. The data bit width of the output interface of the core compression module of the computer is generally 32 bits, while the data bit width of DDR is generally 64 bits or 128 bits. Therefore, in this embodiment, the parallel mechanism of the first sub-memory and the second sub-memory is utilized. At the write end, the memory write control module data is written into the first sub-memory and the second sub-memory in units of 32 bits at a time.

[0161] In this embodiment, the memory resource scheduling management control module implements statistics on the remaining memory resources. When the remaining space is greater than the length of a frame, an indication signal is output to the memory write control module, thereby starting the writing of a new frame of video data to ensure that there is enough space in the first sub-memory and the second sub-memory to cache the data of the current video frame.

[0162] In this embodiment, the specific steps of writing the target video frame into the parallel memory in parallel include:

[0163] S13-1: Determine the minimum writing unit of the target video frame according to the bit width of the interface for sending the first video data.

[0164] In this embodiment, the minimum writing unit of the target video frame is determined first.

[0165] In this embodiment, the minimum writing unit of the target video frame is determined according to the data bit width of the interface that sends the first video data.

[0166] For example, the data bit width of the output interface of compressed data such as JPEG / H.264 is 32 bits. In this case, the minimum writing unit of the target video frame is determined to be 32 bits.

[0167] S13-2: Writing the target video frame into a plurality of sub-memories in the parallel memory in sequence according to the minimum writing unit.

[0168] In this embodiment, the target video frame is written into multiple sub-memory in the parallel memory at one time in minimum write units. When writing, data of a minimum write unit is first written into the first sub-memory, and then data of the second minimum write unit is written into the second sub-memory, and so on.

[0169] For example, when there are two sub-memories in parallel, 32 bits of data are written into the first sub-memory, and then 32 bits of data are written into the second sub-memory, and this cycle is repeated until the data of the target video frame is completely written into the parallel memory.

[0170] In this embodiment, the method further includes:

[0171] S13-3: Determine the storage address of the target video frame in each of the sub-memories.

[0172] In this embodiment, the target video frame fragments are stored in a plurality of parallel sub-memory devices. After each sub-memory device stores the data of the target video frame, the start address and the end address of the data need to be determined.

[0173] S13-4: Record the storage address in the frame information table.

[0174] In this embodiment, the frame information table is a table storing the address of each video frame.

[0175] In this embodiment, the frame information table creation module records the start address and the end address of each video frame in the memory in the frame information table.

[0176] For example, the storage format is {frame_num,ram_num,addr_begin_0,addr_end_0,addr_begin_1,addr_begin_1}, where frame_num is the frame number, and the initial value is 0. The value of ram_num is 0 or 1, where the value 0 represents RAM_0, and the value 1 represents RAM_1. Addr_begin represents the starting address of the current frame data in RAM, and addr_end represents the ending address of the current frame data in RAM. The reason why there are addr_begin_0 / addr_begin_1 and addr_end_0 / addr_end_1 is that the frame data may not be stored continuously in RAM, but is stored in fragments.

[0177] S14: when the number of the video frames corresponding to the first video data stored in the parallel memory meets a preset transmission frame number threshold, reading the video frames corresponding to the first video data stored in the parallel memory from the parallel memory.

[0178] In this embodiment, the transmission frame number threshold is a preset frame number threshold. When the frame number threshold is met, the video frame corresponding to the first video data is read from the parallel memory.

[0179] In this embodiment, when the number of video frames corresponding to the first video data stored in the parallel memory meets a preset transmission frame number threshold, the video frames corresponding to the stored first video data are read from the parallel memory.

[0180] For example, the preset frame number threshold is 4 frames. After 4 frames of data are stored in the parallel memory, the memory read control module starts to read the 4 frames of data that have been stored.

[0181] In this embodiment, the specific steps of reading the video frame corresponding to the first video data stored in the parallel memory include:

[0182] S14-1: Determine the storage address of the video frame corresponding to the first video data in the frame information table.

[0183] In this embodiment, the memory read control module determines the storage address of the video frame corresponding to the first video data in the frame information table.

[0184] S14-2: Read the video frame corresponding to the first video data from a plurality of sub-memories in the parallel memory according to the storage address.

[0185] In this embodiment, according to the obtained storage address, the video frame corresponding to the first video data is read from the multiple sub-memories in the parallel memory. When reading the video frame, data of a fixed number of addresses is read in each clock cycle, for example, data of one address is read in one clock cycle.

[0186] S15: Writing the video frame corresponding to the first video data into an external memory.

[0187] In this embodiment, the external memory is an external memory connected to the video data output control module.

[0188] In this embodiment, after reading multiple video frames of the first video data from the parallel memory, the write address control module writes the read video frames into the external memory.

[0189] In this embodiment, the specific steps of writing the video frame corresponding to the first video data into the external memory include:

[0190] S15-1: Writing the video frames of a corresponding number into the first storage area of ​​the external memory.

[0191] In this embodiment, the external memory is divided into two storage areas, a first storage area and a second storage area, and the data in the two storage areas do not interfere with each other.

[0192] In this embodiment, firstly, video frames of a corresponding number of frames are written into the first storage area of ​​the external memory.

[0193] For example, the number of frames to be written can be determined by oneself. For example, if it is agreed that 4 frames are to be written in total, then 2 frames are written into the first storage area.

[0194] S15-2: When the external driver reads the video frames in the first storage area, write the corresponding number of video frames in the second storage area of ​​the external memory.

[0195] In this embodiment, when the external driver reads the video frames in the first storage area, the data in the first storage area has been written, and the corresponding number of video frames begins to be written in the second storage area of ​​the external memory.

[0196] For example, it is agreed that 4 frames are written in total, the first storage area finishes writing the 0th frame and the 1st frame, and the second storage area starts writing the 2nd frame and the 3rd frame.

[0197] S15-3: When the video frames in the first storage area are completely read, write the corresponding number of video frames into the first storage area.

[0198] In this embodiment, when the video frames in the first storage area have been read, the first storage area has become idle. At this time, video frames corresponding to the number of frames are written into the first storage area.

[0199] S15-4: When the writing to the first storage area is completed, wait for the video frames in the second storage area to be read.

[0200] In this embodiment, when the writing to the first storage area is completed, wait for the video frames in the second storage area to be read.

[0201] S15-5: When the video frames in the second storage area are read, write the video frames corresponding to the number of frames into the second storage area.

[0202] In this embodiment, when the video frames in the second storage area are read, the second storage area becomes idle. At this time, write video frames into the second storage area, and so on in a loop until all the video frames to be written are written into the external memory.

[0203] Reference Figure 5 , Figure 5 is a schematic diagram of the writing in the traditional solution. As Figure 5 shown, in the traditional solution, video data is written into a predefined space outside the chip, and the maximum number of frames (ddr_frame_num) to be written is predefined. For example, it is predefined to write 4 frames in space B of the DDR. When the number of frames written (ddr_frame_cnt) < ddr_frame_num, continue writing. When ddr_frame_cnt = ddr_frame_num, stop writing and wait for the driver to read. When the driver reads one frame, continue writing one frame. The drawback of the traditional solution is that in the compression mode, the size of each compressed frame is not fixed. Therefore, it is possible that the size of the compressed frame of the subsequent frame is larger than that of the previous frame. As a result, after the data output control module writes the 0th / 1st / 2nd / 3rd frames and waits for the driver to read the data of the 0th frame, when the data output control module wants to write the 4th frame, the length of the 4th frame is larger than that of the 0th frame. Therefore, the data of the unread 1st frame will be overwritten, causing chaos and anomalies in the remote display.

[0204] Reference Figure 6 , Figure 6 is a schematic diagram of the writing solution proposed in an embodiment of the present application. As Figure 6As shown, the agreed space outside the chip is divided into two storage areas. The first storage area is written first. When the driver starts to read the data in the first storage area, the data output control module starts to write the second storage area. After the second storage area is written, the writing is suspended. After the driver completely reads the data in the first storage area, the writing starts again. Then, after the first storage area is written and the driver finishes reading and writing the data in the second storage area, the data output control module writes the data in the second storage area. This avoids the problem of data overwriting caused by the different sizes of the previous and next frames in the traditional solution.

[0205] For example, taking ddr_frame_num=4 as an example, the DDR cache area for storing video frames is divided into two storage areas, each storing ddr_frame_num / 2 frames. After the 0th and 1st frames are written in the first storage area, the driver starts to read the first storage area. After starting to read the first storage area, the data output control module starts to write the 2nd / 3rd frames in the second storage area. After the 0th / 1st frames in the first storage area are read by the driver, the 4th / 5th frames are written to the first storage area. This avoids the disadvantage of overwriting the data that has not been read in the DDR in the traditional solution.

[0206] In another embodiment of the present application, the method further includes:

[0207] S21: When the estimated storage space occupied by the target video frame exceeds the remaining capacity of the parallel memory, determine the capacity gap of the parallel memory.

[0208] In this embodiment, when the estimated storage space occupancy of the target video frame exceeds the remaining capacity of the parallel memory, it means that the remaining capacity is insufficient to accommodate the data of the next video frame to be input. At this time, it is necessary to determine the capacity gap of the parallel storage space, that is, to determine how much storage space is needed to store the next video frame data in the parallel memory.

[0209] S22: When the capacity gap is smaller than a preset capacity gap threshold, the parallel memory is accelerated to read.

[0210] In this embodiment, the capacity gap threshold is a pre-set capacity gap value. When the capacity gap of the memory is smaller than the preset capacity gap threshold, the capacity gap can be released by adjusting the reading speed of the memory read control module. When the capacity gap of the memory is larger than the preset capacity gap threshold, the capacity gap cannot be released in this way.

[0211] In this embodiment, when the capacity gap of the parallel memory is less than a preset capacity gap threshold, the parallel memory is accelerated to read, and the specific steps include:

[0212] S22-1: Increase the number of addresses read from the parallel memory in each clock cycle.

[0213] In this embodiment, the memory read control module reads data in a certain number of addresses of the parallel memory in each clock cycle. By increasing the number of addresses read from the parallel memory in each clock cycle, the storage space of the parallel memory can be quickly released, thereby filling the capacity gap.

[0214] For example, when the remaining resources of the parallel memory are insufficient to accommodate the frame data to be input next, but are about to be accommodated, for example, there is still a 3% space gap, that is, RAM_UNUSED_SPACE / frame_length>97%, where RAM_UNUSED_SPACE is the remaining RAM resources, at this time, the module needs to transmit an indication signal to the memory output control module, instructing the memory output control module to speed up the reading speed, from reading one address in one clock cycle to reading two or four addresses in one clock cycle, because RAM has address information, and can realize the function of reading multiple addresses in one clock. After the above-mentioned accelerated RAM reading is executed, the memory resource comprehensive scheduling management control module transmits an indication signal to the RAM write control module to start writing new video frames. The 3% described above can be configured by the user through registers, or the default value can be used.

[0215] S23: writing the target video frame into the parallel memory while accelerating the reading of the parallel memory.

[0216] In this embodiment, when the parallel memory is read at an accelerated speed, the memory space will be released at an accelerated speed, and the capacity gap will be filled. At this time, the target video frame will be written into the parallel memory in parallel, and there will be no shortage of capacity.

[0217] In another embodiment of the present application, the method further includes:

[0218] S31: Receive uncompressed second video data.

[0219] In this embodiment, the second video data is uncompressed video data.

[0220] In this embodiment, the uncompressed format writing control module receives the uncompressed second video data.

[0221] S32: Write the first video frame in the second video data into the parallel memory, and use the first video frame as a reference video frame.

[0222] Because in the actual application scenario of the server management and control chip, the video screen is the operating system interface of the server host, and when in the monitoring state, the change of the server operating system interface is very small. Therefore, in the YUV mode, that is, when the received data is uncompressed data, the image difference between the previous and next frames or adjacent frames is very small. The processing of the YUV mode in the traditional solution is to completely transmit each frame of data, and perform on-chip caching and writing operations on DDR for each frame of YUV data. In fact, because there is a large amount of duplicate data between adjacent frames, the traditional solution has a large amount of duplicate on-chip caching and repeated writing of DDR operations for duplicate data. At the same time, because the amount of video data in the YUV mode is large (because it is not compressed), the YUV mode in the traditional solution occupies a large amount of system bus resources, memory bandwidth and memory resources.

[0223] In this embodiment, the first video frame in the second video data is written into the first sub-memory of the parallel memory, and the written video frame is used as a reference video frame.

[0224] S33: Compare each subsequent video frame in the second video data with the reference video frame to obtain different pixels between each subsequent video frame and the reference video frame.

[0225] In this embodiment, the distinguishing pixel points are pixel points that are distinguished between the subsequent video frame and the reference video frame.

[0226] In this embodiment, starting from the second video frame, each subsequent video frame of the second video frame is compared with the reference video frame, and the pixel position of the reference video frame and each subsequent video frame is matched, and then it is determined whether the pixel values ​​of the two pixels at the corresponding positions are the same. If the values ​​of the two pixels are different, the pixel is used as a distinguishing pixel.

[0227] For example, the pixel value of a pixel in the second video frame is YUV_value_cur, and the pixel value of the corresponding position in the reference video frame stored in RAM_0 is YUV_value_ref. If YUV_value_ref=YUV_value_cur, the YUV value of the pixel in the current input frame is not written into the second sub-memory (RAM_1). If YUV_value_ref!=YUV_value_cur, the YUV value of the pixel in the current input frame is written into the second sub-memory (RAM_1). In this embodiment, when writing the pixel value into the second sub-memory, the position of the row and column number of the pixel is also written. The module can calculate the row and column number position of the current pixel according to the resolution. For example, with a resolution of 1024*768, a frame of image has 768 rows and 1024 columns, and each row has 1024 pixels. If the count starts from 0, then when the count is N, the row number is N / 1024, and the column number is N -(N / 1024)*1024.

[0228] S34: writing the pixel value of the distinguishing pixel corresponding to each of the subsequent video frames into the parallel memory.

[0229] In this embodiment, the comparison is performed by the above method, and the distinguishing pixel value corresponding to each subsequent video frame is written into the parallel memory.

[0230] In this embodiment, the method further includes:

[0231] S35: Read the reference video frame and the pixel values ​​of the distinguishing pixels from the parallel memory.

[0232] In this embodiment, after the reference video frame and the pixel values ​​of the distinguishing pixels in the second video data are written into the parallel memory, the reference video frame and the pixel values ​​of the distinguishing pixels are read from the parallel memory.

[0233] S36: Writing the reference video frame and the pixel values ​​of the distinguishing pixels into the external memory.

[0234] In this embodiment, the reference video frame and the pixel values ​​of the distinguishing pixels are written into an external memory.

[0235] S37: Send the reference video frame and the pixel values ​​of the distinguishing pixels in the external memory to the remote end.

[0236] In this embodiment, the reference video frame and the distinguishing pixels in the external memory are sent to the remote end, and the row and column numbers corresponding to the distinguishing pixels also need to be sent to the remote end.

[0237] S38: The remote end obtains each of the subsequent video frames in the second video data according to the reference video frame and the pixel values ​​of the distinguishing pixels.

[0238] In this embodiment, after obtaining the reference video frame, the remote end determines the position of the distinguishing pixel on the reference video frame by the row and column numbers of the distinguishing pixel, and restores each subsequent video frame by overlaying the pixel value of the distinguishing pixel on the reference video frame.

[0239] In another embodiment of the present application, the method further includes:

[0240] S41: after writing each of the subsequent video frames into the parallel memory, determining the number of the distinguishing pixels corresponding to the subsequent video frames.

[0241] In this embodiment, after each subsequent video frame is written into the parallel memory, the number of distinguishing pixels corresponding to the subsequent video frame is determined, and for each of the subsequent video frames, the number of distinguishing pixels in the frame relative to the reference video frame is counted.

[0242] S42: When the number of distinguishing pixels exceeds a preset pixel number threshold, use the next stored video frame as the reference video frame.

[0243] In this embodiment, the pixel number threshold is a pre-set maximum value of the number of distinguishing pixels. When the number of distinguishing pixels in a certain video frame exceeds the preset pixel number threshold, it means that the difference between the video frame and the reference video frame is too large, and the reference video frame needs to be updated. At this time, the next stored video frame is used as the reference video frame.

[0244] For example, the number of data changes diff_num of the current input frame compared to the reference frame in RAM_0 is counted. When the value of diff_num is greater than the default value default_num (such as 100) or the configuration value sent by the user, it means that the data of the previous and next frames has changed greatly and the reference frame needs to be updated. In this case, the next frame of the current frame is written to RAM_0 (because the current frame has completed the comparison and the different values ​​are written to RAM_1).

[0245] In this embodiment, the design of writing only the data whose corresponding positions in the previous and next frames have changed into the on-chip cache avoids repeated writing of duplicate data into the on-chip cache and external memory, reduces the occupancy of the system bus and memory bandwidth, and improves the overall performance of the chip.

[0246] refer to Figure 7 , Figure 7 FIG. 1 is a schematic diagram of a processing flow of a data output control module proposed in an embodiment of the present application. Figure 7As shown, first input the compressed format data, then estimate the current frame size, write the current frame into the memory when the remaining memory space can be fully written, if not, adjust the reading speed and write the current frame into the memory when it can be fully written, if not, the current frame is dropped, for uncompressed data, determine a reference frame in the memory, compare the corresponding pixels, write only the data that changes subsequently into the memory, and then transmit the data.

[0247] In another embodiment of the present application, the memory resource comprehensive scheduling management control module detects the memory usage of the parallel memory inside the module every preset time period. When the overall load of the parallel memory is higher than the preset load threshold, a sub-memory is added to the parallel memory to reduce the overall load of the parallel memory, thereby allowing the video processing function of the entire chip to run stably.

[0248] In the above-mentioned embodiments of the present application, a multi-memory parallel mechanism is implemented, and the sequential writing and parallel reading of video data are realized, which reduces the time of video data caching in the chip, speeds up the data transmission speed, reduces the frame loss rate, and improves the user experience. A memory resource comprehensive management scheduling module is designed, and the size of the memory space occupied by the video frame is estimated based on the compression time of the video frame, so as to realize the function of accurately receiving data, avoid the writing of invalid data, and reduce the occupation of the system bus, memory bandwidth, and memory resources. A real-time synchronization verification mechanism based on the comparison frame is implemented at the output end of the video, and only the pixel value of the pixel point that has changed is written and transmitted, which greatly reduces the occupation of memory bandwidth and memory capacity in the YUV mode. When writing data to the external memory (DDR), the external memory is partitioned to avoid overwriting the unread data, ensure that the video frame will not be lost, and further improve the user's experience of watching videos or operating pages.

[0249] Based on the same inventive concept, an embodiment of the present application provides a video data processing device. Figure 8 , Figure 8 FIG. 8 is a schematic diagram of a video data processing device 800 proposed in an embodiment of the present application. Figure 8 As shown, the device comprises:

[0250] The first data receiving module 801 is used to receive first video data in a compressed format;

[0251] A size estimation module 802, configured to obtain an estimated storage space occupation of a target video frame in the first video data according to an actual storage space occupation of a cached video frame and a compression time of the video frame;

[0252] A parallel storage module 803, configured to write the target video frame into the parallel memory when the estimated storage space occupancy does not exceed the remaining capacity of the parallel memory;

[0253] A first data reading module 804 is configured to read the video frame corresponding to the first video data stored in the parallel memory from the parallel memory when the number of the video frames corresponding to the first video data stored in the parallel memory meets a preset transmission frame number threshold;

[0254] The first data writing module 805 is used to write the video frame corresponding to the first video data into an external memory.

[0255] Optionally, the device further comprises:

[0256] A capacity gap determination module, configured to determine the capacity gap of the parallel memory when the estimated storage space occupied by the target video frame exceeds the remaining capacity of the parallel memory;

[0257] An accelerated reading module, used for accelerating reading of the parallel memory when the capacity gap is less than a preset capacity gap threshold;

[0258] The accelerated writing module is used to write the target video frame into the parallel memory while accelerating the reading of the parallel memory.

[0259] Optionally, the accelerated reading module includes:

[0260] The address quantity increasing submodule is used to increase the number of addresses read from the parallel memory in each clock cycle.

[0261] Optionally, the size estimation module includes:

[0262] An information confirmation submodule, used to determine the actual storage space occupied by each of the video frames that have completed caching and the compression time of the video frame;

[0263] A compression coefficient calculation submodule, used to obtain a corresponding compression coefficient according to the actual storage space occupied by the video frame and the compression time;

[0264] The size calculation submodule is used to obtain the estimated storage space occupied by the target video frame through the compression coefficient, the compression time of the target video frame and the preset dynamic parameters.

[0265] Optionally, the device further comprises:

[0266] An actual storage amount determination submodule, used to determine the actual storage space occupation of the target video frame when the target video frame has completed writing;

[0267] The parameter adjustment submodule is used to adjust the dynamic parameter according to the actual storage space occupied by the target video frame and the estimated storage space occupied by the target video frame.

[0268] Optionally, the first data writing module includes:

[0269] A minimum writing unit determination submodule, used to determine the minimum writing unit of the target video frame according to the bit width of the interface for sending the first video data;

[0270] The sequential writing submodule is used to sequentially write the target video frame into the multiple sub-memory in the parallel memory in the minimum writing unit.

[0271] Optionally, the device further comprises:

[0272] A storage address determination submodule, used to determine the storage address of the target video frame in each of the sub-memory;

[0273] The address recording submodule is used to record the storage address in a frame information table.

[0274] Optionally, the first data reading module includes:

[0275] A storage address search submodule, used to determine the storage address of the video frame corresponding to the first video data in the frame information table;

[0276] The data reading submodule is used to read the video frame corresponding to the first video data from a plurality of sub-memories in the parallel memory according to the storage address.

[0277] Optionally, the first data writing module includes:

[0278] A first writing submodule, used for writing the video frames of a corresponding number of frames into a first storage area of ​​the external memory;

[0279] A second writing submodule is used for writing the video frames of a corresponding number of frames into the second storage area of ​​the external memory when the external driver reads the video frames in the first storage area;

[0280] The third writing submodule writes the video frames of a corresponding number of frames into the first storage area when the video frames in the first storage area are read completely;

[0281] A waiting writing submodule, used for waiting for the video frame in the second storage area to be read after the first storage area is written;

[0282] The fourth writing submodule is used to write the video frames of a corresponding number of frames into the second storage area when the video frames in the second storage area have been read.

[0283] Optionally, the device further comprises:

[0284] A second data receiving module, used for receiving uncompressed second video data;

[0285] A reference frame determination module, used for writing the first video frame in the second video data into the parallel memory, and using the first video frame as a reference video frame;

[0286] A difference acquisition module, used for comparing each subsequent video frame in the second video data with the reference video frame to obtain difference pixels between each subsequent video frame and the reference video frame;

[0287] The pixel value writing module is used to write the pixel value of the distinguishing pixel corresponding to each subsequent video frame into the parallel memory.

[0288] Optionally, the device further comprises:

[0289] The pixel value reading module reads the reference video frame and the pixel values ​​of the distinguishing pixels from the parallel memory;

[0290] A second data writing module, used for writing the reference video frame and the pixel value of the distinguishing pixel into the external memory;

[0291] A data sending submodule, used for sending the reference video frame and the pixel value of the distinguishing pixel in the external memory to a remote end;

[0292] A frame restoration module is used for the remote end to obtain each of the subsequent video frames in the second video data based on the reference video frame and the pixel value of the distinguishing pixel point.

[0293] Optionally, the device further comprises:

[0294] A quantity determination module, configured to determine the quantity of the distinguishing pixels corresponding to each subsequent video frame after writing the subsequent video frame into the parallel memory;

[0295] The reference frame updating module is used to use the next stored video frame as the reference video frame when the number of the distinguishing pixels exceeds a preset pixel number threshold.

[0296] Based on the same inventive concept, another embodiment of the present application provides a computer program product, including a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the steps in the information display method described in any of the above embodiments of the present application are implemented.

[0297] Based on the same inventive concept, another embodiment of the present application provides a readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the video data processing method described in any of the above embodiments of the present application are implemented.

[0298] Based on the same inventive concept, another embodiment of the present application provides an electronic device, Fig. 9 It is a schematic diagram of an electronic device 900 provided in one embodiment of the present application, including a memory 902, a processor 901 and a computer program stored in the memory and executable on the processor, wherein when the processor is executed, the steps in the video data processing method described in any of the above embodiments of the present application are implemented.

[0299] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0300] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0301] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0302] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0303] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0304] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0305] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0306] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0307] The video data processing method, product, device and storage medium provided by the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A video data processing method, characterized in that: The method comprises: receiving first video data in a compressed format; Obtaining an estimated storage space occupancy of a target video frame in the first video data according to an actual storage space occupancy of the video frame that has been cached and a compression time of the video frame; When the estimated storage space occupancy does not exceed the remaining capacity of the parallel memory, writing the target video frame into the parallel memory; When the number of the video frames corresponding to the first video data stored in the parallel memory meets a preset transmission frame number threshold, reading the video frames corresponding to the first video data stored in the parallel memory from the parallel memory; Writing the video frame corresponding to the first video data into an external memory; The step of obtaining an estimated storage space occupancy of a target video frame in the first video data according to the actual storage space occupancy of the cached video frame and the compression time of the video frame includes: Determine the actual storage space occupied by each of the video frames that have completed caching and the compression time of the video frame; Obtaining a corresponding compression coefficient according to the actual storage space occupied by the video frame and the compression time; The estimated storage space occupied by the target video frame is obtained by the compression coefficient, the compression time of the target video frame and the preset dynamic parameters, and the expression includes: Wherein, T_cur is the compression time of the target video frame, PARAM is a dynamic parameter with a default value of 1, frame_real_length is the actual storage space occupied by each video frame, and T_N is the compression time.

2. The video data processing method according to claim 1, characterized in that: The method further comprises: When the estimated storage space occupied by the target video frame exceeds the remaining capacity of the parallel memory, determining the capacity gap of the parallel memory; When the capacity gap is smaller than a preset capacity gap threshold, accelerating reading of the parallel memory; The target video frame is written into the parallel memory while the parallel memory is read at an accelerated speed.

3. The video data processing method according to claim 2, characterized in that: The step of accelerating reading of the parallel memory comprises: The number of addresses read from the parallel memory in each clock cycle is increased.

4. The video data processing method according to claim 1, characterized in that: The method further comprises: When the target video frame is written, determining the actual storage space occupied by the target video frame; The dynamic parameter is adjusted according to the actual storage space occupancy of the target video frame and the estimated storage space occupancy of the target video frame.

5. The video data processing method according to claim 1, characterized in that: Writing the target video frame into the parallel memory in parallel comprises: Determining a minimum writing unit of the target video frame according to a bit width of an interface for sending the first video data; The target video frame is sequentially written into a plurality of sub-memories in the parallel memory in the minimum writing unit.

6. The video data processing method according to claim 5, characterized in that: The method further comprises: Determine a storage address of the target video frame in each of the sub-memory; The storage address is recorded in the frame information table.

7. The video data processing method according to claim 1, characterized in that: The step of reading the video frame corresponding to the first video data stored in the parallel memory includes: Determine in the frame information table a storage address of the video frame corresponding to the first video data; The video frame corresponding to the first video data is read from a plurality of sub-memories in the parallel memory according to the storage address.

8. The video data processing method according to claim 1, characterized in that: Writing the video frame corresponding to the first video data into an external memory includes: Writing the video frames of the corresponding frame number into the first storage area of ​​the external memory; When the external driver reads the video frames in the first storage area, writing the video frames of a corresponding number in the second storage area of ​​the external memory; When the video frames in the first storage area are completely read, writing the video frames of a corresponding number of frames into the first storage area; When writing to the first storage area is completed, waiting for the video frame in the second storage area to be read; When the reading of the video frames in the second storage area is completed, the video frames of the corresponding number of frames are written into the second storage area.

9. The video data processing method according to claim 1, characterized in that: The method further comprises: receiving uncompressed second video data; Writing a first video frame in the second video data into the parallel memory, and using the first video frame as a reference video frame; Comparing each subsequent video frame in the second video data with the reference video frame to obtain different pixels between each subsequent video frame and the reference video frame; The pixel value of the distinguishing pixel corresponding to each of the subsequent video frames is written into the parallel memory.

10. The video data processing method according to claim 9, characterized in that: The method further comprises: Reading the reference video frame and the pixel values ​​of the distinguishing pixels from the parallel memory; Writing the reference video frame and the pixel values ​​of the distinguishing pixels into the external memory; Sending the reference video frame and the pixel values ​​of the distinguishing pixels in the external memory to a remote end; The remote end obtains each of the subsequent video frames in the second video data according to the reference video frame and the pixel values ​​of the distinguishing pixels.

11. The data processing method according to claim 9, characterized in that: The method further comprises: After writing each of the subsequent video frames into the parallel memory, determining the number of the distinguishing pixels corresponding to the subsequent video frames; When the number of the distinguishing pixels exceeds a preset pixel number threshold, the next stored video frame is used as the reference video frame.

12. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps in the method according to any one of claims 1 to 11 are implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 11 are implemented.

14. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.

Images