Video processing device, method, equipment and storage medium

Abstract

The embodiment of the present application discloses a video processing device, method, equipment and storage medium, the device comprising: a pre-processing module, used to obtain the initial video frame of the input video, and after each initial video frame is obtained, the obtained initial video frame is processed based on the pre-processing parameters, and after receiving the parameter update request, the initial video frames obtained after the first time interval are processed in sequence based on the first preset parameters; a cache module, used to store the intermediate video frames obtained by processing each initial video frame into a preset cache space; a post-processing module, used to read the intermediate video frames in the preset cache space, and process the read intermediate video frames based on the post-processing parameters; a video determination module, used to determine the output video based on the target video frame obtained by processing each intermediate video frame. The embodiment of the present application can improve the efficiency of video processing and has high applicability.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a video processing device, method, equipment and storage medium. Background Art

[0002] In many existing video processing application scenarios, it is often necessary to perform certain processing on the video frames of the input video before outputting the final video. The existing video processing flow often includes video frame pre-processing and video frame post-processing.

[0003] The video frame pre-processing and video frame post-processing links are implemented based on independent processing parameters respectively. When the processing parameters of the video frame pre-processing link need to be adjusted, the processing parameters of the video frame post-processing link need to be adjusted accordingly. However, the prior art often cannot accurately control when to adjust the processing parameters of the video frame post-processing link, so the processing of the video frame will be suspended to stop outputting the video frame until the processing parameters of the video frame pre-processing link and the video frame post-processing link are adjusted, and then the video frame will continue to be processed and output, which will lead to low video processing efficiency and affect the playback effect of the output video. Summary of the invention

[0004] The embodiments of the present application provide a video processing device, method, equipment and storage medium, which can improve video processing efficiency, improve the playback effect of output video, and have high applicability.

[0005] On the one hand, an embodiment of the present application provides a video processing device, the device comprising:

[0006] A pre-processing module, used to obtain each initial video frame of the input video, and after each initial video frame is obtained, process the obtained initial video frame based on the pre-processing parameters, update the above pre-processing parameters to first preset parameters within a first time interval after receiving a parameter update request, and sequentially process the initial video frames obtained after the above first time interval based on the above first preset parameters;

[0007] A cache module, used for storing intermediate video frames obtained by processing each of the above initial video frames into a preset cache space;

[0008] A post-processing module, used for reading the intermediate video frames in the above-mentioned preset cache space, and processing the read intermediate video frames based on the post-processing parameters;

[0009] The video determination module is used to determine the output video based on the target video frame obtained by processing each of the above intermediate video frames.

[0010] On the other hand, an embodiment of the present application provides a video processing method, the method comprising:

[0011] Acquire each initial video frame of the input video, and process the acquired initial video frame based on the pre-processing parameter after each initial video frame is acquired, update the pre-processing parameter to a first preset parameter within a first time interval after receiving a parameter update request, and sequentially process the initial video frames acquired after the first time interval based on the first preset parameter;

[0012] storing the intermediate video frames obtained by processing each of the above initial video frames into a preset cache space;

[0013] Reading the intermediate video frame in the preset cache space, and processing the read intermediate video frame based on the post-processing parameters;

[0014] The output video is determined based on the target video frame obtained by processing each of the above intermediate video frames.

[0015] On the other hand, an embodiment of the present application provides an electronic device, including a processor and a memory, wherein the processor and the memory are connected to each other;

[0016] The memory is used to store computer programs;

[0017] The above-mentioned processor is configured to execute the video processing method provided in the embodiment of the present application when calling the above-mentioned computer program.

[0018] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the video processing method provided by the embodiment of the present application.

[0019] On the other hand, an embodiment of the present application provides a computer program product, which includes a computer program. When the above computer program is executed by a processor, it implements the video processing method provided by the embodiment of the present application.

[0020] Based on the video processing device and method provided by the embodiment of the present application, in the process of processing the initial video frame of the input video based on the pre-processing parameters to obtain the intermediate video frame, if the parameter update request is received, the pre-processing parameters can be updated in the first time interval, and the initial video frame obtained after the first time interval can be processed based on the updated first preset parameters, so that the update of the pre-processing parameters can be completed while ensuring the continuous processing of the initial video frame. At the same time, after the intermediate video frame obtained by processing the initial video frame is stored, the intermediate video frame in the preset cache space can be read to process the read intermediate video frame based on the post-processing parameters, and in the process of updating the pre-processing parameters, the intermediate video frame can also be read and processed based on the post-processing parameters, so as to further ensure the continuous output of the target video frame obtained by processing the intermediate video frame, so as to improve the playback effect of the output video obtained based on the target video frame. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 is a structural schematic diagram of a video processing device provided in an embodiment of the present application;

[0023] Figure 2 is an application schematic diagram of a video processing device provided in an embodiment of the present application;

[0024] Figure 3 is a schematic diagram of the execution flow of the video processing device provided in an embodiment of the present application;

[0025] Figure 4 It is a flowchart of a video processing method provided in an embodiment of the present application;

[0026] Figure 5 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] In many application scenarios of image and video processing, when the format, frame size, aspect ratio or content of the input or output video changes, it is necessary to dynamically switch the relevant processing parameters (such as register configuration parameters) of the video pre-processing link in real time. Due to the particularity of video frame processing, the processing parameters can only be changed at a specific time (such as the vertical blanking period) to avoid damaging the integrity of the picture.

[0029] In addition, when the frame rates of the input video and the output video are inconsistent, the processing parameters corresponding to the video pre-processing link and the processing parameters corresponding to the video post-processing link need to be changed in the corresponding video frames so as not to affect the playback effect of the output video or prevent the inclusion of dirty images (garbage). In a video processing system where the input video and the output video have frame rate conversion, when the frame rate conversion ratio is fixed (such as 1:1, 1:2 or 1:N, N is an integer greater than 2), the corresponding relationship between the output video and each video frame of the input video can be easily predicted. If it is necessary to switch the processing parameters corresponding to the video pre-processing link, it is easier to determine when to adjust the processing parameters corresponding to the video post-processing link. However, when the frame rate conversion ratio relationship is not fixed, if the processing parameters corresponding to the video pre-processing link are adjusted, it is impossible to determine when to adjust the processing parameters corresponding to the video post-processing link. Therefore, in the prior art, if the processing parameters corresponding to the video post-processing link are adjusted in a non-fixed frame rate conversion system, the general practice is to mask the output video, and then adjust the video frames corresponding to the video pre-processing link and the video post-processing link based on the adjusted parameters after all the parameters corresponding to the video pre-processing link and the video post-processing link are adjusted to obtain a normal output video. However, masking the output video will cause the output video to play unsmoothly, affecting the viewing experience.

[0030] Based on the above factors, a video processing device provided in an embodiment of the present application can be specifically referred to in Figure 1 . Figure 1 is a schematic diagram of the structure of a video processing device provided in an embodiment of the present application, such as Figure 1 As shown, the video processing device provided in the embodiment of the present application can be applied to an image processing and display system, wherein the video processing device provided in the embodiment of the present application can specifically include:

[0031] A pre-processing module 11 is used to obtain each initial video frame of the input video, and after each initial video frame is obtained, the obtained initial video frame is processed based on the pre-processing parameters, and when a parameter update request is received, the pre-processing parameters are updated to first preset parameters within a first time interval, and the initial video frames obtained after the first time interval are processed in sequence based on the first preset parameters;

[0032] A cache module 12, used for storing the intermediate video frames obtained by processing each of the above initial video frames into a preset cache space;

[0033] A post-processing module 13, configured to read the intermediate video frames in the above-mentioned preset cache space, and process the read intermediate video frames based on the post-processing parameters;

[0034] The video determination module 14 is used to determine the output video based on the target video frame obtained by processing each of the above intermediate video frames.

[0035] Among them, the input video in the embodiment of the present application can be a video transmitted on a broadcast digital television, the Internet and a mobile network, or it can be a video read through a specific storage medium (such as a CD, a USB flash drive, etc.), or it can be a video generated in real time based on an image, etc. The specific video can be determined based on the actual application scenario requirements and is not limited here.

[0036] When acquiring each initial video frame of the input video, the pre-processing module 11 may acquire the initial video frames in sequence based on the playback order of the initial video frames in the input video, such as acquiring from the first initial video frame to the last initial video frame in sequence.

[0037] The pre-processing module 11 sequentially obtains each initial video frame of the input video and processes each initial video frame based on the pre-processing parameters after obtaining the initial video frame.

[0038] Among them, the pre-processing parameters are relevant parameters required for image processing of each initial video frame of the input video, such as relevant parameters for adjusting the image format, aspect ratio, image content, etc. of each initial video frame of the input video, etc., and can also be relevant parameters for adjusting the frame rate of the input video. The specific parameters can be determined based on the actual application scenario requirements and are not limited here.

[0039] After receiving the parameter update request, the pre-processing module 11 may process the initial video frames acquired after the first time interval in sequence based on the first preset parameters.

[0040] The first preset parameter is also a relevant parameter required for image processing of each initial video frame of the input video, and the first preset parameter is different from the pre-processing parameter. The first preset parameter can be a pre-configured relevant parameter or a relevant parameter carried by the parameter update request, which can be determined based on the actual application scenario requirements and is not limited here.

[0041] After receiving the parameter update request, the pre-processing module 11 may update the pre-processing parameters to first preset parameters in the first time interval, so that the initial video frames acquired after the first time interval may be processed in sequence based on the first preset parameters.

[0042] That is, the pre-processing module 11 processes each acquired initial video frame based on the pre-processing parameters before the first time interval. The pre-processing module 11 processes each acquired initial video frame based on the first preset parameters after the first time interval.

[0043] In the process where the pre-processing module 11 processes the initial video based on the pre-processing parameters and the cache module 12 stores the processed intermediate video frames in sequence into the preset cache space, the post-processing module 13 can read the intermediate video frames from the preset cache space and process the read intermediate video frames based on the post-processing parameters.

[0044] The post-processing module 13 may process an intermediate video frame based on the post-processing parameters after reading an intermediate video frame from the preset buffer space.

[0045] Among them, the post-processing parameters are relevant parameters required for image processing of the intermediate video frames, such as relevant parameters for adjusting the image format, aspect ratio, image content, etc. of each intermediate video frame, etc., and can also be relevant parameters for adjusting the frame rate of the output video generated based on the intermediate video frames. The specific parameters can be determined based on the actual application scenario requirements and are not limited here.

[0046] In some feasible implementation manners, after receiving the parameter update request, the pre-processing module 11 is further configured to:

[0047] Determine the first vertical blanking period corresponding to each of the initial video frames and the reception time of the parameter update request;

[0048] The first vertical blanking period which is after the receiving time and closest to the receiving time is determined as the first time interval.

[0049] The first time interval may be any time interval after the parameter update request is received.

[0050] Optionally, since each initial video frame acquired by the pre-processing module 11 is actually in the form of an electrical signal, for the input video, in the scanning process of converting each initial video frame of the input video from an optical signal to an electrical signal, each initial video frame needs to be scanned in sequence, and the scanning interval between two initial video frames is a vertical blanking period.

[0051] For the input video, since it includes multiple original video frames, the input video may correspond to multiple vertical blanking periods.

[0052] Based on this, after receiving the parameter update request, the pre-processing module 11 can determine the vertical blanking periods corresponding to the initial video (hereinafter referred to as the first vertical blanking period for convenience of description) and the reception time of the parameter update request. Then, the pre-processing module 11 can determine the first vertical blanking period after the reception time of the parameter update request and closest to the reception time as the first time interval.

[0053] That is, if the pre-processing module 11 receives a parameter update request at a certain moment, the first vertical blanking period of the input video after the moment may be determined as the first time interval.

[0054] Among them, for the first initial video frame obtained in the process of updating the pre-processing parameters to the first preset parameters, the pre-processing module 11 can continue to process the first initial video frame after the parameters are updated, or can discard the first initial video frame to process the initial video frame obtained after the parameters are updated based on the first preset parameters. The specific processing method can be determined based on the actual application scenario requirements and is not limited here.

[0055] In some feasible implementations, when the cache module 12 stores the intermediate video frames obtained by processing each of the initial video frames into a preset cache space, it is used to:

[0056] After each of the above initial video frames is processed to obtain an intermediate video frame, if the remaining cache capacity of the preset cache space is not less than the data amount of the intermediate video frame, the intermediate video frame is stored in the above preset cache space; if the remaining cache capacity of the above preset cache space is less than the data amount of the intermediate video frame, the intermediate video frame is discarded.

[0057] The pre-processing module 11 can obtain the corresponding first processed video frame after processing each initial video frame (for the convenience of description, the processed video frame obtained by processing the initial video frame based on the pre-processing parameters or the first preset parameters will be referred to as an intermediate video frame below). After obtaining each intermediate video frame, the cache module 12 can store it in the preset cache space.

[0058] That is, the processing of the initial video frame by the pre-processing module 11 and the storage of the obtained intermediate video frame by the buffer module 12 can be performed simultaneously, and the storage order of the intermediate video frames is consistent with the processing order of their corresponding initial video frames.

[0059] Optionally, since the storage capacity of the cache space is limited, after the pre-processing module 11 processes an initial video frame to obtain an intermediate video frame, the cache module 12 can determine whether to discard the intermediate video frame or store the intermediate video frame in the preset cache space based on the storage status of the preset cache space.

[0060] After each intermediate video frame is obtained, if the remaining cache capacity of the preset cache space is not less than the data amount of the intermediate video frame, the cache module 12 stores the intermediate video frame in the preset cache space. If the remaining cache capacity of the preset cache space is less than the data amount of the intermediate video frame, the cache module 12 discards the intermediate video frame.

[0061] In some feasible implementations, when the post-processing module 13 processes the read intermediate video frame based on the post-processing parameters, it is configured to:

[0062] When it is determined that the next intermediate video frame read after reading the first video frame is the second video frame, updating the post-processing parameters to second preset parameters within a second time interval, and sequentially processing the intermediate video frames read after the second time interval based on the second preset parameters;

[0063] The first video frame is any intermediate video frame in the preset cache space, and the second video frame is the first intermediate video frame stored in the preset cache space after the first time interval.

[0064] When the post-processing module 13 receives a parameter update request and updates the pre-processing parameters to the first preset parameters, when it is determined that the intermediate video frame read next after reading the first video frame is the first intermediate video frame stored in the preset cache space after the first time interval, the post-processing module 13 may update the post-processing parameters to the second preset processing parameters within the second time interval, and process the intermediate video frames read after the second time interval in sequence based on the second preset parameters.

[0065] The second preset parameter is also a relevant parameter required for image processing of each intermediate video frame, and the second preset parameter is different from the post-processing parameter. The second preset parameter can be a pre-configured relevant parameter or a relevant parameter carried by the parameter update request, which can be determined based on the actual application scenario requirements and is not limited here.

[0066] Wherein, when the first preset parameter and the second preset parameter are related parameters carried by the parameter update request, the parameter update request may carry a pre-processing identifier and a post-processing identifier. The pre-processing identifier is used to mark that the first preset parameter is used to process the initial video frame, and the post-processing identifier is used to mark that the second preset parameter is used to process the intermediate video frame.

[0067] In some feasible implementation manners, after the cache module 12 stores the second video frame in the preset cache space, the cache module 12 is further configured to:

[0068] Determine a first identifier of the second video frame, and store the first identifier;

[0069] When determining that the next intermediate video frame to be read after reading the first video frame is the second video frame, the post-processing module 13 is configured to:

[0070] Determining a second identifier of an intermediate video frame to be read next when reading the first video frame;

[0071] If the first identifier is the same as the second identifier, it is determined that the next intermediate video frame read after reading the first video frame is the second video frame.

[0072] After storing the second video frame into the preset cache space, that is, after storing the first intermediate video frame obtained by processing the initial video frame based on the first preset parameter into the preset cache space, the cache module 12 can determine the first identifier of the second video frame and store the first identifier.

[0073] The cache module 12 may store the first identifier in a preset cache space or in other storage spaces, which may be determined based on actual application scenario requirements and is not limited here.

[0074] Furthermore, the post-processing module 13 can predict the second identifier of the intermediate video frame to be read next time when reading each intermediate video frame from the preset cache space. When reading the first video frame (any intermediate video frame), the post-processing module 13 matches the predicted second identifier of the intermediate video frame to be read next time with the first identifier of the stored first video frame. If the two are consistent, it means that the intermediate video frame to be read next after reading the first video frame is the second video frame, that is, the first intermediate video frame stored in the preset cache space after the first time interval.

[0075] Among them, the first identifier and the second identifier can be the frame index corresponding to the intermediate video frame, or can correspond to the storage address of the intermediate video frame (the storage addresses of different intermediate video frames are different). The specific address can be determined based on the actual application scenario requirements and is not limited here.

[0076] In some feasible implementations, when reading the intermediate video frame in the preset cache space, the post-processing module 13 is used to:

[0077] Reading the intermediate video frames in the preset cache space in sequence based on the storage order of the intermediate video frames in the preset cache space;

[0078] Among them, after reading the last intermediate video frame in the above-mentioned preset cache space, if the above-mentioned last intermediate video frame is not the intermediate video frame corresponding to the last initial video frame, the above-mentioned last intermediate video frame is repeatedly read until the above-mentioned preset cache space stores a new intermediate video frame, and then the above-mentioned new intermediate video frame continues to be read.

[0079] The reading order of each intermediate video frame in the preset cache space is consistent with its storage order. That is, when the post-processing module 13 reads the intermediate video frames from the preset cache space, it can read the intermediate video frames in the preset cache space in sequence based on the storage order of the intermediate videos in the preset cache space.

[0080] Among them, after the post-processing module 13 reads the last intermediate video frame in the preset cache space, if the last intermediate video frame is not the intermediate video frame corresponding to the last initial video frame, it repeatedly reads the last intermediate video frame until the preset cache stores a new intermediate video frame, and then continues to read a new intermediate video frame.

[0081] That is, after the post-processing module 13 reads the first intermediate video frame from the preset cache space, if there are no other intermediate video frames stored in the preset cache space after the first intermediate video frame, the first intermediate video frame can be repeatedly read to process a plurality of consecutive first intermediate video frames. When the preset cache space stores the second intermediate video frame after the first intermediate video frame, the post-processing module 13 can suspend the repeated reading of the first intermediate video frame and continue to read the second intermediate video frame.

[0082] Based on the above implementation, the post-processing module 13 can maintain continuous reading and processing of intermediate video frames when the rate at which the intermediate video frames are read from the preset cache space is greater than the rate at which the cache module 12 stores the intermediate video frames into the preset cache space.

[0083] In some feasible implementation manners, after determining the second video frame, the post-processing module 13 is further configured to:

[0084] Determine the second vertical blanking period corresponding to each of the above intermediate video frames;

[0085] A second vertical blanking period after the first moment and closest to the first moment is determined as the second time interval, and the first moment is the moment when the next intermediate video frame read after reading the first video frame is determined to be the second video frame.

[0086] The second time interval may be any time interval after the first moment, wherein the first moment is the moment when the next intermediate video frame read after the first video frame is determined to be the second video frame.

[0087] Optionally, when determining the second time interval for updating the post-processing parameters, the post-processing module 13 can determine the vertical blanking period corresponding to each intermediate video frame when reading each intermediate video frame (for the convenience of description, referred to as the second vertical blanking period below), and then the second vertical blanking period closest to the first moment after the first moment can be determined as the second time interval.

[0088] That is to say, in the process of reading intermediate video frames from the preset cache space, if the post-processing module 13 determines at a certain moment that the intermediate video frame to be read next is the first intermediate video frame stored in the preset cache space after the first time interval, then the first second vertical blanking period after this moment can be determined as the second time interval.

[0089] Among them, for the intermediate video frames obtained in the process of updating the post-processing parameters to the second preset parameters, the post-processing module 13 can continue to process the intermediate video frames after the parameters are updated, or can discard the intermediate video frames to process the intermediate video frames obtained after the parameters are updated based on the second preset parameters. The specific processing method can be determined based on the actual application scenario requirements and is not limited here.

[0090] Based on the video processing device provided in the embodiment of the present application, in the process of processing the acquired initial video frame based on the pre-processing parameters to obtain the intermediate video frame, if the pre-processing module receives a parameter update request, the pre-processing parameters can be updated to the first preset parameters in the most recent vertical blanking period, and the initial video frame acquired after the first preset parameters can be processed to obtain the intermediate video frame, thereby ensuring that the update process of the pre-processing parameters by the pre-processing module does not affect the processing process of the initial video frame, thereby ensuring the continuous processing of the initial video frame.

[0091] Meanwhile, for the post-processing module, before reading the intermediate video frame obtained by processing the initial video frame based on the first preset parameter, the post-processing module may update the post-processing parameter to the second preset parameter in the most recent vertical blanking period before reading the intermediate video frame, so that after reading the intermediate video frame, the intermediate video frame and the intermediate video frame read after reading the intermediate video frame are processed based on the second preset parameter. Based on this method, the post-processing module can determine when to update the post-processing parameter and process the intermediate video frame obtained by processing the initial video frame based on the first preset parameter and the intermediate video frame thereafter based on the updated second preset parameter while maintaining continuous processing of the read intermediate video frame.

[0092] In the case of frame rate conversion between the input video and the output video, regardless of whether the frame rate conversion ratio is fixed, after the pre-processing module completes the update of the pre-processing parameters, the post-processing module can accurately determine when to update the post-processing parameters, thereby eliminating the need to mask the output video and stop masking after the post-processing parameter video is updated to continue processing the read intermediate video frames based on the updated parameters, thereby improving the output video playback and affecting the viewing experience. In addition, based on the video processing device provided in the embodiment of the present application, the pre-processing parameters and post-processing parameters can be adjusted when the pre-processing module continuously processes the initial video frame and the post-processing module continuously processes the read intermediate video frame, thereby ensuring the continuous processing of the initial video frame and the intermediate video frame, and improving the video processing efficiency.

[0093] See also Figure 2 , Figure 2 Schematic diagram of the application of the video processing device provided in the embodiment of the present application. Figure 2 The architecture shown may be an actual application architecture of the video processing device provided in an embodiment of the present application, and the architecture includes a pre-processing unit, a frame buffer unit, a post-processing unit, a register configuration control (Config_ctrl) unit, a write memory control (Wr_buf_ctrl) unit, and a read memory control (Rd_Buf_ctrl) unit.

[0094] The pre-processing unit is used to process each initial video frame of the acquired input video to obtain an intermediate video frame, and the pre-processing unit processes the initial video frame based on its corresponding register configuration parameter Config_register_file(pre).

[0095] The post-processing unit is used to process each intermediate video frame of the read input video to obtain a target video frame, and the post-processing unit processes the intermediate video frame based on its corresponding register configuration parameter Config_register_file(post).

[0096] The packaging unit is used to package the processed intermediate video frames, and the unpacking unit is used to unpack the read intermediate video frames.

[0097] The write memory control unit is used to store the intermediate video frame into the cache unit, and the read memory control unit is used to read the intermediate video frame from the cache unit.

[0098] The register configuration control unit is used to control the updating of register configuration parameters corresponding to the pre-processing unit and the post-processing unit.

[0099] The CPU is used to initiate a parameter update request.

[0100] The pre-processing module 11 processes each initial video frame of the acquired input video to obtain an intermediate video frame through the pre-processing unit, and processes the initial video frame based on its corresponding register configuration parameter Config_register_file(pre).

[0101] The post-processing module 13 processes the intermediate video frames of the read input video to obtain the target video frame through the post-processing unit, and processes the intermediate video frame based on the corresponding register configuration parameter Config_register_file (post).

[0102] The pre-processing module 11 packs the processed intermediate video frames through a packing unit, and the post-processing module unpacks the read intermediate video frames through an unpacking unit.

[0103] The cache module 12 stores the intermediate video frames in the cache unit by writing the memory control unit, and the post-processing module 13 reads the intermediate video frames from the cache unit by reading the memory control unit.

[0104] The pre-processing module 11 and the post-processing module 13 implement the update of register configuration parameters through the register configuration control unit.

[0105] The CPU is used to initiate a parameter update request.

[0106] Combine the following Figure 3 right Figure 2 The execution mode of the application architecture shown is further explained.

[0107] The CPU initiates a register configuration parameter update signal (pre_config_update), that is, initiates a parameter update request, and the register configuration control unit sends the parameter update request to the pre-processing unit.

[0108] After receiving the register configuration parameter update signal, the pre-processing unit will immediately update the register configuration parameter in the latest vertical blanking period, that is, update the pre-processing parameter to the first preset parameter.

[0109] At the same time, after each intermediate video frame is obtained, the write memory control unit will determine whether to perform frame drop processing or write processing on the currently obtained intermediate video frame according to the storage status of the current cache unit. In addition, the write memory control unit will also record the frame storage number (wr_index_keep) of the first intermediate video frame written into the cache unit after the pre-processing unit updates the register configuration parameters, and output the register configuration parameter update flag Pre_config_update_flag signal to the register configuration control unit to indicate that the pre-processing unit has updated the register configuration parameters (the signal value is 1).

[0110] The read memory control unit determines in real time whether the intermediate video frame to be read next time should repeat the intermediate video frame currently being read according to the storage status of the cache unit, and determines the frame storage number (rd_index) of the intermediate video currently being read.

[0111] Among them, the read memory control unit will indicate whether the currently read intermediate video frame needs to be read repeatedly according to the identification signal (nxt_frame_change). When the nxt_frame_change signal value is 0, it means that repeated reading is required, and when the signal value is 1, it means that repeated reading is not required.

[0112] The frame storage number is also the identifier of the video frame. Figure 3 The address can be stored for the frame.

[0113] The register configuration control unit receives the frame storage number (wr_index_keep) and the register configuration parameter update flag Pre_config_update_flag signal sent by the write memory control unit, and the flag signal (nxt_frame_change) and the frame storage number (rd_index) sent by the read memory control unit.

[0114] At a certain moment, if the signal value of the Pre_config_update_flag signal is 1 and the value of the nxt_frame_change signal is 1, when the frame storage number (rd_index+1) of the intermediate video frame read by the read memory control unit next time is the same as the frame storage number (wr_index_keep) of the intermediate video frame first written to the cache unit after the pre-processing unit updates the register configuration parameters, the register configuration control unit outputs a register configuration parameter update post_config_update signal to the post-processing unit to instruct the post-processing unit to update its corresponding register configuration parameters in the first vertical blanking period after it receives the post_config_update signal, that is, to update the post-processing parameters to the second preset parameters.

[0115] Based on the video processing method provided by the embodiment of the present application, in the process of sequentially processing each initial video frame of the initial video to obtain an intermediate video frame, after receiving a parameter update request, the pre-processing parameter can be updated to the first preset parameter within the corresponding first time interval. In the case where the first time interval is the vertical blanking period closest to the reception time of the parameter update request, the pre-processing parameter can be updated in the processing interval of the initial video frame, thereby improving the video processing efficiency.

[0116] In the process of processing the intermediate video frames to obtain the target video frames, if the first intermediate video frame obtained after the first preset parameter is read, the post-processing parameters can be updated to the second preset parameters in the corresponding second time interval. In the case where it is determined that the moment when the intermediate video frame to be read next is the first video frame obtained based on the first preset parameter is the first moment, and the second time interval is the vertical blanking period closest to the first moment, the post-processing parameters can also be updated in the processing interval of the intermediate video frames, so as to maintain the continuous output of the target video frame to improve the video processing efficiency and the playback effect of the output video.

[0117] The video processing device and method provided by the embodiment of the present application can be applied to the frame rate conversion scenario, that is, the frame rate conversion is completed in the process of processing the input video to obtain the input video, so as to convert the input video with a low frame rate into the output video with a high frame rate, or convert the input video with a high frame rate into the output video with a low frame rate. And in the frame rate conversion process, the continuous processing of the intermediate video frames can also be maintained to determine the sustainable output target video frames, thereby improving the playback effect of the output video obtained based on the target video frames.

[0118] See also Figure 4 The embodiment of the present application also provides a video processing method, which may specifically include the following steps:

[0119] Step 41, obtain each initial video frame of the input video, and process the obtained initial video frame based on the pre-processing parameters after each initial video frame is obtained, update the pre-processing parameters to first preset parameters in the first time interval after receiving a parameter update request, and process the initial video frames obtained after the first time interval in sequence based on the first preset parameters.

[0120] Step S42: storing the intermediate video frames obtained by processing each initial video frame into a preset cache space.

[0121] Step S43, read the intermediate video frames in the preset cache space, and process the intermediate video frames based on the post-processing parameters after each intermediate video frame is read; when it is determined that the next intermediate video frame read after reading the first video frame is the second video frame, update the post-processing parameters to second preset parameters within a second time interval, and process the intermediate video frames read after the second time interval in sequence based on the second preset parameters.

[0122] Step S44: determining an output video based on a target video frame obtained by processing each intermediate video frame.

[0123] In some feasible implementations, the processing of the read intermediate video frame based on the post-processing parameters includes:

[0124] When it is determined that the next intermediate video frame read after reading the first video frame is the second video frame, updating the post-processing parameters to second preset parameters within a second time interval, and sequentially processing the intermediate video frames read after the second time interval based on the second preset parameters;

[0125] The first video frame is any intermediate video frame in the preset cache space, and the second video frame is the first intermediate video frame stored in the preset cache space after the first time interval.

[0126] In some feasible implementations, after receiving the parameter update request, the method further includes:

[0127] Determine the first vertical blanking period corresponding to each of the initial video frames and the reception time of the parameter update request;

[0128] The first vertical blanking period which is after the receiving time and closest to the receiving time is determined as the first time interval.

[0129] In some feasible implementations, storing the intermediate video frames obtained by processing each of the initial video frames into a preset cache space includes:

[0130] After each of the above initial video frames is processed to obtain an intermediate video frame, if the remaining cache capacity of the preset cache space is not less than the data amount of the intermediate video frame, the intermediate video frame is stored in the above preset cache space; if the remaining cache capacity of the above preset cache space is less than the data amount of the intermediate video frame, the intermediate video frame is discarded.

[0131] In some feasible implementations, after storing the second video frame in the preset cache space, the method further includes:

[0132] Determine a first identifier of the second video frame, and store the first identifier;

[0133] Determining that the next intermediate video frame read after the first video frame is read is the second video frame includes:

[0134] Determining a second identifier of an intermediate video frame to be read next when reading the first video frame;

[0135] If the first identifier is the same as the second identifier, it is determined that the next intermediate video frame read after reading the first video frame is the second video frame.

[0136] In some feasible implementations, the reading of the intermediate video frame in the preset cache space includes:

[0137] Reading the intermediate video frames in the preset cache space in sequence based on the storage order of the intermediate video frames in the preset cache space;

[0138] Among them, after reading the last intermediate video frame in the above-mentioned preset cache space, if the above-mentioned last intermediate video frame is not the intermediate video frame corresponding to the last initial video frame, the above-mentioned last intermediate video frame is repeatedly read until the above-mentioned preset cache space stores a new intermediate video frame, and then the above-mentioned new intermediate video frame continues to be read.

[0139] In some feasible implementations, after determining the second video frame, the method further includes:

[0140] Determine the second vertical blanking period corresponding to each of the above intermediate video frames;

[0141] A second vertical blanking period after the first moment and closest to the first moment is determined as the second time interval, and the first moment is the moment when the next intermediate video frame read after reading the first video frame is determined to be the second video frame.

[0142] In a specific implementation, the video processing method provided in the embodiment of the present application can be implemented by various functional modules built into the video processing device. For details, see Figure 1 The implementation methods provided by each module will not be repeated here.

[0143] See also Figure 5 , Figure 5 Schematic diagram of the structure of the electronic device provided in the embodiment of the present application. Figure 5As shown, the electronic device 500 in this embodiment may include: a processor 501, a network interface 504 and a memory 505. In addition, the above-mentioned electronic device 500 may also include: an object interface 503, and at least one communication bus 502. Among them, the communication bus 502 is used to realize the connection and communication between these components. Among them, the object interface 503 may include a display screen (Display), a keyboard (Keyboard), and the object interface 503 may optionally include a standard wired interface and a wireless interface. The network interface 504 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 505 may be a high-speed RAM memory, or it may be a non-volatile memory (NVM), such as at least one disk storage. The memory 505 may optionally also be at least one storage device located away from the aforementioned processor 501. As Figure 5 As shown, the memory 505 as a computer-readable storage medium may include an operating system, a network communication module, an object interface module, and a device control application program.

[0144] exist Figure 5 In the electronic device 500 shown, the network interface 504 can provide a network communication function; the object interface 503 is mainly used to provide an input interface for the object; and the processor 501 can be used to call the device control application stored in the memory 505 to achieve:

[0145] Acquire each initial video frame of the input video, and process the acquired initial video frame based on the pre-processing parameter after each initial video frame is acquired, update the pre-processing parameter to a first preset parameter within a first time interval after receiving a parameter update request, and sequentially process the initial video frames acquired after the first time interval based on the first preset parameter;

[0146] storing the intermediate video frames obtained by processing each of the above initial video frames into a preset cache space;

[0147] Reading the intermediate video frames in the preset cache space, and processing each intermediate video frame based on the post-processing parameters after reading an intermediate video frame;

[0148] The output video is determined based on the target video frame obtained by processing each of the above intermediate video frames.

[0149] In some feasible implementations, when processing the read intermediate video frame based on the post-processing parameters, the processor 501 is used to:

[0150] When it is determined that the next intermediate video frame read after reading the first video frame is the second video frame, updating the post-processing parameters to second preset parameters within a second time interval, and sequentially processing the intermediate video frames read after the second time interval based on the second preset parameters;

[0151] The first video frame is any intermediate video frame in the preset cache space, and the second video frame is the first intermediate video frame stored in the preset cache space after the first time interval.

[0152] In some feasible implementations, after receiving the parameter update request, the processor 501 is further configured to:

[0153] Determine the first vertical blanking period corresponding to each of the initial video frames and the reception time of the parameter update request;

[0154] The first vertical blanking period which is after the receiving time and closest to the receiving time is determined as the first time interval.

[0155] In some feasible implementations, the processor 501 is configured to:

[0156] After each of the above initial video frames is processed to obtain an intermediate video frame, if the remaining cache capacity of the preset cache space is not less than the data amount of the intermediate video frame, the intermediate video frame is stored in the above preset cache space; if the remaining cache capacity of the above preset cache space is less than the data amount of the intermediate video frame, the intermediate video frame is discarded.

[0157] In some feasible implementations, after storing the second video frame in the preset cache space, the processor 501 is further configured to:

[0158] Determine a first identifier of the second video frame, and store the first identifier;

[0159] Determining a second identifier of an intermediate video frame to be read next when reading the first video frame;

[0160] If the first identifier is the same as the second identifier, it is determined that the next intermediate video frame read after reading the first video frame is the second video frame.

[0161] In some feasible implementations, the processor 501 is configured to:

[0162] Reading the intermediate video frames in the preset cache space in sequence based on the storage order of the intermediate video frames in the preset cache space;

[0163] Among them, after reading the last intermediate video frame in the above-mentioned preset cache space, if the above-mentioned last intermediate video frame is not the intermediate video frame corresponding to the last initial video frame, the above-mentioned last intermediate video frame is repeatedly read until the above-mentioned preset cache space stores a new intermediate video frame, and then the above-mentioned new intermediate video frame continues to be read.

[0164] In some feasible implementations, after determining the second video frame, the processor 501 is further configured to:

[0165] Determine the second vertical blanking period corresponding to each of the above intermediate video frames;

[0166] A second vertical blanking period after the first moment and closest to the first moment is determined as the second time interval, and the first moment is the moment when the next intermediate video frame read after reading the first video frame is determined to be the second video frame.

[0167] It should be understood that in some feasible implementations, the processor 501 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0168] In a specific implementation, the electronic device 500 can execute the above-mentioned functions through its built-in functional modules. Figure 4 For the implementation methods provided in each step, please refer to the implementation methods provided in the above steps for details, which will not be repeated here.

[0169] The present application also provides a computer-readable storage medium, which stores a computer program and is executed by a processor to implement Figure 4 For the methods provided in each step, please refer to the implementation methods provided in the above steps for details, which will not be repeated here.

[0170] The computer-readable storage medium may be an internal storage unit of the device or electronic device provided in any of the aforementioned embodiments, such as a hard disk or memory of an electronic device. The computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (smart media card, SMC), a secure digital (secure digital, SD) card, a flash card (flash card), etc. equipped on the electronic device. The computer-readable storage medium may also include a magnetic disk, an optical disk, a read-only memory (read-only memory, ROM) or a random access memory (random access memory, RAM), etc. Further, the computer-readable storage medium may also include both an internal storage unit of the electronic device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the electronic device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0171] The present invention provides a computer program product, which includes a computer program. The computer program is executed by a processor. Figure 4 The methods provided in each step.

[0172] The terms "first", "second", etc. in the claims, specification and drawings of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or electronic device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or electronic devices. Mentioning "embodiment" in this article means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present application. Displaying the phrase at various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. The term "and / or" used in the present application specification and the attached claims refers to any combination of one or more of the items listed in association and all possible combinations, and includes these combinations.

[0173] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0174] The above disclosure is only the preferred embodiment of the present application, which cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A video processing device, characterized in that: The device comprises: A pre-processing module, used to obtain each initial video frame of the input video, and after each initial video frame is obtained, process the obtained initial video frame based on the pre-processing parameter, update the pre-processing parameter to a first preset parameter within a first time interval after receiving a parameter update request, and sequentially process the initial video frames obtained after the first time interval based on the first preset parameter; A cache module, used for storing intermediate video frames obtained by processing each of the initial video frames into a preset cache space; A post-processing module, configured to read the intermediate video frames in the preset cache space, and when it is determined that the intermediate video frame to be read next after the first video frame is read is the second video frame, determine the second vertical blanking period corresponding to each of the intermediate video frames, and determine the second vertical blanking period after the first moment and closest to the first moment as the second time interval; update the post-processing parameters to the second preset parameters within the second time interval, and sequentially process the intermediate video frames read after the second time interval based on the second preset parameters; The first video frame is any intermediate video frame in the preset cache space, and the second video frame is the first intermediate video frame stored in the preset cache space after the first time interval; the first moment is the moment of determining the second video frame; The video determination module is used to determine the output video based on the target video frame obtained by processing each of the intermediate video frames.

2. The device according to claim 1, characterized in that After receiving the parameter update request, the pre-processing module is further configured to: Determining a first vertical blanking period corresponding to each of the initial video frames and a reception time of the parameter update request; The first vertical blanking period which is after the receiving moment and closest to the receiving moment is determined as the first time interval.

3. The device according to claim 1, characterized in that When the cache module stores the intermediate video frames obtained by processing each of the initial video frames into the preset cache space, it is used to: After each initial video frame is processed to obtain an intermediate video frame, if the remaining cache capacity of the preset cache space is not less than the data amount of the intermediate video frame, the intermediate video frame is stored in the preset cache space; if the remaining cache capacity of the preset cache space is less than the data amount of the intermediate video frame, the intermediate video frame is discarded.

4. The device according to claim 1, characterized in that After the cache module stores the second video frame in the preset cache space, the cache module is further used to: Determine a first identifier of the second video frame, and store the first identifier; When the post-processing module determines that the next intermediate video frame read after reading the first video frame is the second video frame, it is used to: Determining a second identifier of an intermediate video frame to be read next when reading the first video frame; If the first identifier is the same as the second identifier, it is determined that the next intermediate video frame read after reading the first video frame is the second video frame.

5. The device according to claim 1, characterized in that When reading the intermediate video frame in the preset cache space, the post-processing module is used to: sequentially reading the intermediate video frames in the preset cache space based on the storage order of the intermediate video frames in the preset cache space; Among them, after reading the last intermediate video frame in the preset cache space, if the last intermediate video frame is not the intermediate video frame corresponding to the last initial video frame, the last intermediate video frame is repeatedly read until the preset cache space stores a new intermediate video frame, and then the new intermediate video frame continues to be read.

6. A video processing method, characterized in that: The method comprises: Acquire each initial video frame of the input video, and process the acquired initial video frame based on the pre-processing parameter after each initial video frame is acquired, update the pre-processing parameter to a first preset parameter within a first time interval after receiving a parameter update request, and sequentially process the initial video frames acquired after the first time interval based on the first preset parameter; storing the intermediate video frames obtained by processing each of the initial video frames into a preset cache space; Reading the intermediate video frames in the preset cache space, and when it is determined that the intermediate video frame to be read next after the first video frame is read is the second video frame, determining the second vertical blanking period corresponding to each of the intermediate video frames, and determining the second vertical blanking period after the first moment and closest to the first moment as the second time interval; updating the post-processing parameters to the second preset parameters within the second time interval, and sequentially processing the intermediate video frames read after the second time interval based on the second preset parameters; The first video frame is any intermediate video frame in the preset cache space, and the second video frame is the first intermediate video frame stored in the preset cache space after the first time interval; the first moment is the moment of determining the second video frame; An output video is determined based on a target video frame obtained by processing each of the intermediate video frames.

7. An electronic device, characterized in that: comprising a processor and a memory, wherein the processor and the memory are connected to each other; The memory is used to store computer programs; The processor is configured to execute the method of claim 6 when calling the computer program.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method of claim 6.

Images