Video frame processing method and device, electronic equipment and related equipment

Abstract

The embodiment of the invention provides a video frame processing method and device, electronic equipment and related equipment, which can be applied to various scenes such as cloud technology, artificial intelligence, intelligent transportation, Internet of Vehicles and the like. The method comprises the following steps: acquiring historical coding information corresponding to historical video frames in a to-be-processed video; based on the historical coding information, content detection is carried out on a current video frame, and the current video frame comprises a next video frame of the historical video frame in the to-be-processed video; when content transformation exists in the current video frame, obtaining content transformation information and real-time transmission parameters of the current video frame; determining a coding reference parameter of the current video frame based on the content conversion information, and adjusting a preset coding parameter of the current video frame based on the coding reference parameter and the real-time transmission parameter to obtain a target coding parameter; and according to the target coding parameter, coding the current video frame to obtain a coded target video frame. The image quality of the video frame can be improved.

Description

Technical Field

[0001] The present application relates to the field of coding technology, and in particular to a video frame processing method, device, electronic device and related devices, wherein the related devices include a computer-readable storage medium and a computer program product. Background Art

[0002] There are many ways to transmit information. For example, a sender can transmit information to a receiver in the form of a video stream. When a sender sends a video stream, the sender needs to encode the video stream using an encoder and then send the encoded video stream to the receiver.

[0003] In order to save network bandwidth, currently, smaller encoding parameters are generally used to encode video frames in a video stream. However, after encoding with smaller encoding parameters, the video frames may be easily blurred or even have mosaics.

[0004] In summary, there is currently a problem of poor image quality of video frames. Summary of the invention

[0005] The embodiments of the present application provide a video frame processing method, device, electronic device and related devices, which can improve the image quality of video frames.

[0006] A video frame processing method, comprising:

[0007] Obtain historical encoding information corresponding to historical video frames in the video to be processed;

[0008] Based on the historical coding information, content detection is performed on the current video frame, where the current video frame includes the next video frame of the historical video frame in the video to be processed;

[0009] When there is a content change in the current video frame, obtaining content change information and real-time transmission parameters of the current video frame;

[0010] Determine the encoding reference parameters of the current video frame based on the content transformation information, and adjust the preset encoding parameters of the current video frame based on the encoding reference parameters and the real-time transmission parameters to obtain target encoding parameters;

[0011] According to the target encoding parameters, the current video frame is encoded to obtain an encoded target video frame.

[0012] Accordingly, an embodiment of the present application provides a video frame processing device, including:

[0013] The first acquisition unit may be used to acquire historical coding information corresponding to historical video frames in the video to be processed;

[0014] A detection unit may be used to perform content detection on a current video frame based on historical encoding information, where the current video frame includes a next video frame of a historical video frame in a video to be processed;

[0015] The second acquisition unit may be used to acquire content change information and real-time transmission parameters of the current video frame when there is a content change in the current video frame;

[0016] The adjustment unit may be used to determine the coding reference parameters of the current video frame based on the content transformation information, and adjust the preset coding parameters of the current video frame based on the coding reference parameters and the real-time transmission parameters to obtain the target coding parameters;

[0017] The encoding unit can be used to encode the current video frame according to the target encoding parameter to obtain the encoded target video frame.

[0018] Optionally, in some embodiments, the adjustment unit can be used to obtain candidate initial encoding parameters of the current video frame, and classify the current video frame according to the candidate initial encoding parameters and the encoding reference parameters to obtain the frame type of the current video frame; based on the frame type and real-time transmission parameters, adjust the preset encoding parameters of the current video frame to obtain the target encoding parameters.

[0019] Optionally, in some embodiments, the adjustment unit can be used to fuse the candidate initial encoding parameters and the encoding reference parameters to obtain fused reference parameters; when the parameter value of the fused reference parameters is the same as the preset value, the frame type of the current video frame is determined to be a key frame; when the parameter value of the fused reference parameters is different from the preset value, it is determined that the frame type of the current video frame is not a key frame.

[0020] Optionally, in some embodiments, the adjustment unit can be used to obtain coding amplification parameters when the frame type is a key frame, and determine the target coding parameters of the current video frame based on the coding amplification parameters and the real-time transmission parameters; when the frame type is not a key frame, extract the initial coding parameters from the candidate initial coding parameters, and fuse the coding reference parameters and the initial coding parameters to obtain fused coding parameters, and determine the target coding parameters of the current video frame from the fused coding parameters and the real-time transmission parameters.

[0021] Optionally, in some embodiments, the detection unit can be used to filter out target historical coding information from multiple historical coding information, and extract preset detection parameters corresponding to the target historical coding information from a preset detection parameter set; based on the preset detection parameters and the target historical coding information, perform content detection on the current video frame to obtain a detection result.

[0022] Optionally, in some embodiments, the detection unit can be used to verify the target historical encoding information based on preset detection parameters; when the verification passes, the detection result is determined to be that there is a content change in the current video frame; when the verification fails, the detection result is determined to be that there is no content change in the current video frame.

[0023] Optionally, in some embodiments, the encoding unit can also be used to obtain current encoding information of a target video frame, and use the target video frame as a historical video frame and the current encoding information as historical encoding information; return to execute the step of performing content detection on the current video frame based on the historical encoding information until all video frames in the video to be processed are current video frames, thereby obtaining an encoded video.

[0024] In addition, an embodiment of the present invention further provides an electronic device, including a processor and a memory, wherein the memory stores an application program, and the processor is used to run the application program in the memory to implement the video frame processing method provided by the embodiment of the present invention.

[0025] In addition, an embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. The computer program is suitable for loading by a processor to execute any video frame processing method provided in the embodiment of the present application.

[0026] In addition, an embodiment of the present application also provides a computer program product, including a computer program, which implements any video frame processing method provided in the embodiment of the present application when the computer program is executed by a processor.

[0027] The embodiment of the present application can obtain historical coding information corresponding to historical video frames in the video to be processed; based on the historical coding information, perform content detection on the current video frame, and the current video frame includes the next video frame of the historical video frame in the video to be processed; when there is a content change in the current video frame, obtain the content change information and real-time transmission parameters of the current video frame; based on the content change information, determine the coding reference parameters of the current video frame, and based on the coding reference parameters and the real-time transmission parameters, adjust the preset coding parameters of the current video frame to obtain target coding parameters; according to the target coding parameters, encode the current video frame to obtain the encoded target video frame. Since the present application can perform content detection on the current video frame based on the historical encoding information of the historical video frame, based on this, when there is a content change in the current video frame, the encoding reference parameters of the current video frame can be determined using the acquired content encoding information, so that the preset encoding parameters of the current video frame can be adjusted in real time based on the encoding reference parameters and the real-time transmission parameters of the current video frame to obtain the target encoding parameters. In this way, the current video frame can be encoded using the target encoding parameters, so that the image quality of the encoded target video frame can be improved, and then after the encoded target video frame is decoded, the image quality of the decoded target video frame can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0029] Figure 1 is a scene schematic diagram of the video frame processing method provided in an embodiment of the present application;

[0030] Figure 2 1 is a flowchart of a video frame processing method provided in an embodiment of the present application;

[0031] Figure 3 FIG2 is a flow chart of a video frame processing method provided in an embodiment of the present application;

[0032] Figure 4 is a structural diagram of a video frame processing device provided in an embodiment of the present application;

[0033] Figure 5 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0035] The embodiment of the present application provides a video frame processing method, device, electronic device and related devices. The video frame processing device can be integrated in an electronic device, and the electronic device can be a server or a terminal.

[0036] Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), and big data and artificial intelligence platforms. The terminal can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, car terminal, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.

[0037] For example, see Figure 1 Taking the video frame processing device integrated in an electronic device as an example, the electronic device can obtain historical coding information corresponding to historical video frames in the video to be processed; based on the historical coding information, content detection is performed on the current video frame, and the current video frame includes the next video frame of the historical video frame in the video to be processed; when there is a content change in the current video frame, the content change information and real-time transmission parameters of the current video frame are obtained; based on the content change information, the coding reference parameters of the current video frame are determined, and based on the coding reference parameters and the real-time transmission parameters, the preset coding parameters of the current video frame are adjusted to obtain target coding parameters; according to the target coding parameters, the current video frame is encoded to obtain the encoded target video frame.

[0038] Among them, the video frame processing method of the present application involves coding technology. The present application can perform content detection on the current video frame based on the historical coding information of the historical video frame; when there is a content change in the current video frame, the preset coding parameters of the current video frame can be adjusted in real time to obtain target coding parameters that can improve the image quality of the video frame. In this way, after encoding the current video frame using the target coding parameters obtained by real-time adjustment, the image quality of the encoded target video frame can be improved, and then after decoding the encoded target video frame, the image quality of the decoded video frame can be improved.

[0039] It can be understood that in the specific implementation of this application, the relevant data such as the video to be processed, historical encoding information, encoding reference parameters, etc., when the following embodiments of this application are applied to specific products or technologies, need to obtain permission or consent, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0040] It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments.

[0041] This embodiment will be described from the perspective of a video frame processing device, which can be specifically integrated in an electronic device, which can be a server or a terminal. The terminal can include a tablet computer, a laptop computer, a personal computer (PC), a wearable device, a virtual reality device, or other smart devices that can acquire data.

[0042] A video frame processing method, comprising:

[0043] Obtain historical coding information corresponding to historical video frames in the video to be processed; based on the historical coding information, perform content detection on the current video frame, the current video frame includes the next video frame of the historical video frame in the video to be processed; when there is a content change in the current video frame, obtain the content change information and real-time transmission parameters of the current video frame; based on the content change information, determine the coding reference parameters of the current video frame, and based on the coding reference parameters and the real-time transmission parameters, adjust the preset coding parameters of the current video frame to obtain target coding parameters; according to the target coding parameters, encode the current video frame to obtain the encoded target video frame.

[0044] like Figure 2 As shown, the specific process of the video frame processing method is shown in steps S201 to S205:

[0045] S201. Obtain historical encoding information corresponding to historical video frames in a video to be processed.

[0046] The video to be processed may be any type of video, such as a game video, a TV series video, a live video, etc. Specifically, the video to be processed may be a real-time video. The video frame to be processed may include multiple video frames.

[0047] The historical video frame may refer to a video frame encoded in a historical time. The historical video frame may include at least one video frame.

[0048] The historical coding information may refer to coding information generated during the coding process or when the coding of the original video frame corresponding to the historical video frame is completed in the historical time. Specifically, the historical coding information may include but is not limited to at least one of the intra-frame prediction pixel ratio corresponding to the historical video frame and the code stream size corresponding to the historical video frame. The original video frame belongs to the video to be processed.

[0049] The intra-frame prediction pixel ratio may refer to the ratio between the number of blocks obtained by intra-frame prediction and the number of blocks obtained by other prediction methods in the historical video frame. Among them, the other prediction method may be an inter-frame prediction method. For example, for the historical video frame, a block of size 16×16 is defined as a reference block, and the size of the historical video frame is 1920×1080. Based on this, the historical video frame may have 8100 blocks. If 270 blocks of the historical video frame are obtained by intra-frame prediction, then the intra-frame prediction pixel ratio = 270 / 8100 = 1 / 30.

[0050] The intra-frame prediction method may refer to a method of using adjacent coded pixels in the same video frame to predict the pixels of the currently coded video frame without referring to the coded video frame. The inter-frame prediction method may refer to a method of using adjacent coded video frames to predict the pixels of the currently coded video frame without referring to the coded video frame.

[0051] In step S201, the method of "obtaining historical coding information corresponding to historical video frames in the video to be processed" can be: obtaining the video identifier of the historical video frame in the video to be processed; based on the video identifier, extracting the historical coding information corresponding to the historical video frame in the preset buffer area corresponding to the video to be processed. Alternatively, obtaining historical coding data and preset keywords of the historical video frame in the video to be processed; extracting candidate coding information corresponding to the preset keyword from the historical coding data, and using the candidate coding information as the historical coding information.

[0052] S202: Perform content detection on the current video frame based on historical encoding information.

[0053] The video frame to be processed may include not only the historical video frame but also the current video frame. The current video frame includes the next video frame of the historical video frame in the video to be processed.

[0054] The current video frame may be a video frame that has not been encoded in the video to be processed. For example, the video to be processed includes a first video frame, a second video frame, and a third video frame, wherein the first video frame and the second video frame are both encoded historical video frames, and the third video frame is a current video frame that has not been encoded.

[0055] After obtaining the historical coding information, the present application can use the historical coding information to perform content detection on the current video frame. It can be understood here that for step S202, the present application can use the historical coding information to determine whether there is a content change in the current video frame, without using the image content of the current video frame itself to determine whether there is a content change in the current video frame. This can improve the efficiency of content detection of the current video frame.

[0056] Specifically, in step S202, the method of "performing content detection on the current video frame based on historical encoding information" can be as shown in steps S2021 to S2022:

[0057] Step S2021: Filter out target historical coding information from multiple historical coding information, and extract preset detection parameters corresponding to the target historical coding information from the preset detection parameter set.

[0058] In step S2021, there can be multiple ways to implement the step of "screening out target historical coding information from multiple historical coding information": for example, the present application can obtain a preset keyword, and extract at least one candidate historical coding information corresponding to the preset keyword from multiple historical coding information; and fuse the candidate historical coding information to obtain the target historical coding information.

[0059] For another example, the step of "filtering out target historical coding information from multiple historical coding information" can be: searching for historical coding information identified by preset information from multiple historical coding information; when historical coding information identified by preset information exists, using the historical coding information identified by preset information as the target historical coding information.

[0060] The preset information identifier may refer to a pre-set identifier corresponding to the historical coding information, and each historical coding information has a corresponding preset information identifier.

[0061] The target historical coding information may include the intra-frame prediction pixel ratio of the historical video frame, or the code stream size of the historical video frame.

[0062] The preset detection parameters may refer to pre-set parameters used to perform content detection on the current video frame.

[0063] After the target historical coding information is screened out from multiple historical coding information, the present application can extract the preset detection parameters corresponding to the target historical coding information. The present application can pre-construct the association relationship between the target historical coding information and the preset detection parameters. Based on this, for the step "extracting the preset detection parameters corresponding to the target historical coding information from the preset detection parameter set", the present application can extract the preset detection parameters corresponding to the target historical coding information from the preset detection parameter set based on the association relationship.

[0064] Among them, each target historical coding information has a corresponding preset detection parameter. For example, when the target historical coding information is the intra-frame predicted pixel ratio of the historical video frame, the preset detection parameter can be a detection parameter with a preset threshold as the parameter value; wherein the preset threshold can be a threshold manually set according to engineering and project experience, and the specific size range of the preset threshold can be [0,1], and further, the specific size range of the preset threshold can be set to [0.5,1].

[0065] When the target historical encoding information is the code stream size of the historical video frame, the preset detection parameter may include multiple candidate detection parameters.

[0066] The candidate detection parameters may include a candidate detection parameter with a preset candidate threshold as a parameter value, a coding bit rate corresponding to a historical video frame, and a coding frame rate corresponding to a historical video frame. The coding bit rate may refer to the transmission size of a video frame per unit time; the coding frame rate may refer to the number of video frames transmitted per unit time.

[0067] Step S2022: Based on preset detection parameters and target historical coding information, content detection is performed on the current video frame to obtain a detection result.

[0068] Among them, in step S2022, the step of "performing content detection on the current video frame based on preset detection parameters and target historical coding information to obtain a detection result" can be: based on the preset detection parameters, verifying the target historical coding information; when the verification passes, determining that the detection result is that there is a content change in the current video frame; when the verification fails, determining that the detection result is that there is no content change in the current video frame.

[0069] It can be understood here that the content detection method of the current video frame in the present application can be indirectly achieved by directly verifying historical data such as the target historical encoding information. This can avoid detecting the content of the current video frame itself, thereby improving the content detection efficiency, so that the video quality can be quickly improved.

[0070] Specifically, the method of "verifying the target historical coding information based on the preset detection parameters" is shown in Method 1 and Method 2:

[0071] Method 1: For example, when the target historical coding information is the intra-frame prediction pixel ratio of the historical video frame, the intra-frame prediction pixel ratio is compared with the parameter value corresponding to the preset detection parameter to obtain a comparison result, and the verification result of the target historical coding information is determined based on the comparison result.

[0072] When the intra-frame predicted pixel ratio is greater than the parameter value corresponding to the preset detection parameter, it is determined that the verification of the target historical coding information has passed; when the intra-frame predicted pixel ratio is less than or equal to the parameter value corresponding to the preset detection parameter, it is determined that the verification of the target historical coding information has failed.

[0073] Regarding method one, it can be understood here that when the ratio of predicted pixels within a frame is greater than the parameter value corresponding to the preset detection parameter, it indicates that a scene switch may occur in the content of the historical video frame, or a relatively obvious motion occurs, or a lens rotation occurs. Since the current video frame is the next video frame of the historical video frame, the time interval between the current video frame and the historical video frame is very short and can be almost ignored. Under normal circumstances, there will also be a corresponding scene switch in the current video frame, or a relatively obvious motion occurs, or a lens rotation occurs. Therefore, it is necessary to avoid blurred image quality in the current video frame.

[0074] This situation also shows that in the historical video frames, the inter-frame prediction method cannot find suitable blocks or can only find fewer blocks, so the intra-frame prediction method finds more blocks, and thus the intra-frame prediction pixel ratio is significantly increased. Based on this, the present application uses a preset threshold as the parameter value of the preset detection parameter, and compares the intra-frame prediction pixel ratio with the preset threshold to perform content detection on the current video frame.

[0075] Method 2: For example, when the target historical coding information is the bitstream size of a historical video frame, multiple candidate detection parameters included in the preset detection parameters are fused to obtain fused detection parameters; the parameter value of the fused detection parameter is compared with the bitstream size to obtain a comparison result, and based on the comparison result, the verification result of the target historical coding information is determined.

[0076] The multiple candidate detection parameters included in the preset detection parameters are fused to obtain the fused detection parameters, which can be seen in formula (1):

[0077] T = hr3*enc_bitrate / set_fps Formula (1)

[0078] Among them, T may refer to a fusion detection parameter; the physical meaning of enc_bitrate / set_fps may refer to the size of the historical video frames being evenly distributed under the encoding frame rate corresponding to the historical video frames and the encoding bit rate corresponding to the historical video frames; hr3 may refer to a candidate detection parameter with a preset candidate threshold as a parameter value, wherein the preset candidate threshold may be a threshold manually set based on engineering and project experience, the size range of the preset candidate threshold may be (1.0, +∞), and the size range of the preset candidate threshold is preferably [1.2, +∞); enc_bitrat may refer to the encoding bit rate corresponding to the historical video frames; set_fps may refer to the encoding frame rate corresponding to the historical video frames.

[0079] Accordingly, when the code stream size is greater than the parameter value corresponding to the fusion detection parameter, it is determined that the verification of the target historical coding information has passed; when the code stream size is less than or equal to the parameter value corresponding to the fusion detection parameter, it is determined that the verification of the target historical coding information has failed.

[0080] Regarding method 2, it can be understood here that when the bitstream size is greater than the parameter value corresponding to the fusion detection parameter, it means that the encoding bit rate corresponding to the historical video frame is insufficient. In this case, the historical video frame may have encountered more complex situations or scenes. Since the current video frame is the next video frame of the historical video frame, under normal circumstances, there will also be more complex situations and scenes in the current video frame. Therefore, it is necessary to avoid blurred image quality of the current video frame.

[0081] S203: When there is a content change in the current video frame, obtain content change information and real-time transmission parameters of the current video frame.

[0082] Among them, the content transformation information can be information representing the existence of content transformation in the current video frame, and the content encoding information can be specifically represented as characters. For example, when the content transformation information is 1, it indicates that there is content transformation in the current video frame; when the content transformation information is 0, it indicates that there is no content transformation in the current video frame.

[0083] In step S203, the method for obtaining the content transformation information of the current video frame can be: mapping the detection result obtained based on the detection of the current video frame to obtain the content transformation information corresponding to the detection result; using the content transformation information corresponding to the detection result as the content transformation information corresponding to the current video frame.

[0084] In step S203, the method for obtaining the real-time transmission parameters of the current video frame may be: sending a network test request; performing a network test based on the network test request to obtain the real-time transmission parameters of the current video frame. In this case, the application may send a network test request when encoding an associated frame of the current video frame, wherein the associated frame may be the previous frame of the current video frame, or the first two frames of the current video frame.

[0085] The real-time transmission parameter may refer to a real-time network transmission parameter of the current video frame. For example, the real-time transmission parameter may be a real-time network bandwidth. Specifically, the real-time transmission parameter may be a real-time network maximum bandwidth.

[0086] S204: Determine the coding reference parameters of the current video frame based on the content transformation information, and adjust the preset coding parameters of the current video frame based on the coding reference parameters and the real-time transmission parameters to obtain target coding parameters.

[0087] After obtaining the content transformation parameters, the present application can determine the encoding reference parameters of the current video frame. The encoding reference parameters may refer to parameters related to the determination of the actual target encoding parameters of the current video frame. For example, the encoding reference parameters may include the sorting sequence number of the current video frame in the video to be processed and the encoding calculation factor related to the encoding. The encoding calculation factor can be set to a target value. Among them, the size range of the target value can be set to (1.0, +∞). Preferably, the size range of the target value is usually [1.2, +∞).

[0088] In step S204, the method of "determining the coding reference parameters of the current video frame based on the content transformation information" may be: obtaining a preset coding reference parameter set; searching the preset coding reference parameter set for the preset coding reference parameter corresponding to the content transformation information, and using the preset coding reference parameter as the coding reference parameter corresponding to the current video frame. Alternatively, mapping the content transformation information to obtain mapping parameters corresponding to the content transformation information; using the mapping parameters as the coding reference parameters of the current video frame.

[0089] After obtaining the coding reference parameters, the present application can adjust the preset coding parameters of the current video frame to obtain the target coding parameters, so as to encode the current video frame based on the target coding parameters to improve the image quality of the encoded target video frame. Specifically, there can be many ways to implement the step of "adjusting the preset coding parameters of the current video frame based on the coding reference parameters and the real-time transmission parameters to obtain the target coding parameters". For example, an adjustment function can be obtained; based on the adjustment function, the coding reference parameters, the real-time transmission parameters and the preset coding parameters are merged to obtain the target coding parameters. Among them, the adjustment function can be set according to specific needs and is not limited here.

[0090] For another example, the method of the step of “adjusting the preset encoding parameters of the current video frame based on the encoding reference parameters and the real-time transmission parameters to obtain the target encoding parameters” can be as shown in steps S2041 to S2042:

[0091] S2041. Acquire candidate initial encoding parameters of the current video frame, and classify the current video frame according to the candidate initial encoding parameters and encoding reference parameters to obtain a frame type of the current video frame.

[0092] The candidate initial encoding parameters may refer to initialized parameters related to encoding the current video frame. Specifically, the parameter values ​​of the candidate initial encoding parameters may be default values, or may be determined based on real-time transmission parameters. Specifically, the present application may obtain preset parameters; and merge the preset parameters with the real-time transmission parameters to obtain the candidate initial encoding parameters of the current video frame.

[0093] The candidate initial encoding parameters may include at least one of an initial encoding bit rate, a picture group length, and an initial encoding frame rate.

[0094] The initial encoding bit rate may refer to the size of the video frame transmitted per unit time after the initialization; the initial encoding bit rate is generally set to be relatively small to save traffic. The length of the group of pictures may refer to the length of a continuous group of pictures, where the group of pictures may include a key frame (IDR, Instantaneous Decoder Refresh), a forward prediction coded frame (P, Predictive-frame), and a bi-directional prediction interpolated coded frame (B, Bi-Directional interpolated predictionframe). The first video frame in the group of pictures is a key frame, that is, a key frame must be encoded for each number of video frames corresponding to the length of the coded group of pictures.

[0095] It can be understood here that key frames can only be obtained using intra-frame prediction. Generally speaking, the compression efficiency of key frames is much lower than that of forward predictive coding frames and bidirectional predictive interpolation coding frames. In other words, key frames must use a higher encoding bit rate to achieve a similar image quality level as forward predictive coding frames or bidirectional predictive interpolation coding frames.

[0096] After obtaining the candidate initial encoding parameters of the current video frame, the present application can use the candidate initial encoding parameters to classify the current video frame, so as to adjust the preset encoding parameters of the current video frame based on the frame type obtained by the classification.

[0097] Specifically, the step of "classifying the current video frame according to the candidate initial encoding parameters and the encoding reference parameters to obtain the frame type of the current video frame" can be: fusing the candidate initial encoding parameters and the encoding reference parameters to obtain the fused reference parameters; when the parameter value of the fused reference parameters is the same as the preset value, determining that the frame type of the current video frame is a key frame; when the parameter value of the fused reference parameters is different from the preset value, determining that the frame type of the current video frame is not a key frame.

[0098] The step of "merging the candidate initial coding parameters and the coding reference parameters to obtain the fused reference parameters" may be performed by using a fusion function to fuse the picture group length included in the candidate initial coding parameters and the sorting sequence number of the current video frame included in the coding reference parameters to obtain the fused reference parameters. The fusion function may be specifically shown in formula (2):

[0099] G=cnt%GOP_size formula (2)

[0100] Among them, G may refer to the reference parameter after fusion; cnt may refer to the sorting sequence number of the current video frame; GOP_size may refer to the length of the picture group.

[0101] Among them, the preset value can be set to 0. It can be understood here that when the parameter value of the reference parameter after fusion is equal to 0, it indicates that the current video frame is a key frame; when the parameter value of the reference parameter after fusion is not equal to 0, it indicates that the current video frame is not a key frame.

[0102] After the step of "merging the candidate initial encoding parameters and the encoding reference parameters to obtain fused reference parameters", the present application may compare the parameter values ​​of the fused reference parameters with the preset values.

[0103] S2042: Based on the frame type and the real-time transmission parameters, adjust the preset encoding parameters of the current video frame to obtain target encoding parameters.

[0104] After determining the frame type, the present application can use the frame type and real-time transmission parameters to adjust the preset encoding parameters of the current video frame. Specifically, the step of "adjusting the preset encoding parameters of the current video frame based on the frame type and real-time transmission parameters to obtain the target encoding parameters" can be divided into two cases, such as case 1 and case 2:

[0105] Case 1: When the frame type is a key frame, the encoding upscaling parameters are obtained, and based on the encoding upscaling parameters and the real-time transmission parameters, the target encoding parameters of the current video frame are determined.

[0106] The coding amplification parameter may be a pre-set parameter used to amplify and adjust the preset coding parameter. Specifically, the coding amplification parameter may be a fixed value.

[0107] For situation one, the step of "determining the target coding parameters of the current video frame based on the coding amplification parameters and the real-time transmission parameters" can be as follows: extracting the initial coding parameters from the candidate initial coding parameters, and fusing the coding amplification parameters with the initial coding parameters to obtain the amplified coding parameters; comparing the parameter values ​​of the real-time transmission parameters with the parameter values ​​of the amplified coding parameters, and based on the comparison results, extracting the target coding parameters of the current video frame from the amplified coding parameters and the real-time transmission parameters.

[0108] The initial encoding parameters may include an initial encoding bit rate. The target encoding parameters may refer to a target encoding bit rate. The preset encoding parameters may include a historical encoding bit rate of a historical video frame, or a default encoding bit rate.

[0109] For situation 1, the present application can express situation 1 as formula (3):

[0110] enc_bitrate1= min(max_bitrate, thr1* set_bitrate) formula (3)

[0111] Among them, enc_bitrate1 may refer to a target encoding parameter; max_bitrate may refer to a real-time transmission parameter; thr1 may refer to an encoding amplification parameter; set_bitrate may refer to an initial encoding parameter; and thr1*set_bitrate may refer to an amplified encoding parameter.

[0112] It can be understood here that the parameter value of the coding amplification parameter is a threshold manually set based on engineering and project experience. The parameter value of the coding amplification parameter is usually greater than 1.0. The role of the coding amplification parameter is to amplify the currently set initial coding parameter by thr1 times to avoid the blurred quality of the encoded key frame due to insufficient bit rate. The smaller value is taken between the parameter value of the amplified coding parameter and the parameter value of the real-time transmission parameter to ensure that the target coding bit rate must be less than or equal to the channel bandwidth, which can avoid the occurrence of freezes.

[0113] Case 2: When the frame type is not a key frame, initial encoding parameters are extracted from candidate initial encoding parameters, and target encoding parameters of the current video frame are determined based on the initial encoding parameters and encoding reference parameters.

[0114] For situation two, the step of "determining the target coding parameters of the current video frame based on the initial coding parameters and the coding reference parameters" can be: fusing the coding reference parameters and the initial coding parameters to obtain fused coding parameters, and determining the target coding parameters of the current video frame from the fused coding parameters and the real-time transmission parameters.

[0115] For situation two, it can be understood here that, for the first video frame in the video to be processed, the target value of the encoding calculation factor included in the encoding reference parameter is set to 1; for other video frames after the first video frame in the video to be processed, the target value of the encoding calculation factor included in the encoding reference parameter can be adaptively adjusted based on the historical video frames.

[0116] When the frame type is not a key frame, it indicates that the current video frame is a forward prediction coding frame or a bidirectional prediction interpolation coding frame.

[0117] Among them, for situation 2, the present application can express situation 2 as formula (4):

[0118] enc_bitrate1 = min(max_bitrate, n * set_bitrate) Formula (4)

[0119] Among them, enc_bitrate1 may refer to the target encoding parameter; max_bitrate may refer to the real-time transmission parameter; n*set_bitrate may refer to the fused encoding parameter; n may refer to the encoding calculation factor included in the encoding reference parameter; set_bitrate may refer to the initial encoding parameter.

[0120] S205 . Encode the current video frame according to the target encoding parameters to obtain an encoded target video frame.

[0121] With respect to step S205, the present application may use an encoder to encode the current video frame according to the target encoding parameters to obtain an encoded target video frame. The encoder may be set on an electronic device.

[0122] After step S205, the present application can also encode other video frames in the video frame to be processed, specifically: obtain the current encoding information of the target video frame, and use the target video frame as the historical video frame, and use the current encoding information as the historical encoding information; return to execute the step of performing content detection on the current video frame based on the historical encoding information until all the video frames in the video to be processed are current video frames, so as to obtain the encoded video.

[0123] The current coding information may be coding information generated during or after coding the current video frame. Specifically, the current coding information may include but is not limited to at least one of a current intra-frame predicted pixel ratio of the current video frame and a current bitstream size of the current video frame.

[0124] The present application may use the current coding information as the historical coding information, and store the historical coding information in a preset buffer area.

[0125] In summary, it can be understood that in most cases, the encoding bitrate of the video frame is maintained at a low bitrate level. However, when it is detected that the current video frame has a scene switch, or has a relatively obvious motion or lens rotation, the target encoding parameters of the current video frame, such as the target encoding bitrate, will be adaptively increased to ensure good image quality.

[0126] The embodiment of the present application can obtain historical coding information corresponding to historical video frames in the video to be processed; based on the historical coding information, perform content detection on the current video frame, and the current video frame includes the next video frame of the historical video frame in the video to be processed; when there is a content change in the current video frame, obtain the content change information and real-time transmission parameters of the current video frame; based on the content change information, determine the coding reference parameters of the current video frame, and based on the coding reference parameters and the real-time transmission parameters, adjust the preset coding parameters of the current video frame to obtain target coding parameters; according to the target coding parameters, encode the current video frame to obtain the encoded target video frame. Since the present application can perform content detection on the current video frame based on the historical encoding information of the historical video frame, based on this, when there is a content change in the current video frame, the encoding reference parameters of the current video frame can be determined using the acquired content encoding information, so that the preset encoding parameters of the current video frame can be adjusted in real time based on the encoding reference parameters and the real-time transmission parameters of the current video frame to obtain the target encoding parameters. In this way, the current video frame can be encoded using the target encoding parameters, so that the image quality of the encoded target video frame can be improved, and then after the encoded target video frame is decoded, the image quality of the decoded target video frame can be improved.

[0127] According to the method described in the above embodiment, the following is further described in detail with examples.

[0128] In this embodiment, the video frame processing device is specifically integrated into an electronic device, and the electronic device may be a server, wherein the server may be a cloud server.

[0129] like Figure 3 As shown, a video frame processing method, the specific process is as follows: Step S301 to Step S312:

[0130] S301. The server obtains historical encoding information corresponding to historical video frames in the video to be processed.

[0131] The historical coding information may include the intra-frame prediction pixel ratio corresponding to the historical video frame and the code stream size corresponding to the historical video frame.

[0132] In step S301, the method of "the server obtaining historical coding information corresponding to historical video frames in the video to be processed" can be: obtaining the video identifier of the historical video frame in the video to be processed; based on the video identifier, extracting the historical coding information corresponding to the historical video frame in the preset cache area corresponding to the video to be processed.

[0133] S302: The server performs content detection on the current video frame based on historical encoding information to obtain a detection result.

[0134] In step S302, the method of "the server performs content detection on the current video frame based on the historical coding information to obtain a detection result" can be: the server filters out target historical coding information from multiple historical coding information, and extracts preset detection parameters corresponding to the target historical coding information from a preset detection parameter set; based on the preset detection parameters and the target historical coding information, the server performs content detection on the current video frame to obtain a detection result.

[0135] Specifically, the server may search for historical coding information identified by preset information in multiple historical coding information; when historical coding information identified by preset information exists, the historical coding information identified by preset information is used as target historical coding information.

[0136] After obtaining the target historical coding information, the server can obtain the pre-built association relationship between the target historical coding information and the preset detection parameters; and extract the preset detection parameters corresponding to the target historical coding information from the preset detection parameter set.

[0137] After obtaining the preset detection parameters, the server can perform content detection on the current video frame. Specifically, the server can verify the target historical encoding information based on the preset detection parameters; when the verification passes, the detection result is determined to be that there is a content change in the current video frame; when the verification fails, the detection result is determined to be that there is no content change in the current video frame.

[0138] Among them, when the target historical coding information is the intra-frame prediction pixel ratio of the historical video frame, the intra-frame prediction pixel ratio is compared with the parameter value corresponding to the preset detection parameter to obtain a comparison result, and the verification result of the target historical coding information is determined based on the comparison result.

[0139] Among them, when the target historical coding information is the bitstream size of the historical video frame, multiple candidate detection parameters included in the preset detection parameters are fused to obtain fused detection parameters; the parameter value of the fused detection parameter is compared with the bitstream size to obtain a comparison result, and based on the comparison result, the verification result of the target historical coding information is determined.

[0140] S303: Determine whether the detection result indicates that there is a content change in the current video frame.

[0141] The current video frame includes the next video frame of the historical video frame in the video to be processed.

[0142] When the detection result is that there is a content change in the current video frame, step S304 is executed.

[0143] When there is no content change in the current video frame, candidate encoding reference parameters and real-time transmission parameters are obtained, and based on the candidate encoding reference parameters and the real-time transmission parameters, preset encoding parameters of the current video frame are adjusted to obtain candidate encoding parameters; based on the candidate encoding parameters, the current video frame is encoded to obtain an encoded target video frame.

[0144] The candidate coding reference parameter is different from the coding reference parameter, the candidate coding reference parameter is smaller than the coding reference parameter, and the candidate coding calculation factor included in the candidate coding reference parameter can be set to 1.

[0145] It should be noted here that the method of "adjusting the preset encoding parameters of the current video frame based on the candidate encoding reference parameters and the real-time transmission parameters to obtain the candidate encoding parameters" can refer to the process of "adjusting the preset encoding parameters of the current video frame based on the encoding reference parameters and the real-time transmission parameters to obtain the target encoding parameters", which will not be repeated here; among them, the process of "encoding the current video frame based on the candidate encoding parameters to obtain the encoded target video frame" can refer to the process of "encoding the current video frame according to the target encoding parameters to obtain the encoded target video frame", which will not be repeated here.

[0146] S304: The server obtains content transformation information and real-time transmission parameters of the current video frame.

[0147] The real-time transmission parameter may be a maximum bandwidth of a real-time network.

[0148] In step S304, the server may perform mapping processing on the detection result to obtain content transformation information corresponding to the detection result; and use the content transformation information corresponding to the detection result as content transformation information corresponding to the current video frame.

[0149] In step S304, the server may send a network test request; perform a network test based on the network test request, obtain the real-time network maximum bandwidth of the current video frame, and use the real-time network maximum bandwidth as a real-time transmission parameter.

[0150] Regarding step S304, specifically, the server may be deployed with a network speed measurement module, and the server may test the real-time maximum network bandwidth of the current video frame through the network speed measurement module.

[0151] S305: The server determines encoding reference parameters of the current video frame based on the content transformation information.

[0152] In step S305, the server may obtain a preset encoding reference parameter set; search the preset encoding reference parameter set for preset encoding reference parameters corresponding to the content transformation information, and use the preset encoding reference parameters as encoding reference parameters corresponding to the current video frame.

[0153] For steps S301 to S305, specifically, the server can be deployed with a picture quality detection module, and the server uses the picture quality detection module to obtain historical coding information corresponding to historical video frames in the video to be processed; based on the historical coding information, the server performs content detection on the current video frame to obtain a detection result; determines whether the detection result indicates that there is a content change in the current video frame; obtains the content change information of the current video frame; and determines the coding reference parameters of the current video frame based on the content change information.

[0154] The image quality detection module may set the coding calculation factor included in the coding reference parameter to be greater than 1. The coding calculation factor may also be obtained by: selecting candidate historical coding information from the historical coding information; and mapping the candidate historical coding information to obtain the coding calculation factor.

[0155] For example, a target keyword may be obtained, and candidate historical coding information may be screened out from the historical coding information based on the target keyword. When the candidate historical coding information is the bitstream size corresponding to the historical video frame, the bitstream size is mapped to obtain the coding calculation factor.

[0156] When the candidate historical coding information is the intra-frame prediction pixel ratio corresponding to the historical video frame, the candidate detection parameters including the encoding bit rate corresponding to the historical video frame, the encoding frame rate corresponding to the historical video frame, and the intra-frame prediction pixel ratio are fused to obtain fused data; the fused data is mapped to obtain the coding calculation factor.

[0157] The method of fusing the intra-frame prediction pixel ratio, the initial encoding bit rate and the initial encoding frame rate can be referred to in formula (5):

[0158] k=frame_size / (enc_bitrate / set_fps) Formula (5)

[0159] Among them, k may refer to fusion data; frame_size may refer to the ratio of intra-frame prediction pixels; enc_bitrate may refer to the encoding bit rate corresponding to the historical video frame; set_fps may refer to the encoding frame rate corresponding to the historical video frame.

[0160] The image quality detection module may use a frame counter to obtain the sorting sequence number of the current video frame included in the encoding reference parameter. Each time a video frame is encoded, the frame counter may add 1 to the sequence number of the historical video frame just encoded to obtain the sorting sequence number of the current video frame.

[0161] S306: The server obtains candidate initial coding parameters of the current video frame, and merges the candidate initial coding parameters with coding reference parameters to obtain merged reference parameters.

[0162] The candidate initial coding reference parameters may include an initial coding bit rate, a picture group length, and an initial coding frame rate.

[0163] In step S306, the server may use the aforementioned formula (2) to merge the candidate initial encoding parameters and the encoding reference parameters to obtain merged reference parameters.

[0164] S307: The server determines whether the parameter value of the reference parameter after fusion is the same as the preset value.

[0165] When the parameter value of the reference parameter after fusion is the same as the preset value, step S308 is executed.

[0166] When the parameter value of the reference parameter after fusion is different from the preset value, step S310 is executed.

[0167] S308: The server determines that the frame type of the current video frame is a key frame.

[0168] S309: The server obtains a coding upscaling parameter, and determines a target coding parameter of the current video frame based on the coding upscaling parameter and the real-time transmission parameter.

[0169] In step S309, the server can extract the initial coding parameters from the candidate initial coding parameters, and merge the coding amplification parameters with the initial coding parameters to obtain the amplified coding parameters; compare the parameter values ​​of the real-time transmission parameters with the parameter values ​​of the amplified coding parameters, and based on the comparison results, extract the target coding parameters of the current video frame from the amplified coding parameters and the real-time transmission parameters.

[0170] S310: The server determines that the frame type of the current video frame is not a key frame.

[0171] S311. The server extracts initial coding parameters from candidate initial coding parameters, and determines target coding parameters of the current video frame based on the initial coding parameters and coding reference parameters.

[0172] In step S311, after extracting the initial coding parameters from the candidate initial coding parameters, the server may fuse the coding reference parameters and the initial coding parameters to obtain fused coding parameters, and determine the target coding parameters of the current video frame from the fused coding parameters and the real-time transmission parameters.

[0173] For steps S306 to S311, a coding adjustment module can be deployed on the server. The server obtains the candidate initial coding parameters of the current video frame through the coding adjustment module, and fuses the candidate initial coding parameters and the coding reference parameters to obtain fused reference parameters; and determines whether the parameter value of the fused reference parameter is the same as the preset value.

[0174] When the parameter value of the reference parameter after fusion is the same as the preset value, the encoding adjustment module determines that the frame type of the current video frame is a key frame; obtains the encoding magnification parameter, and determines the target encoding parameter of the current video frame based on the encoding magnification parameter and the real-time transmission parameter.

[0175] When the parameter value of the fused reference parameter is different from the preset value, the encoding adjustment module determines that the frame type of the current video frame is not a key frame; extracts the initial encoding parameter from the candidate initial encoding parameters, and determines the target encoding parameter of the current video frame based on the initial encoding parameter and the encoding reference parameter.

[0176] S312: Encode the current video frame according to the target encoding parameters to obtain an encoded target video frame.

[0177] The target encoding parameter may be a target encoding bit rate.

[0178] After step S312, the present application can also encode other video frames in the video frame to be processed, specifically: obtain current encoding information of the target video frame, and use the target video frame as the historical video frame, and use the current encoding information as the historical encoding information; return to execute step S301.

[0179] After step S312, the server may also send the target video frame to the receiving end, so that the receiving end receives the target video frame and decodes the target video frame to obtain a decoded video frame; then, the receiving end displays the decoded video frame. It can be understood here that the image quality of the decoded video frame displayed by the receiving end is improved.

[0180] The present application has many advantages, for example, (1) the present application can still encode at a lower bit rate in most cases, so it can effectively reduce traffic and bandwidth costs. (2) The present application can effectively identify IDR frames, scene switching, fast motion or lens rotation and shaking scenes through the image quality detection module, and timely increase the target encoding bit rate of the current video frame through the encoder adjustment module to avoid blur, and the image quality can be effectively improved. (3) The present application fully considers the network situation, and effectively guarantees the stability of the bit rate through the collaboration of the network speed measurement module and the encoding adjustment module, avoiding freezes caused by the encoding bit rate exceeding the maximum network bandwidth.

[0181] This application can alleviate the problem of video frame quality degradation and blurring in low real-time communication scenarios by coordinating the network speed measurement module, the image quality detection module and the encoding adjustment module, and effectively improve the user experience. Among them, the network speed measurement module is mainly responsible for providing the real-time maximum network bandwidth of servers such as the cloud and terminals in real time; the image quality detection module will detect the historical encoding information of each historical video frame in real time, and determine whether the preset encoding parameters of the current video frame such as the preset encoding bit rate are required; the encoding adjustment module will adjust the preset encoding bit rate in real time according to the results of the network speed measurement module and the image quality detection module. The effective collaboration of the network speed measurement module, the image quality detection module and the encoding adjustment module can avoid the problem of image quality degradation and blurring when scene switching, fast motion, etc. occurs, and effectively improve the image quality.

[0182] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0183] The embodiment of the present application can obtain historical coding information corresponding to historical video frames in the video to be processed; based on the historical coding information, perform content detection on the current video frame, and the current video frame includes the next video frame of the historical video frame in the video to be processed; when there is a content change in the current video frame, obtain the content change information and real-time transmission parameters of the current video frame; based on the content change information, determine the coding reference parameters of the current video frame, and based on the coding reference parameters and the real-time transmission parameters, adjust the preset coding parameters of the current video frame to obtain target coding parameters; according to the target coding parameters, encode the current video frame to obtain the encoded target video frame. Since the present application can perform content detection on the current video frame based on the historical encoding information of the historical video frame, based on this, when there is a content change in the current video frame, the encoding reference parameters of the current video frame can be determined using the acquired content encoding information, so that the preset encoding parameters of the current video frame can be adjusted in real time based on the encoding reference parameters and the real-time transmission parameters of the current video frame to obtain the target encoding parameters. In this way, the current video frame can be encoded using the target encoding parameters, so that the image quality of the encoded target video frame can be improved, and then after the encoded target video frame is decoded, the image quality of the decoded target video frame can be improved.

[0184] In order to better implement the above method, an embodiment of the present application also provides a video frame processing device, which can be integrated into an electronic device, such as a server or terminal, and the terminal may include a tablet computer, a laptop computer and / or a personal computer, etc.

[0185] For example, Figure 4 As shown, the video frame processing device may include a first acquisition unit 301, a detection unit 302, a second acquisition unit 303, an adjustment unit 304 and an encoding unit 305, as follows:

[0186] (1) a first acquisition unit 301;

[0187] The first acquisition unit 301 may be used to acquire historical encoding information corresponding to historical video frames in the video to be processed.

[0188] (2) a detection unit 302;

[0189] The detection unit 302 may be configured to perform content detection on a current video frame based on historical encoding information, where the current video frame includes a next video frame of a historical video frame in the video to be processed.

[0190] In some embodiments, the detection unit 302 can be used to filter out target historical coding information from multiple historical coding information, and extract preset detection parameters corresponding to the target historical coding information from a preset detection parameter set; based on the preset detection parameters and the target historical coding information, perform content detection on the current video frame to obtain a detection result.

[0191] (3) a second acquisition unit 303;

[0192] The second acquisition unit 303 may be configured to acquire content transformation information and real-time transmission parameters of the current video frame when there is content transformation in the current video frame.

[0193] (4) adjustment unit 304;

[0194] The adjustment unit 304 may be configured to determine the coding reference parameters of the current video frame based on the content transformation information, and adjust the preset coding parameters of the current video frame based on the coding reference parameters and the real-time transmission parameters to obtain target coding parameters.

[0195] In some embodiments, the adjustment unit 304 can be used to obtain candidate initial encoding parameters of the current video frame, and classify the current video frame according to the candidate initial encoding parameters and the encoding reference parameters to obtain the frame type of the current video frame; based on the frame type and real-time transmission parameters, adjust the preset encoding parameters of the current video frame to obtain the target encoding parameters.

[0196] In some embodiments, the adjustment unit 304 can be used to fuse the candidate initial encoding parameters and the encoding reference parameters to obtain fused reference parameters; when the parameter value of the fused reference parameters is the same as the preset value, the frame type of the current video frame is determined to be a key frame; when the parameter value of the fused reference parameters is different from the preset value, it is determined that the frame type of the current video frame is not a key frame.

[0197] (5) Coding unit 305;

[0198] The encoding unit 305 may be configured to encode the current video frame according to the target encoding parameter to obtain an encoded target video frame.

[0199] In some embodiments, the encoding unit 305 can also be used to obtain current encoding information of the target video frame, and use the target video frame as a historical video frame and the current encoding information as historical encoding information; return to execute the step of performing content detection on the current video frame based on the historical encoding information until all video frames in the video to be processed are current video frames, thereby obtaining the encoded video.

[0200] As can be seen from the above, the first acquisition unit 301 of the embodiment of the present application can be used to obtain historical coding information corresponding to historical video frames in the video to be processed; the detection unit 302 can be used to perform content detection on the current video frame based on the historical coding information, and the current video frame includes the next video frame of the historical video frame in the video to be processed; the second acquisition unit 303 can be used to obtain the content transformation information and real-time transmission parameters of the current video frame when there is a content transformation in the current video frame; the adjustment unit 304 can be used to determine the encoding reference parameters of the current video frame based on the content transformation information, and adjust the preset encoding parameters of the current video frame based on the encoding reference parameters and the real-time transmission parameters to obtain the target encoding parameters; the encoding unit 305 can be used to encode the current video frame according to the target encoding parameters to obtain the encoded target video frame. Since the present application can perform content detection on the current video frame based on the historical encoding information of the historical video frame, based on this, when there is a content change in the current video frame, the encoding reference parameters of the current video frame can be determined using the acquired content encoding information, so that the preset encoding parameters of the current video frame can be adjusted in real time based on the encoding reference parameters and the real-time transmission parameters of the current video frame to obtain the target encoding parameters. In this way, the current video frame can be encoded using the target encoding parameters, so that the image quality of the encoded target video frame can be improved, and then after the encoded target video frame is decoded, the image quality of the decoded target video frame can be improved.

[0201] The present application also provides an electronic device, such as Figure 5 As shown, it shows a schematic diagram of the structure of an electronic device involved in an embodiment of the present invention, specifically:

[0202] The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that Figure 5 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0203] The processor 401 is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 402, and calling data stored in the memory 402, the processor 401 performs various functions of the electronic device and processes data. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 401.

[0204] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0205] The electronic device also includes a power supply 403 for supplying power to each component. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 403 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0206] The electronic device may further include an input unit 404, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0207] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 401 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 will run the application programs stored in the memory 402, thereby realizing various functions, as follows:

[0208] Obtain historical coding information corresponding to historical video frames in the video to be processed; based on the historical coding information, perform content detection on the current video frame, the current video frame includes the next video frame of the historical video frame in the video to be processed; when there is a content change in the current video frame, obtain the content change information and real-time transmission parameters of the current video frame; based on the content change information, determine the coding reference parameters of the current video frame, and based on the coding reference parameters and the real-time transmission parameters, adjust the preset coding parameters of the current video frame to obtain target coding parameters; according to the target coding parameters, encode the current video frame to obtain the encoded target video frame.

[0209] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0210] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0211] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. The computer program can be loaded by a processor to execute any video frame processing method provided in the embodiment of the present application.

[0212] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0213] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0214] Since the instructions stored in the computer-readable storage medium can execute the steps in any one of the video frame processing methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the video frame processing methods provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0215] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the method provided in various optional implementations provided in the above embodiments.

[0216] The above is a detailed introduction to a video frame processing method, device, electronic device and related equipment provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A video frame processing method, characterized in that: include: Obtain historical encoding information corresponding to historical video frames in the video to be processed; Based on the historical encoding information, content detection is performed on a current video frame, where the current video frame includes a next video frame of the historical video frame in the video to be processed; When there is a content change in the current video frame, obtaining content change information and real-time transmission parameters of the current video frame; Determine the encoding reference parameters of the current video frame based on the content transformation information, and adjust the preset encoding parameters of the current video frame based on the encoding reference parameters and the real-time transmission parameters to obtain target encoding parameters; The current video frame is encoded according to the target encoding parameters to obtain an encoded target video frame.

2. The video frame processing method according to claim 1, characterized in that: The step of adjusting the preset encoding parameters of the current video frame based on the encoding reference parameters and the real-time transmission parameters to obtain target encoding parameters includes: Acquire candidate initial encoding parameters of the current video frame, and classify the current video frame according to the candidate initial encoding parameters and the encoding reference parameters to obtain a frame type of the current video frame; Based on the frame type and the real-time transmission parameter, the preset encoding parameters of the current video frame are adjusted to obtain target encoding parameters.

3. The video frame processing method according to claim 2, characterized in that: The classifying the current video frame according to the candidate initial encoding parameter and the encoding reference parameter to obtain the frame type of the current video frame includes: Fusing the candidate initial coding parameters and the coding reference parameters to obtain fused reference parameters; When the parameter value of the fused reference parameter is the same as the preset value, determining that the frame type of the current video frame is a key frame; When the parameter value of the fused reference parameter is different from the preset value, it is determined that the frame type of the current video frame is not a key frame.

4. The video frame processing method according to claim 2, characterized in that: The adjusting the preset encoding parameters of the current video frame based on the frame type and the real-time transmission parameter to obtain the target encoding parameters includes: When the frame type is a key frame, obtaining a coding upscale parameter, and determining a target coding parameter of the current video frame based on the coding upscale parameter and the real-time transmission parameter; When the frame type is not a key frame, initial encoding parameters are extracted from the candidate initial encoding parameters, and target encoding parameters of the current video frame are determined based on the initial encoding parameters and the encoding reference parameters.

5. The video frame processing method according to claim 4, characterized in that: The determining, based on the coding amplification parameter and the real-time transmission parameter, a target coding parameter of the current video frame comprises: Extracting an initial coding parameter from the candidate initial coding parameters, and fusing the coding amplification parameter with the initial coding parameter to obtain an amplified coding parameter; The parameter value of the real-time transmission parameter is compared with the parameter value of the amplified encoding parameter, and based on the comparison result, the target encoding parameter of the current video frame is extracted from the amplified encoding parameter and the real-time transmission parameter.

6. The video frame processing method according to claim 1, characterized in that: The performing content detection on the current video frame based on the historical encoding information includes: Filtering out target historical coding information from the plurality of historical coding information, and extracting preset detection parameters corresponding to the target historical coding information from the preset detection parameter set; Based on the preset detection parameters and the target historical encoding information, content detection is performed on the current video frame to obtain a detection result.

7. The video frame processing method according to claim 6, characterized in that: The performing content detection on the current video frame based on the preset detection parameter and the target historical encoding information to obtain a detection result includes: Based on the preset detection parameters, verifying the target historical coding information; When the check passes, determining that the detection result is that there is a content change in the current video frame; When the check fails, it is determined that the detection result is that there is no content change in the current video frame.

8. The video frame processing method according to claim 1, characterized in that: After encoding the current video frame according to the target encoding parameter to obtain the encoded target video frame, the method further includes: Acquire current encoding information of a target video frame, and use the target video frame as the historical video frame, and use the current encoding information as the historical encoding information; Return to the step of performing content detection on the current video frame based on the historical encoding information until all video frames in the video to be processed are the current video frames, thereby obtaining an encoded video.

9. A video frame processing device, characterized in that: include: A first acquisition unit is used to acquire historical coding information corresponding to historical video frames in the video to be processed; A detection unit, configured to perform content detection on a current video frame based on the historical encoding information, wherein the current video frame includes a next video frame of the historical video frame in the video to be processed; A second acquisition unit, configured to acquire content transformation information and real-time transmission parameters of the current video frame when there is content transformation in the current video frame; an adjusting unit, configured to determine a coding reference parameter of the current video frame based on the content transformation information, and adjust a preset coding parameter of the current video frame based on the coding reference parameter and a real-time transmission parameter to obtain a target coding parameter; The encoding unit is used to encode the current video frame according to the target encoding parameter to obtain an encoded target video frame.

10. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores an application program, and the processor is used to run the application program in the memory to execute the steps in the video frame processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the video frame processing method according to any one of claims 1 to 8.

12. A computer program product, characterized in that The computer program product stores a computer program, and the computer program is suitable for being loaded by a processor to execute the video frame processing method according to any one of claims 1 to 8.

Images