Video encoding method, apparatus, device, and storage medium

Abstract

The present disclosure relates to a video encoding method, device, electronic equipment and storage medium. The method comprises: obtaining a video frame of a video to be encoded, and performing feature extraction on the video frame to obtain a video frame feature of the video frame in at least one evaluation dimension; determining a data feature of video data of the video based on the video frame feature of the video frame in each evaluation dimension; determining a target video encoding application matched with the data feature of the video data, and encoding the video data of the video by using the target video encoding application. In the present disclosure, the data feature of the video data can be obtained according to the video content of each video, and the video is encoded by using the video encoding application matched with the data feature, so that the video can be encoded by using the adaptive encoding application according to the features of the video content in a differentiated manner, the unnecessary encoding information can be avoided to be increased while the computing resources are saved, and the video encoding efficiency is effectively improved.

Description

Technical Field

[0001] This disclosure relates to the field of video processing technology, and in particular to a video encoding method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of computer technology, in order to transmit video efficiently, video encoding applications are often used to encode the video, and then the encoded video is transmitted. In the encoding process, multiple video encoding applications are often used.

[0003] In related technologies, to improve video encoding efficiency, pre-trained neural networks can be used to predict the encoding parameters of corresponding video encoding applications. This allows each video encoding application to obtain encoding parameters suitable for its current video encoding task, and then run the corresponding video encoding application according to these parameters. However, in practice, it has been found that the above approach still suffers from low video encoding efficiency. Summary of the Invention

[0004] This disclosure provides a video encoding method, apparatus, electronic device, and storage medium to at least solve the problem of low video encoding efficiency in related technologies. The technical solution of this disclosure is as follows:

[0005] According to a first aspect of the present disclosure, a video encoding method is provided, comprising:

[0006] Acquire video frames of the video to be encoded, and extract features from the video frames to obtain video frame features of the video frames in at least one evaluation dimension;

[0007] Based on the video frame features of the video frames under each evaluation dimension, the data features of the video data of the video are determined;

[0008] A target video encoding application that matches the data features of the video data is identified, and the video data of the video is encoded using the target video encoding application.

[0009] In one embodiment, the step of extracting features from the video frame to obtain video frame features of the video frame in at least one evaluation dimension includes:

[0010] The video frame is divided into blocks to obtain multiple image regions corresponding to the video frame;

[0011] At least one evaluation dimension of the video frame is obtained. According to the statistical rules of the evaluation dimension, the pixel values ​​of multiple pixels in each image region are statistically analyzed, and the video frame features under the evaluation dimension are determined based on the statistical results.

[0012] In one embodiment, if the evaluation dimension includes image texture, then the step of statistically analyzing the pixel values ​​of multiple pixels in each image region according to the statistical rules of the evaluation dimension, and determining the video frame features under the evaluation dimension based on the statistical results, includes:

[0013] Each image region is divided into blocks to obtain multiple image blocks;

[0014] Obtain the pixel value variance of multiple pixels in each image block, and determine the texture similarity of the image region based on the similarity of the pixel value variances of multiple image blocks;

[0015] The image texture features of the video frame are determined based on the texture similarity of multiple image regions, and are used as the video frame features of the video frame.

[0016] In one embodiment, if the evaluation dimension includes the number of image colors, then the step of statistically analyzing the pixel values ​​of multiple pixels in each image region according to the statistical rules of the evaluation dimension, and determining the video frame features under the evaluation dimension based on the statistical results, includes:

[0017] The number of colors in the image region is determined based on the pixel values ​​of each pixel in the image region;

[0018] The color quantity feature of the video frame is determined based on the number of colors in multiple image regions, and is used as the video frame feature of the video frame.

[0019] In one embodiment, if the evaluation dimension includes image noise, the step of statistically analyzing the pixel values ​​of multiple pixels in each image region according to the statistical rules of the evaluation dimension, and determining the video frame features under the evaluation dimension based on the statistical results, includes:

[0020] Obtain the pixel value residual for each pixel in each of the image regions; each pixel value residual is determined based on the difference between the pixel value of each pixel and the average pixel value of the corresponding image region;

[0021] Obtain the coefficient matrix corresponding to the residual values ​​of multiple pixels in each image region, and determine the image region noise of each image region based on the coefficients of a preset region in each coefficient matrix;

[0022] The image noise features of the video frame are determined based on noise in multiple image regions, and are used as the video frame features of the video frame.

[0023] In one embodiment, obtaining video frames of the video to be encoded includes:

[0024] The video to be encoded is acquired, and the video is sampled at a preset video frame sampling interval to extract multiple sampled video frames.

[0025] The extracted sampled video frames are downsampled to obtain multiple downsampled video frames; the resolution of the downsampled video frames is smaller than the resolution of the corresponding sampled video frames.

[0026] Multiple downsampled video frames are used as the video frames of the video.

[0027] In one embodiment, the data features of the video data include video frame features under each evaluation dimension, and the step of determining a target video coding application that matches the data features of the video data includes:

[0028] Obtain the feature matching conditions corresponding to each candidate video coding application, and determine the video frame features under the evaluation dimension associated with each candidate video coding application from the data features of the video data;

[0029] For each candidate video coding application, if the video frame features of the evaluation dimension associated with the candidate video coding application satisfy the feature matching conditions corresponding to the candidate video coding application, then the candidate video coding application is determined as the target video coding application.

[0030] According to a second aspect of the present disclosure, a video encoding apparatus is provided, comprising:

[0031] The video frame feature acquisition unit is configured to acquire video frames of the video to be encoded, and extract features from the video frames to obtain video frame features of the video frames in at least one evaluation dimension.

[0032] The video data feature acquisition unit is configured to determine the data features of the video data based on the video frame features of the video frame under each evaluation dimension.

[0033] The encoding application determination unit is configured to determine a target video encoding application that matches the data features of the video data, and to encode the video data of the video using the target video encoding application.

[0034] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0035] processor;

[0036] Memory used to store the processor's executable instructions;

[0037] The processor is configured to execute the instructions to implement the video encoding method as described in any of the preceding claims.

[0038] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform the video encoding method as described in any of the preceding claims.

[0039] According to a fifth aspect of the present disclosure, a computer program product is provided, the computer program product including instructions that, when executed by a processor of an electronic device, enable the electronic device to perform the video encoding method as described in any of the preceding claims.

[0040] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:

[0041] In this disclosure, data features of video data can be obtained based on the video content of each video, and video encoding applications that match the data features can be used for video encoding. This allows for differentiated encoding of videos using appropriate encoding applications based on the characteristics of the video content, avoiding the use of the same encoding application for all videos. This saves computing resources and avoids adding unnecessary encoding information, effectively improving video encoding efficiency.

[0042] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0043] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0044] Figure 1 This is a flowchart illustrating a video encoding method according to an exemplary embodiment.

[0045] Figure 2 This is a flowchart illustrating a step of counting pixels in a video frame according to an exemplary embodiment.

[0046] Figure 3 This is a flowchart illustrating a step of determining a target encoding application according to an exemplary embodiment.

[0047] Figure 4 This is a flowchart illustrating another video encoding method according to an exemplary embodiment.

[0048] Figure 5 This is a block diagram illustrating a video encoding apparatus according to an exemplary embodiment.

[0049] Figure 6This is a block diagram illustrating an electronic device according to an exemplary embodiment.

[0050] Figure 7 This is a block diagram illustrating another electronic device according to an exemplary embodiment. Detailed Implementation

[0051] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0052] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0053] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0054] Figure 1 This is a flowchart illustrating a video encoding method according to an exemplary embodiment. This embodiment uses the application of this method to a server as an example. The server may have a corresponding data storage system that stores one or more videos to be encoded. In practical applications, the data storage system may be integrated on server 104, or it may be located in the cloud or on another network server. The video to be encoded may originate from a terminal or from another server. For example, after receiving a video publishing request from a terminal, the video carried in the publishing request can be identified as the video to be encoded. Similarly, after receiving a video encoding request from a server, the video targeted by the video encoding request can be identified as the video to be encoded.

[0055] In a specific implementation, the server can acquire video frames of the video to be encoded and extract features from the video frames to obtain video frame features in at least one evaluation dimension. Then, based on the video frame features in each evaluation dimension, the server can determine the data features of the video data. Subsequently, the server can determine the target video encoding application that matches the data features of the video data and use the target video encoding application to encode the video data.

[0056] Understandably, this method can also be applied to terminals, as well as systems that include terminals and servers, and is implemented through the interaction between the terminals and servers. The server can be a standalone server or a server cluster consisting of multiple servers; the terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, etc.

[0057] In this embodiment, the following steps are included:

[0058] In step S110, video frames of the video to be encoded are obtained, and feature extraction is performed on the video frames to obtain video frame features of the video frames in at least one evaluation dimension.

[0059] As an example, an evaluation dimension can be understood as an evaluation perspective for determining the characteristics of a video frame. It can be used to evaluate the data composition characteristics or data quality of a video frame. For the same video frame, it can be evaluated from different perspectives to obtain the characteristics of the video frame under the corresponding perspective. In specific implementation, the evaluation dimensions of a video frame can include evaluation dimensions for the video frame itself and / or evaluation dimensions for multiple video frames. Among them, the evaluation dimensions for the video frame itself can be determined based on the image content of the video frame itself, such as including but not limited to: number of colors, image noise, image texture, composition method, sharpness, and intra-frame coding complexity. The evaluation dimensions for multiple video frames can be determined based on multiple video frames of the video, such as the content correlation between multiple video frames, inter-frame coding complexity, etc.

[0060] In this step, video frames of the video to be encoded can be obtained. These video frames can be obtained by the server after parsing the video to be encoded, or they can be obtained directly from the terminal or other servers. The video frames can be a portion of the video frames or all of the video frames.

[0061] After obtaining the video frame, at least one evaluation dimension can be acquired, and the video frame can be feature extracted according to the feature extraction method corresponding to the evaluation dimension. Based on the feature extraction results, the video frame features of the video frame under at least one evaluation dimension can be obtained.

[0062] In step S120, the data features of the video data are determined based on the video frame features of the video frames under each evaluation dimension.

[0063] As an example, video data can be the content data that constitutes a video, and the data characteristics of video data can characterize the data composition features or processing features of video data.

[0064] In practical applications, video data is composed of video frames. By analyzing these video frames, we can obtain the data characteristics of the video data. Accordingly, after obtaining the video frame characteristics under each evaluation dimension, we can summarize the characteristics of each video frame to obtain the data characteristics of the video frame.

[0065] In an optional embodiment, if the video frame features are the video frame features of multiple video frames under various evaluation dimensions, then for each evaluation dimension, the target video frame features under that evaluation dimension can be obtained based on the video frame features of multiple video frames under that evaluation dimension. The target video frame features under each evaluation dimension are then summarized to obtain the data features of the video data. For example, for an evaluation dimension, feature extraction is performed on n video frames respectively to obtain n video frame features. The average or median of the n video frame features can then be determined, and the target video frame features can be determined based on the average or median.

[0066] In step S130, a target video coding application that matches the data features of the video data is determined, and the video data of the video is encoded using the target video coding application.

[0067] As an example, a video coding application can also be called a video coding tool, which can refer to the application used in the video coding process. In practical applications, the video coding process can involve at least one of the following processing steps: prediction, transformation, quantization, reconstruction, and filtering. Each processing step can involve one or more video coding applications.

[0068] Specifically, after encoding a large number of videos, the inventors discovered that different video encoding applications exhibit varying video encoding efficiencies when encoding different videos. In some cases, for the same encoding quality requirements, using certain video encoding applications can actually introduce additional bitrate, reducing video encoding efficiency. In other words, different videos are suited to different video encoding applications. If multiple identical video encoding applications are used to encode different videos, on the one hand, it leads to increased server computation, consuming additional computing resources; on the other hand, it also adds unnecessary encoding information, reducing video encoding efficiency.

[0069] Based on this, after determining the data characteristics of the video data, a video encoding application that matches the data characteristics of the video data can be selected from multiple preset video encoding applications as the target video encoding application, and then the video data of the video can be encoded using the target video encoding application.

[0070] In the aforementioned video encoding method, video frames of the video to be encoded can be acquired, and feature extraction can be performed on the video frames to obtain video frame features under at least one evaluation dimension. Then, based on the video frame features under each evaluation dimension, the data features of the video data can be determined, a target video encoding application matching the data features of the video data can be identified, and the video data of the video can be encoded using the target video encoding application. In this disclosure, the data features of the video data can be obtained based on the video content of each video, and video encoding can be performed using a video encoding application matching the data features. This allows for differentiated encoding of videos using appropriate encoding applications based on the characteristics of the video content, avoiding the sampling of the same encoding application for all videos. This saves computational resources and avoids adding unnecessary encoding information, effectively improving video encoding efficiency.

[0071] In one exemplary embodiment, such as Figure 2 As shown, in step S110, feature extraction is performed on the video frame to obtain the video frame features in at least one evaluation dimension, which may include the following steps:

[0072] In step S210, the video frame is divided into blocks to obtain multiple image regions corresponding to the video frame.

[0073] In practice, video coding can be based on image regions in a video frame. For example, the video frame can be divided into multiple image regions, and steps such as prediction, transformation, quantization, reconstruction, and filtering can be performed on a per-image-region basis.

[0074] In this step, the video frame can be divided into blocks. For example, the video frame can be divided into multiple image regions according to a preset size, or it can be divided into multiple image regions according to the image content of the video frame (such as objects, people, etc. appearing in the video frame), or it can be divided into blocks according to other methods. Thus, multiple image regions corresponding to the video frame can be obtained according to the block division results. Each image region can include multiple pixels.

[0075] In step S220, at least one evaluation dimension of the video frame is obtained. According to the statistical rules of the evaluation dimension, the pixel values ​​of multiple pixels in each image region are statistically analyzed, and the video frame features under the evaluation dimension are determined based on the statistical results.

[0076] Furthermore, at least one evaluation dimension of the video frame and the statistical rules corresponding to each evaluation dimension can be obtained. For example, the evaluation dimension may include at least one of the following: image texture, number of image colors, image noise, and video frame encoding complexity.

[0077] For each evaluation dimension, the pixel values ​​of multiple pixels in each image region can be statistically analyzed according to the statistical rules of that evaluation dimension to obtain statistical results. For example, for each image region, the pixel values ​​of multiple pixels in that image region can be calculated according to the statistical rules to obtain the calculation results for that image region. Then, the calculation results of multiple image regions can be combined to obtain statistical results. After obtaining the statistical results, the video frame features under that evaluation dimension can be determined based on the statistical results.

[0078] In this embodiment, on the one hand, by dividing the video frame into multiple image regions and performing statistics on each image region to obtain statistical results, it can be adapted to the method of processing by image region in the video encoding process, effectively improving the reliability of the obtained video frame features. On the other hand, by performing statistics on the pixel values ​​of each image region in the video frame, the video frame features can be quickly obtained based on the statistical results, which can simplify the video frame feature extraction process, reduce the computing resources consumed when determining the target video encoding application, reduce the amount of computation, and take into account the evaluation efficiency while evaluating whether the use of a specified video encoding tool can achieve bitrate savings.

[0079] In an exemplary embodiment, if the evaluation dimension includes image texture, then according to the statistical rules of the evaluation dimension, the pixel values ​​of multiple pixels in each image region are statistically analyzed, and the video frame features under the evaluation dimension are determined based on the statistical results. This may include the following steps:

[0080] Each image region is divided into blocks to obtain multiple image blocks; the pixel value variance of multiple pixels in each image block is obtained, and the texture similarity of the image region is determined based on the similarity of the pixel value variances of multiple image blocks; the image texture features of the video frame are determined based on the texture similarity of multiple image regions, and used as the video frame features of the video frame.

[0081] In practical applications, if at least one evaluation dimension includes image texture, each image region can be further divided into multiple image blocks, each containing multiple pixels. For example, a video frame with a resolution of 1920x1080 can be divided into 8x8 blocks, resulting in 240x135 = 32400 image regions. Then, each 8x8 image region can be further subdivided into four 4x4 image blocks.

[0082] For each image block, the pixel value variance of the image block can be obtained based on the pixel values ​​of multiple points in the image block. Then, based on the pixel value variances of each image block in the same image region, the similarity between multiple pixel value variances can be determined, and the texture similarity of the image region can be obtained based on the similarity. For example, the similarity of pixel value variances can be determined as the texture similarity of the image region.

[0083] For example, continuing the previous example, after an 8x8 image region is divided into four 4x4 image blocks, for each 4x4 image block, the pixel value variance of that image block can be obtained based on the pixel values ​​of its four pixels. The pixel value variances of the four image blocks can then be recorded as F1, F2, F3, and F4. The similarity of the multiple pixel value variances within the 8x8 image region can then be determined based on F1, F2, F3, and F4. For example, this similarity can be determined as follows:

[0084] F=((F1==F2)+(F1==F3)+(F2==F4)+(F3==F4)) / 4

[0085] The "==" operator is a relational operator used to determine whether two values ​​are the same. It returns 1 when the left-hand value is the same as the right-hand value, and 0 otherwise.

[0086] After obtaining the texture similarity of each image region, the texture similarity of multiple image regions can be combined to determine the image texture features of the video frame, and this value can be used as the video frame feature. Specifically, for example, after obtaining the texture similarity of N image regions, the average value of multiple texture similarities can be obtained, and this average value can be used as the image texture feature of the video frame.

[0087] In this embodiment, by statistically analyzing the pixel values ​​of multiple pixels in each image region, the image texture features of the video frame can be obtained, providing a basis for determining the appropriate video encoding application based on the image texture characteristics of the video frame.

[0088] In an exemplary embodiment, if the evaluation dimension includes the number of image colors, then according to the statistical rules of the evaluation dimension, the pixel values ​​of multiple pixels in each image region are statistically analyzed, and the video frame features under the evaluation dimension are determined based on the statistical results. This may include the following steps:

[0089] The number of colors in an image region is determined based on the pixel values ​​of each pixel in that region. The number of colors in multiple image regions is then used to determine the color quantity features of a video frame, which are then used as the video frame features.

[0090] As an example, the number of colors can be the number of colors contained in an image region.

[0091] Specifically, if the evaluation dimension includes the number of image colors, for each image region, the number of colors in the image region can be determined based on the pixel values ​​of each pixel in the image region. For example, multiple preset pixel value ranges can be obtained, and each pixel value range can correspond to a color. For each pixel in the image region, the color corresponding to that pixel can be determined based on the pixel value range that the pixel value falls into.

[0092] After obtaining the number of colors in each image region, the color quantity features of the video frame can be determined by combining the color quantity values ​​of multiple image regions. For example, the average color quantity value of multiple image regions can be obtained, that is, the color quantity values ​​of multiple image regions are summed and then divided by the total number of image regions. Therefore, the color quantity features of the video frame can be used as the video frame features.

[0093] In this embodiment, by counting the number of colors in multiple image regions, the color quantity characteristics used to evaluate video colors can be quickly obtained.

[0094] In an exemplary embodiment, if the evaluation dimension includes image noise, the pixel values ​​of multiple pixels in each image region are statistically analyzed according to the statistical rules of the evaluation dimension, and the video frame features under the evaluation dimension are determined based on the statistical results. This may include the following steps:

[0095] Obtain the pixel value residual for each image region; obtain the coefficient matrix corresponding to multiple pixel value residuals in each image region, and determine the image region noise for each image region based on the coefficients of the preset region in each coefficient matrix; determine the image noise features of the video frame based on the multiple image region noises, and use them as the video frame features of the video frame.

[0096] The residual for each pixel value is determined based on the difference between the pixel value of each pixel and the average pixel value of the corresponding image region.

[0097] In a specific implementation, the evaluation dimension can include image noise. In one example, the evaluation dimension is the partial video frame features of image noise, which can reflect the complexity of video frame noise, or it can reflect the image texture complexity of video frames.

[0098] If the evaluation dimension includes image noise, for each image region, the average pixel value of multiple pixels in the image region can be obtained. Then, for each pixel in the image region, the pixel value residual can be obtained based on the difference between the pixel value of the pixel and the average pixel value. For example, if the image region includes 8*8 pixels, an 8*8 pixel value residual matrix can be obtained, where each element in the matrix is ​​a pixel value residual.

[0099] For each image region, a coefficient matrix corresponding to the residual values ​​of multiple pixel values ​​in the image region can be obtained. Specifically, for each image region, after obtaining the residual values ​​of multiple pixel values ​​in that region, the residual values ​​can be transformed, and then the transformation result can be quantized based on a preset quantization step size. In this way, a coefficient matrix can be generated based on the quantization result of the residual values ​​of multiple pixel values.

[0100] After obtaining the coefficient matrix corresponding to the image region, the coefficients of a preset region can be obtained. For example, the coefficients of the preset region may include at least one of the following: a low-frequency region and a high-frequency region. The low-frequency region may be the upper left region of the coefficient matrix, and the high-frequency region may be a region outside the low-frequency region. After determining the coefficients of the preset region, the absolute values ​​of each coefficient in the preset region can be summed, and the noise of the current image region can be determined based on the summation result, thus obtaining the image region noise. For example, after summing the absolute values ​​of each coefficient in the low-frequency region, the low-frequency noise of the image region can be determined based on the summation result. The summation result of the absolute values ​​of each coefficient in the high-frequency region can be used to determine the high-frequency noise of the image region.

[0101] After obtaining the image region noise for each image region, the image noise features of the video frame can be determined based on the image region noise of multiple image regions, and used as the video frame features. Specifically, for example, after obtaining the low-frequency noise or high-frequency noise of multiple image regions, the average value of the low-frequency noise or high-frequency noise of multiple image regions can be determined, and this average value can be used as the image noise feature.

[0102] In this embodiment, the pixel values ​​of pixels in the image region of a video frame can be statistically analyzed to obtain pixel value residuals. Then, the image noise characteristics of the video frame can be quickly determined based on multiple pixel value residuals, providing a decision-making basis for subsequently determining suitable video encoding applications.

[0103] In an exemplary embodiment, obtaining video frames of the video to be encoded in step S110 may include the following steps:

[0104] The video to be encoded is acquired, and the video is sampled at a preset video frame sampling interval to extract multiple sampled video frames; the extracted multiple sampled video frames are downsampled to obtain multiple downsampled video frames; the multiple downsampled video frames are used as the video frames of the video.

[0105] The resolution of the downsampled video frame is smaller than the resolution of the corresponding sampled video frame.

[0106] The video frame sampling interval can be used to indicate that after extracting one frame from multiple original video frames, the next frame is extracted after an interval of N frames. Alternatively, it can be used to indicate that one video frame is extracted every N frames (e.g., one frame is extracted every 8 frames), where N is a positive integer.

[0107] In practical applications, the server can acquire the video to be encoded and obtain preset video frame sampling parameters. These parameters can include the video frame sampling interval and the downsampling rate. Then, the server can extract multiple video frames from the original video according to the sampling interval, using these extracted frames as sampled video frames.

[0108] For each sampled video frame, it can be downsampled according to a preset downsampling ratio. For example, a 1920x1080 resolution sampled video frame can be downsampled by a factor of 2 to obtain a 960x540 resolution video frame. After downsampling multiple sampled video frames, the resulting multiple downsampled video frames can be used as video frames for subsequent feature identification, while other video frames in the video (video frames other than those obtained after downsampling) do not need to have their features extracted.

[0109] In this embodiment, by extracting frames from the video according to a preset video frame sampling interval and downsampling multiple sampled video frames, video sampling can be achieved in both the temporal and spatial domains. This ensures the reliability of the video frame feature extraction results while reducing the computational complexity in the video frame feature acquisition process. When it is necessary to calculate video frame features of multiple evaluation dimensions, the computational load can be significantly reduced, meeting the needs of various computational complexity scenarios.

[0110] In one exemplary embodiment, the data features of the video data include video frame features under each evaluation dimension, such as... Figure 3 As shown, in step S130, determining the target video coding application that matches the data features of the video data may include:

[0111] In step S310, the feature matching conditions corresponding to each candidate video coding application are obtained, and the video frame features associated with the evaluation dimension of each candidate video coding application are determined from the data features of the video data.

[0112] In practical implementation, the video frame features associated with different video coding applications can differ. Specifically, the coding steps involved in different video coding applications may differ. For example, video coding application A is for the filtering step, while video coding application B is for the transform step. Therefore, the data processed or the evaluation dimensions of interest during the execution of these coding steps may also differ. Similarly, for different video coding applications performing the same coding step, such as video coding applications C and D, both used in the reconstruction step of the coding process, but with different data processing logic, the evaluation dimensions associated with video coding applications C and D, or the video frame features under the same evaluation dimension, may also differ. In other words, the evaluation dimensions for the association between different video coding applications can be different, and even under the same evaluation dimension, the video frame features matching the video coding applications can differ.

[0113] In this step, multiple candidate video coding applications can be obtained. These applications may involve various steps in the video coding process, including but not limited to prediction, transformation, quantization, reconstruction, and filtering. Furthermore, feature matching conditions (such as thresholds or intervals) corresponding to each candidate video coding application can be determined, and video frame features associated with each video coding application under the evaluation dimension can be identified from the data features of the video data.

[0114] In step S320, for each candidate video coding application, if the video frame features of the evaluation dimension associated with the candidate video coding application satisfy the feature matching conditions corresponding to the candidate video coding application, then the candidate video coding application is determined as the target video coding application.

[0115] Once the video frame features associated with each candidate video coding application under the evaluation dimensions are obtained, it can be determined whether the video frame features meet the feature matching conditions. For example, if the evaluation dimensions associated with video coding application A are dimension 1 and dimension 2, and its preset feature matching condition is a feature threshold, then video frame features a and b under dimension 1 and dimension 2 can be obtained, and it can be determined whether video frame features a and video frame features b are greater than their respective feature thresholds.

[0116] If the video frame features of a candidate video coding application meet the feature matching conditions under the associated evaluation dimension, then the candidate video coding application can be identified as the target video coding application. If the video frame features of a candidate video coding application do not meet the feature matching conditions, then the candidate video coding application is not compatible with the video data of the current video, and the candidate video coding application can be left unused to encode the current video.

[0117] In this embodiment, the video frame features associated with each candidate video coding application can be compared with the corresponding feature matching conditions. When the video frame features associated with the evaluation dimension of the candidate video coding application meet the corresponding feature matching conditions, the candidate video coding application is determined as the target video coding application. This allows for the rapid acquisition of the target video coding application that matches the video content of the current video from multiple candidate video coding applications, avoiding the introduction of additional video coding applications during the encoding process. This saves computing resources while avoiding the introduction of encoding information from incompatible video coding applications, effectively improving encoding efficiency.

[0118] To enable those skilled in the art to better understand the above steps, the following example illustrates the embodiments of this disclosure, but it should be understood that the embodiments of this disclosure are not limited thereto.

[0119] like Figure 4 As shown, video data can be acquired, which may include multiple original video frames of the video. Then, multiple original video frames can be sampled according to a preset video frame sampling interval and downsampling rate to obtain multiple sampled video frames. Furthermore, initial encoding parameters can also be acquired, which may include multiple candidate video encoding applications involved in the preset encoding process and feature matching conditions for each candidate video encoding application.

[0120] Then, video content analysis can be performed on multiple adopted video frames to obtain the video frame features of each video frame under different evaluation dimensions, and the evaluation dimension associated with each candidate video coding application can be determined. The video frame features under this evaluation dimension are then compared with the feature matching conditions. If the video frame features meet the feature matching conditions, the video coding application can be used as the target video coding application to continue using it in subsequent encoding processes. If the video frame features do not meet the feature matching conditions, the encoding parameters of the video coding application can be adjusted. The adjusted encoding parameters are used to indicate that the application should not be used for video encoding, thus obtaining optimized encoding parameters.

[0121] Then, the original video frames and optimized encoding parameters can be input into multiple preset video encoding applications. Each video encoding application can determine whether to encode the video frames based on the input parameters. For example, other video encoding applications not identified as the target video encoding application can choose not to process the received video frames based on the input encoding parameters. After encoding by the video encoding applications, a compressed video stream can be obtained. In this disclosure, based on the analysis results of video frame characteristics, a relatively small amount of computation can be used to estimate whether using preset video encoding applications during the video encoding process has a bitrate saving effect. Based on the estimation results, the corresponding video encoding application switches (i.e., whether to use the video encoding application for encoding) can be adjusted to achieve a balance between video encoding efficiency and encoding time.

[0122] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0123] It is understood that the same / similar parts between the various embodiments of the methods described above in this specification can be referred to each other. Each embodiment focuses on the differences from other embodiments, and relevant parts can be referred to the description of other method embodiments.

[0124] Based on the same inventive concept, this disclosure also provides a video encoding apparatus for implementing the video encoding method described above.

[0125] Figure 5 This is a block diagram illustrating a video encoding apparatus according to an exemplary embodiment. (Refer to...) Figure 5 The device includes a video frame feature acquisition unit 501, a video data feature acquisition unit 502, and an encoding application determination unit 503.

[0126] The video frame feature acquisition unit 501 is configured to acquire video frames of the video to be encoded and extract features from the video frames to obtain video frame features of the video frames in at least one evaluation dimension.

[0127] The video data feature acquisition unit 502 is configured to determine the data features of the video data of the video based on the video frame features of the video frame under each evaluation dimension.

[0128] The encoding application determination unit 503 is configured to determine a target video encoding application that matches the data features of the video data, and to encode the video data of the video using the target video encoding application.

[0129] In one exemplary embodiment, the video frame feature acquisition unit 501 includes:

[0130] The segmentation module is configured to perform segmentation of the video frame to obtain multiple image regions corresponding to the video frame;

[0131] The pixel value statistics module is configured to acquire at least one evaluation dimension of the video frame, perform statistical analysis on the pixel values ​​of multiple pixels in each image region according to the statistical rules of the evaluation dimension, and determine the video frame features under the evaluation dimension based on the statistical results.

[0132] In an exemplary embodiment, if the evaluation dimension includes image texture, the pixel value statistics module is specifically configured to perform:

[0133] Each image region is divided into blocks to obtain multiple image blocks;

[0134] Obtain the pixel value variance of multiple pixels in each image block, and determine the texture similarity of the image region based on the similarity of the pixel value variances of multiple image blocks;

[0135] The image texture features of the video frame are determined based on the texture similarity of multiple image regions, and are used as the video frame features of the video frame.

[0136] In an exemplary embodiment, if the evaluation dimension includes the number of image colors, then the pixel value statistics module is specifically configured to perform:

[0137] The number of colors in the image region is determined based on the pixel values ​​of each pixel in the image region;

[0138] The color quantity feature of the video frame is determined based on the number of colors in multiple image regions, and is used as the video frame feature of the video frame.

[0139] In an exemplary embodiment, if the evaluation dimension includes image noise, the pixel value statistics module is specifically configured to perform:

[0140] Obtain the pixel value residual for each pixel in each of the image regions; each pixel value residual is determined based on the difference between the pixel value of each pixel and the average pixel value of the corresponding image region;

[0141] Obtain the coefficient matrix corresponding to the residual values ​​of multiple pixels in each image region, and determine the image region noise of each image region based on the coefficients of a preset region in each coefficient matrix;

[0142] The image noise features of the video frame are determined based on noise in multiple image regions, and are used as the video frame features of the video frame.

[0143] In one exemplary embodiment, the video frame feature acquisition unit 501 includes:

[0144] The video frame extraction module is configured to acquire the video to be encoded and extract multiple sampled video frames from the video according to a preset video frame sampling interval.

[0145] The downsampling module is configured to downsample multiple extracted sampled video frames to obtain multiple downsampled video frames; the resolution of the downsampled video frames is smaller than the resolution of the corresponding sampled video frames.

[0146] The video frame desampling module is configured to take multiple downsampled video frames as video frames of the video.

[0147] In one exemplary embodiment, the encoding application determination unit 503 is configured to perform:

[0148] Obtain the feature matching conditions corresponding to each candidate video coding application, and determine the video frame features under the evaluation dimension associated with each candidate video coding application from the data features of the video data;

[0149] For each candidate video coding application, if the video frame features of the evaluation dimension associated with the candidate video coding application satisfy the feature matching conditions corresponding to the candidate video coding application, then the candidate video coding application is determined as the target video coding application.

[0150] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0151] Each module in the aforementioned video encoding device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware within or independently of the processor in a computer device, or stored in software within the memory of the computer device, so that the processor can call and execute the operations corresponding to each module.

[0152] Figure 6 This is a block diagram illustrating an electronic device 600 for implementing a video encoding method according to an exemplary embodiment. For example, the electronic device 600 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0153] Reference Figure 6 The electronic device 600 may include one or more of the following components: processing component 602, memory 604, power supply component 606, multimedia component 608, audio component 610, input / output (I / O) interface 612, sensor component 614, and communication component 616.

[0154] Processing component 602 typically controls the overall operation of electronic device 600, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 602 may include one or more processors 620 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 602 may include one or more modules to facilitate interaction between processing component 602 and other components. For example, processing component 602 may include a multimedia module to facilitate interaction between multimedia component 608 and processing component 602.

[0155] Memory 604 is configured to store various types of data to support the operation of electronic device 600. Examples of such data include instructions for any application or method operating on electronic device 600, contact data, phonebook data, messages, pictures, videos, etc. Memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, optical disk, or graphene storage.

[0156] Power supply component 606 provides power to various components of electronic device 600. Power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 600.

[0157] Multimedia component 608 includes a screen that provides an output interface between the electronic device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 608 includes a front-facing camera and / or a rear-facing camera. When the electronic device 600 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0158] Audio component 610 is configured to output and / or input audio signals. For example, audio component 610 includes a microphone (MIC) configured to receive external audio signals when electronic device 600 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 604 or transmitted via communication component 616. In some embodiments, audio component 610 also includes a speaker for outputting audio signals.

[0159] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0160] Sensor assembly 614 includes one or more sensors for providing state assessments of various aspects of electronic device 600. For example, sensor assembly 614 can detect the on / off state of electronic device 600, the relative positioning of components such as the display and keypad of electronic device 600, changes in position of electronic device 600 or its components, the presence or absence of user contact with electronic device 600, orientation or acceleration / deceleration of device 600, and temperature changes of electronic device 600. Sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 614 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 614 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0161] Communication component 616 is configured to facilitate wired or wireless communication between electronic device 600 and other devices. Electronic device 600 can access wireless networks based on communication standards, such as WiFi, carrier networks (such as 2G, 3G, 4G, or 5G), or combinations thereof. In one exemplary embodiment, communication component 616 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 616 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0162] In an exemplary embodiment, the electronic device 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0163] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, which can be executed by a processor 620 of an electronic device 600 to perform the above-described method. For example, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0164] In an exemplary embodiment, a computer program product is also provided, which includes instructions that can be executed by a processor 620 of an electronic device 600 to perform the above-described method.

[0165] Figure 7 This is a block diagram illustrating an electronic device 700 for implementing a video encoding method according to an exemplary embodiment. For example, the electronic device 700 can be a server. (Refer to...) Figure 7 The electronic device 700 includes a processing component 720, which further includes one or more processors, and memory resources represented by memory 722 for storing instructions, such as application programs, that can be executed by the processing component 720. The application programs stored in memory 722 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 720 is configured to execute instructions to perform the methods described above.

[0166] Electronic device 700 may further include: a power supply component 724 configured to perform power management of electronic device 700, a wired or wireless network interface 726 configured to connect electronic device 700 to a network, and an input / output (I / O) interface 728. Electronic device 700 may operate on an operating system stored in memory 722, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, or similar.

[0167] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 722 including instructions, which can be executed by a processor of an electronic device 700 to perform the above-described method. The storage medium may be a computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0168] In an exemplary embodiment, a computer program product is also provided, the computer program product including instructions that can be executed by a processor of an electronic device 700 to perform the above-described method.

[0169] It should be noted that the above-mentioned apparatus, electronic equipment, computer-readable storage medium, computer program product, etc., may also include other implementation methods according to the description of the method embodiments. For specific implementation methods, please refer to the description of the relevant method embodiments, which will not be elaborated here.

[0170] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0171] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A video encoding method, characterized in that, include: Acquire video frames of the video to be encoded, and extract features from the video frames to obtain video frame features of the video frames in at least one evaluation dimension; Based on the video frame features of the video frames under each evaluation dimension, the data features of the video data of the video are determined; The feature matching conditions corresponding to each candidate video coding application are obtained, and the video frame features associated with the evaluation dimension of each candidate video coding application are determined from the data features of the video data. For each candidate video coding application, if the video frame features associated with the evaluation dimension of the candidate video coding application satisfy the feature matching conditions corresponding to the candidate video coding application, then the candidate video coding application is determined as the target video coding application; if the video frame features associated with the evaluation dimension of the candidate video coding application do not satisfy the feature matching conditions, then the candidate video coding application is not used to encode the video. The video data of the video is encoded using the target video coding application. The encoding steps are different for different video coding applications, and the multiple encoding steps include prediction, transformation, quantization, reconstruction, and filtering.

2. The method according to claim 1, characterized in that, The step of extracting features from the video frame to obtain the video frame features in at least one evaluation dimension includes: The video frame is divided into blocks to obtain multiple image regions corresponding to the video frame; At least one evaluation dimension of the video frame is obtained. According to the statistical rules of the evaluation dimension, the pixel values ​​of multiple pixels in each image region are statistically analyzed, and the video frame features under the evaluation dimension are determined based on the statistical results.

3. The method according to claim 2, characterized in that, If the evaluation dimension includes image texture, then the step of statistically analyzing the pixel values ​​of multiple pixels in each image region according to the statistical rules of the evaluation dimension, and determining the video frame features under the evaluation dimension based on the statistical results, includes: Each image region is divided into blocks to obtain multiple image blocks; Obtain the pixel value variance of multiple pixels in each image block, and determine the texture similarity of the image region based on the similarity of the pixel value variances of multiple image blocks; The image texture features of the video frame are determined based on the texture similarity of multiple image regions, and are used as the video frame features of the video frame.

4. The method according to claim 2, characterized in that, If the evaluation dimension includes the number of image colors, then the step of statistically analyzing the pixel values ​​of multiple pixels in each image region according to the statistical rules of the evaluation dimension, and determining the video frame features under the evaluation dimension based on the statistical results, includes: The number of colors in the image region is determined based on the pixel values ​​of each pixel in the image region; The color quantity feature of the video frame is determined based on the number of colors in multiple image regions, and is used as the video frame feature of the video frame.

5. The method according to claim 2, characterized in that, If the evaluation dimension includes image noise, then the step of statistically analyzing the pixel values ​​of multiple pixels in each image region according to the statistical rules of the evaluation dimension, and determining the video frame features under the evaluation dimension based on the statistical results, includes: Obtain the pixel value residual for each pixel in each of the image regions; each pixel value residual is determined based on the difference between the pixel value of each pixel and the average pixel value of the corresponding image region; Obtain the coefficient matrix corresponding to the residual values ​​of multiple pixels in each image region, and determine the image region noise of each image region based on the coefficients of a preset region in each coefficient matrix; The image noise features of the video frame are determined based on noise in multiple image regions, and are used as the video frame features of the video frame.

6. The method according to claim 1, characterized in that, The process of obtaining video frames of the video to be encoded includes: The video to be encoded is acquired, and the video is sampled at a preset video frame sampling interval to extract multiple sampled video frames. The extracted sampled video frames are downsampled to obtain multiple downsampled video frames; the resolution of the downsampled video frames is smaller than the resolution of the corresponding sampled video frames. Multiple downsampled video frames are used as the video frames of the video.

7. A video encoding device, characterized in that, include: The video frame feature acquisition unit is configured to acquire video frames of the video to be encoded, and extract features from the video frames to obtain video frame features of the video frames in at least one evaluation dimension. The video data feature acquisition unit is configured to determine the data features of the video data based on the video frame features of the video frame under each evaluation dimension. The encoding application determination unit is configured to acquire feature matching conditions corresponding to each candidate video encoding application, and determine the video frame features under the evaluation dimension associated with each candidate video encoding application from the data features of the video data; for each candidate video encoding application, if the video frame features under the evaluation dimension associated with the candidate video encoding application satisfy the feature matching conditions corresponding to the candidate video encoding application, then the candidate video encoding application is determined as the target video encoding application; if the video frame features under the evaluation dimension associated with the candidate video encoding application do not satisfy the feature matching conditions, then the candidate video encoding application is not used to encode the video; the video data of the video is encoded using the target video encoding application; wherein, the encoding steps corresponding to different video encoding applications are different, and the multiple encoding steps include prediction, transformation, quantization, reconstruction, and filtering.

8. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the method as described in any one of claims 1 to 6.

Images