Multimedia data coding method, video data coding method and respective devices

Abstract

The embodiment of the invention provides a multimedia data coding method and device, a video data coding method and device, and the multimedia data coding method comprises the steps: determining a current quality index value based on a code rate quality corresponding change relation of multimedia data and a current coding parameter value of a target coding parameter, and adjusting the current coding parameter value of the target coding parameter according to a target adjustment strategy, and adjusting the current coding parameter value to obtain an updated current coding parameter value, determining an updated current quality index value based on the code rate quality corresponding change relation and the updated current coding parameter value, and outputting the target quality index value when the difference between the updated current quality index value and the target quality index value meets a preset output condition. And taking the updated current coding parameter value as the target coding parameter value for coding the multimedia data, thereby obtaining the target coding parameter value which can ensure stable quality of the multimedia data and avoid code rate waste.

Description

[0001] This application claims the priority of the Chinese patent application filed with the China Patent Office on July 25, 2024, with application number "202411007484.2" and invention name "Multimedia data encoding method, video data encoding method and respective devices", all contents of which are recorded in this application by reference. Technical Field

[0002] The embodiments of the present specification relate to the field of computer technology, and more particularly to a multimedia data encoding method, a video data encoding method, and respective devices. Background Art

[0003] Multimedia data coding is a core component of multimedia communication, storage and processing technology, and is essential for modern Internet and streaming media services. Multimedia data coding includes the process of converting original multimedia data into a target coding format, and can also include the process of converting already encoded multimedia data from one coding format to another. Through coding, multimedia data can be transmitted more efficiently on the network or stored on the storage medium, while trying to maintain or optimize its quality.

[0004] At present, most major multimedia platforms use fixed encoding parameter modes to encode multimedia data, which may lead to unstable quality and bit rate waste of the encoded multimedia data. Therefore, how to encode multimedia data to ensure quality and avoid bit rate waste is an urgent problem to be solved. Summary of the invention

[0005] In view of this, the embodiments of this specification provide a multimedia data encoding method. One or more embodiments of this specification also relate to a video data encoding method, a multimedia data encoding device, a video data encoding device, a computing device, a computer-readable storage medium, and a computer program product to solve the technical defects of unstable multimedia data encoding quality and bit rate waste in the prior art.

[0006] According to a first aspect of an embodiment of the present specification, a multimedia data encoding method is provided, comprising: determining a current quality index value based on a rate-quality corresponding change relationship of multimedia data and a current encoding parameter value of a target encoding parameter, wherein the rate-quality corresponding change relationship comprises: a plurality of encoding parameter values ​​of a target encoding parameter, a corresponding change relationship between a rate and a quality index value, and the current encoding parameter value is any one of the plurality of encoding parameter values; adjusting the current encoding parameter value according to a target adjustment strategy to obtain an updated current encoding parameter value; determining an updated current quality index value based on the rate-quality corresponding change relationship and the updated current encoding parameter value; and using the updated current encoding parameter value as a target encoding parameter value for encoding the multimedia data when the difference between the updated current quality index value and the target quality index value meets a preset output condition.

[0007] According to a second aspect of an embodiment of the present specification, a video data encoding method is provided, comprising: determining a current quality index value based on a corresponding change relationship between bit rate and quality of video data and a current CRF parameter value of a CRF parameter, wherein the corresponding change relationship between bit rate and quality comprises: a corresponding change relationship between multiple CRF parameter values, bit rate and quality index values ​​of the CRF parameter, and the current CRF parameter value is any one of the multiple CRF parameter values; adjusting the current CRF parameter value according to a target adjustment strategy to obtain an updated current CRF parameter value; determining an updated current quality index value based on the corresponding change relationship between bit rate and quality and the updated current CRF parameter value; and using the updated current CRF parameter value as the target CRF parameter value for encoding the video data when the difference between the updated current quality index value and the target quality index value meets a preset output condition.

[0008] According to a third aspect of an embodiment of the present specification, a multimedia data encoding device is provided, comprising: a current value determination module, configured to determine a current quality index value based on a rate-quality corresponding change relationship of multimedia data and a current encoding parameter value of a target encoding parameter, wherein the rate-quality corresponding change relationship comprises: a plurality of encoding parameter values ​​of a target encoding parameter, a corresponding change relationship between a rate and a quality index value, and the current encoding parameter value is any one of the plurality of encoding parameter values; a parameter value update module, configured to adjust the current encoding parameter value according to a target adjustment strategy to obtain an updated current encoding parameter value; a quality index update module, configured to determine an updated current quality index value based on the rate-quality corresponding change relationship and the updated current encoding parameter value; and an output module, configured to use the updated current encoding parameter value as a target encoding parameter value for encoding the multimedia data when a difference between the updated current quality index value and the target quality index value meets a preset output condition.

[0009] According to a fourth aspect of the embodiments of this specification, a video data encoding device is provided, including: a current determination module, configured to determine a current quality index value based on a corresponding change relationship between bit rate and quality of video data and a current CRF parameter value of a CRF parameter, wherein the corresponding change relationship between bit rate and quality includes: a plurality of CRF parameter values ​​of the CRF parameter, a corresponding change relationship between bit rate and quality index value, and the current CRF parameter value is any one of the plurality of CRF parameter values; a CRF update module, configured to adjust the current CRF parameter value according to a target adjustment strategy to obtain an updated current CRF parameter value; a quality update module, configured to determine an updated current quality index value based on the corresponding change relationship between bit rate and quality and the updated current CRF parameter value; and a target CRF output module, configured to use the updated current CRF parameter value as a target CRF parameter value for encoding the video data when the difference between the updated current quality index value and the target quality index value meets a preset output condition.

[0010] According to the fifth aspect of the embodiments of this specification, a computing device is provided, including: a memory and a processor; the memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method described in any embodiment of this specification.

[0011] According to the sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores a computer program / instruction, and when the computer program / instruction is executed by a processor, the steps of the method described in any embodiment of this specification are implemented.

[0012] According to a seventh aspect of the embodiments of this specification, a computer program product is provided, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the method described in any embodiment of this specification.

[0013] An embodiment of the present specification implements a multimedia data encoding method. Since the method determines a current quality index value based on a corresponding change relationship between bit rate and quality of multimedia data and a current encoding parameter value, wherein the corresponding change relationship between bit rate and quality includes: a plurality of encoding parameter values ​​of a target encoding parameter, a corresponding change relationship between bit rate and quality index value, on this basis, the method adjusts the current encoding parameter value according to a target adjustment strategy to obtain an updated current encoding parameter value, and determines an updated current quality index value based on the corresponding change relationship between bit rate and quality and the updated current encoding parameter value, so that when the difference between the updated current quality index value and the target quality index value meets a preset output condition, the updated current encoding parameter value is used as the target encoding parameter value for encoding the multimedia data.

[0014] It can be seen that the method combines the corresponding change relationship between the encoding parameter value, the bit rate and the quality index value to analyze the encoding parameter value of the multimedia data, and achieves the purpose of correspondingly adjusting the bit rate and quality by adjusting the current encoding parameter value, so as to find a better balance point between quality and bit rate as needed, and obtain the target encoding parameter value when the difference between the updated current quality index value and the target quality index value meets the preset output condition, and can determine the target encoding parameter value within a narrow range relative to the target quality index value, thereby ensuring the stability of the multimedia data quality and avoiding bit rate waste. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is an architecture diagram of a multimedia data encoding system provided by an embodiment of this specification;

[0016] Figure 2 is a flowchart of a multimedia data encoding method provided by an embodiment of this specification;

[0017] Figure 3a is a schematic diagram of the model structure of a bit rate quality prediction model provided by an embodiment of this specification;

[0018] Figure 3b is a schematic diagram of a user interaction interface provided by an embodiment of this specification;

[0019] Figure 4 is a flowchart of a processing process of a multimedia data encoding method provided by an embodiment of the present specification in a video application;

[0020] Figure 5 is a structural schematic diagram of a multimedia data encoding device provided by an embodiment of this specification;

[0021] Figure 6 is a flow chart of a video data encoding method provided by an embodiment of this specification;

[0022] Figure 7 It is a flowchart of a processing process of applying a video data encoding method provided by an embodiment of this specification in an adaptive CRF encoding parameter strategy;

[0023] Figure 8 is a mass distribution schematic diagram provided by an embodiment of this specification;

[0024] Fig. 9 is a structural diagram of a video data encoding device provided by an embodiment of this specification;

[0025] Fig.10 It is a structural block diagram of a computing device provided by one embodiment of this specification. DETAILED DESCRIPTION

[0026] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0027] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0028] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0029] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0030] First, the terms involved in one or more embodiments of this specification are explained.

[0031] Multimedia data refers to data that combines multiple types of media content, including audio, video, animation, etc.

[0032] Bitrate is the number of bits of multimedia data transmitted or stored per unit time, which can also be understood as the data transmission rate.

[0033] The quality index (VMAF, Video Multimethod Assessment Fusion), also known as video multi-method fusion evaluation, is a metric used to represent video quality.

[0034] The encoding parameters of multimedia data are parameters used to control the encoding of multimedia data, such as CRF (Constant Rate Factor, constant quality factor) and the like.

[0035] CRF (Constant Rate Factor) is one of the encoding parameters used to control the encoding quality of multimedia data. By adjusting the CRF value, the compression efficiency and visual quality of the encoding can be affected.

[0036] At present, most major multimedia platforms use fixed CRF values ​​to encode multimedia data, which may lead to unstable quality and bitrate waste in the encoded multimedia data. Although some video platforms and manufacturers use bitrate quality models to predict the corresponding CRF value under constant VMAF for encoding. However, this strategy can only keep the VMAF of multimedia data constant at a certain value, and the transcoded multimedia data has the same quality. Since the model can only predict a CRF value under a fixed VMAF, its application scenarios are limited, and it is impossible to adapt the quality to the bitrate requirements, and it is impossible to avoid bitrate waste while ensuring quality.

[0037] In order to solve the above technical problems, in this specification, a multimedia data encoding method is provided. This specification also involves a multimedia data encoding device, a video data encoding method, a video data encoding device, a computing device, a computer-readable storage medium, and a computer program product, which are described in detail one by one in the following embodiments.

[0038] See also Figure 1 , Figure 1 An architecture diagram of a multimedia data encoding system provided by an embodiment of the present specification is shown. The multimedia data encoding system may include: a server, a first client, and a second client.

[0039] The first client may be a sending end of multimedia data, for example, it may be a client corresponding to a user (such as a host) who initiates a live broadcast, or it may be a client corresponding to a user who uploads a short video.

[0040] The second client may be a receiving end of multimedia data, for example, a client corresponding to a user watching a live broadcast, or a client corresponding to a user watching a short video.

[0041] The server can be used to provide multimedia data encoding services between a first client and a second client.

[0042] Taking the live broadcast scenario as an example, the first client sends an initial live video stream to the server. The server decodes the initial live video stream to obtain a source live video stream. Afterwards, the server determines the target encoding parameter value of the source live video stream according to the multimedia data encoding method provided in the embodiment of this specification. Specifically, the server determines the current quality index value based on the corresponding change relationship between the bit rate quality of the source live video stream and the current encoding parameter value of the target encoding parameter, adjusts the current encoding parameter value according to the target adjustment strategy, obtains an updated current encoding parameter value, determines the updated current quality index value based on the corresponding change relationship between the bit rate quality and the updated current encoding parameter value, and when the difference between the updated current quality index value and the target quality index value meets the preset output condition, uses the updated current encoding parameter value as the target encoding parameter value for encoding the source live video stream. Thus, the server encodes the source live video stream based on the target encoding parameter value to obtain the target live video stream, and sends the target live video stream to the second client so that the second client plays the target live video stream.

[0043] It should be noted that the first client may be one or more clients, and the second client may be one or more clients. Multiple clients may establish communication connections through a server. In a multimedia platform, a server may be used to provide multimedia data processing and transmission services between multiple clients. Multiple clients may serve as senders or receivers, respectively, and communicate through the server.

[0044] The user can interact with the server through the second client to receive multimedia data sent by the first client, or send multimedia data to the first client. In the live broadcast scenario, the user can send a live video stream to the server through the first client, and the server transcodes the live video stream and pushes it to the second client with which the communication is established.

[0045] The client and the server are connected via a network. The network provides a medium for the communication link between the client and the server. The network can include various connection types, such as wired or wireless communication links or optical fiber cables. The data transmitted by the client may need to be encoded, transcoded, compressed, etc. before being published to the server.

[0046] The client can be a browser, an APP (Application), or a web application such as H5 (HyperText Markup Language 5, Hypertext Markup Language Version 5) application, or a light application (also known as a mini-program, a lightweight application) or a cloud application, etc. The client can be based on the software development kit (SDK, Software Development Kit) of the corresponding service provided by the server, such as based on the real-time communication (RTC, Real Time Communication) SDK development and acquisition. The client can be deployed in an electronic device and needs to rely on the device to run or some APPs in the device to run. For example, the electronic device can have a display screen and support information browsing, such as a personal mobile terminal such as a mobile phone, a tablet computer, a personal computer, etc. Various other types of applications can also be configured in the electronic device, such as human-computer dialogue applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, mailbox clients, social platform software, etc.

[0047] The server may include servers that provide various services, such as servers that provide communication services to multiple clients, servers that provide background training to support models used on clients, and servers that process data sent by clients. It should be noted that the server can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. The server can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content De l ivery Network), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0048] In some live broadcast scenarios, the multimedia data encoding system may also include: an upstream node, a downstream node, and a live broadcast source station. In the live broadcast scenario, the first client is used to send an initial live video stream to the upstream node, and the initial live video stream is sent to the live broadcast source station through the upstream node. The live broadcast source station is used to receive the initial live video stream from the upstream node, encode and package it, and then push it to the server. The server, as the entrance or live broadcast center server of the content distribution network, decodes the initial live video stream pushed by the live broadcast source station to obtain the source live video stream, and then determines the target encoding parameter value of the source live video stream according to the multimedia data encoding method provided in the embodiment of this specification, encodes the source live video stream based on the target encoding parameter value to obtain the target live video stream, and sends the target live video stream to the second client through the downstream node so that the second client plays the target live video stream.

[0049] It is worth noting that the multimedia data encoding method provided in the embodiments of this specification is generally executed by the server, but in other embodiments of this specification, the client may also have similar functions to the server, thereby executing the multimedia data encoding method provided in the embodiments of this specification. In other embodiments, the multimedia data encoding method provided in the embodiments of this specification may also be jointly executed by the client and the server.

[0050] It can be seen from the above system embodiments that when the server in the system needs to encode multimedia data, it analyzes the encoding parameter values ​​of the multimedia data based on the corresponding change relationship between the encoding parameter value, the bit rate and the quality index value, and adjusts the current encoding parameter value to achieve the purpose of correspondingly adjusting the bit rate and quality, thereby finding a better balance between quality and bit rate as needed, and obtaining the target encoding parameter value when the difference between the updated current quality index value and the target quality index value meets the preset output condition, and can determine the target encoding parameter value within a narrow range relative to the target quality index value, thereby ensuring the stability of multimedia data quality and avoiding bit rate waste.

[0051] See also Figure 2 , Figure 2 A flowchart of a multimedia data encoding method provided by an embodiment of this specification is shown, which specifically includes the following steps:

[0052] Step 202: Determine the current quality index value based on the corresponding change relationship between the bit rate and quality of the multimedia data and the current encoding parameter value of the target encoding parameter, wherein the corresponding change relationship between the bit rate and quality includes: multiple encoding parameter values ​​of the target encoding parameter, the corresponding change relationship between the bit rate and the quality index value, and the current encoding parameter value is any one of the multiple encoding parameter values.

[0053] Multimedia data refers to data that combines multiple types of media content, which may include audio, video, animation, etc. The method provided in the embodiments of this specification can be applied to a server of a multimedia platform. A multimedia platform can be a digital platform that integrates multiple media forms, such as a multimedia teaching platform, a multimedia entertainment platform, a multimedia social platform, etc., and provides multimedia services through any media elements such as audio, video, graphics, animation, text, etc.

[0054] The corresponding change relationship between bit rate and quality includes: the corresponding change relationship between multiple coding parameter values, bit rate and quality index value of the target coding parameter. For example, the corresponding change relationship between bit rate and quality can be expressed by a curve or any other method, and this specification does not limit this. It can be understood that the target coding parameter is a coding parameter used to control the coding quality of multimedia data, the bit rate is the number of bits of multimedia data transmitted or stored per unit time, and the quality index value is the value of a metric indicating the quality of the video. Therefore, the larger the target coding parameter value, the stronger the compression will be applied to the coding, the worse the quality will be, and the lower the bit rate will be. Therefore, the size of the target coding parameter value is inversely proportional to the size of the corresponding quality index value, and the size of the target coding parameter value is also inversely proportional to the size of the corresponding bit rate. The acquisition method of the corresponding change relationship between bit rate and quality is not limited, and can be predicted by any statistical method or quality bit rate prediction model. Exemplarily, the corresponding change relationship between bit rate and quality can be expressed as a curve, which includes a CRF value ranging from 20 to 40, a step size of 0.2, and a total of 101 points of the corresponding relationship between bit rate and quality.

[0055] The target coding parameter is a coding parameter used to control the coding quality of multimedia data. For example, the target coding parameter may include but is not limited to any one or more of CRF, QP (Quantization Parameter) and the like.

[0056] Exemplarily, when a server of a multimedia platform receives an initial video stream sent by a client, the initial video stream is decoded to obtain a source video stream. The server uses a rate-quality prediction model to predict the corresponding change relationship between the rate-quality of the source video stream, and determines the current CRF value of the CRF parameter. There is no limit to the method for determining the initial current CRF value. For example, a fixed CRF value in an online fixed coding parameter mode is used as the initial current CRF value. After determining the initial current coding parameter value, a search can be started from the initial current coding parameter value, and finally a target coding parameter value that effectively guarantees the quality of multimedia data and avoids waste of bit rate is searched.

[0057] Since the corresponding change relationship of the encoding quality of multimedia data includes multiple encoding parameter values ​​of the target encoding parameters, the corresponding change relationship between the bit rate and the quality index value, subsequent analysis based on the corresponding change relationship can stabilize the quality within a narrower range.

[0058] Step 204: According to the target adjustment strategy, the current encoding parameter value is adjusted to obtain an updated current encoding parameter value.

[0059] For example: the target adjustment strategy includes: taking the encoding quality of multimedia data approaching the target quality index value as the adjustment direction. Taking the encoding quality of multimedia data approaching the target quality index value as the adjustment direction can be determined by determining the difference between the current quality index value and the target quality index value. The difference between the current quality index value and the target quality index value can be determined by comparison or the like. The difference can be expressed as a comparison result greater than, less than or equal to, or as a numerical value indicating the size of the difference, and this specification does not limit this.

[0060] Exemplarily, when the current quality index value is greater than the target quality index value, it means that the current encoding parameter value of the target encoding parameter is insufficient, and the current encoding parameter value needs to be further increased to make the corresponding quality index value approach the target quality index value, and thus, the corresponding target adjustment strategy is to increase the current encoding parameter value. When the current quality index value is less than the target quality index value, it means that the current encoding parameter value of the target encoding parameter is too large, and the current encoding parameter value needs to be further decreased to make the corresponding quality index value approach the target quality index value, and thus, the corresponding target adjustment strategy is to decrease the current encoding parameter value.

[0061] The updated current encoding parameter value is the encoding parameter value obtained by adjusting the current encoding parameter value. The amplitude of one adjustment can also be understood as a step size, which is set according to the actual application scenario, and this manual does not limit this. For example: the current encoding parameter value is 30, assuming that the adjustment step size is 0.2, when the current encoding parameter value is increased, after adjusting the current encoding parameter value, the updated current encoding parameter value can be obtained as 30.2.

[0062] In the method provided in the embodiment of this specification, the goal of multimedia data encoding is to adjust the bit rate and quality accordingly by adjusting the current encoding parameter value while approaching the target quality index value, so as to find a better balance point between quality and bit rate as needed. Adjusting the current encoding parameter value may include either increasing or decreasing the adjustment direction. In addition, other strategies may be combined to make a decision on whether to increase or decrease. For example, the decision may be made using the gradient corresponding to the current encoding parameter value on the bit rate quality corresponding change relationship curve.

[0063] Step 206: Determine an updated current quality indicator value based on the bit rate quality corresponding change relationship and the updated current encoding parameter value.

[0064] After adjusting the encoding parameter value, it is necessary to further determine whether the quality meets the requirements. Therefore, in this step, it is necessary to further determine the quality index value corresponding to the updated current encoding parameter value obtained after adjustment. Specifically, based on the updated current encoding parameter value, the bit rate of the code stream encoded by the updated current encoding parameter value of the multimedia data can be determined, and the updated current encoding parameter value and the corresponding bit rate, as well as the corresponding updated current quality index value, can be found on the bit rate quality prediction curve.

[0065] Exemplarily, assuming that the updated current encoding parameter value of the CRF parameter is 30.2, the multimedia data code to be encoded is encoded using a CRF value of 30.2, and the resulting code stream has a bit rate of 1.5M. In the corresponding change relationship between bit rate and quality, there is a point whose CRF value is 30.2, the bit rate is 1.5M, and the corresponding quality index value is 85, thereby determining that the updated current quality index value is 85.

[0066] Step 208: When the difference between the updated current quality indicator value and the target quality indicator value meets a preset output condition, use the updated current encoding parameter value as a target encoding parameter value for encoding the multimedia data.

[0067] The preset output condition is a judgment condition for judging whether the updated current encoding parameter value makes the encoding quality reach a sufficiently narrow range. The preset output condition is determined according to the quality requirements in the actual application scenario.

[0068] In one or more embodiments, the preset output condition may be expressed as a condition including a preset difference threshold. When the difference between the updated current quality index value and the target quality index value reaches the preset difference threshold, the balance between quality and bit rate in the application scenario is satisfied, and it is determined that the preset output condition is satisfied. Exemplarily, assuming that the preset output condition is "the difference between the current quality index value and the target quality index value is less than 5", in this example, the preset difference threshold is 5.

[0069] In one or more other embodiments, the preset output condition may be expressed as a condition with a target quality indicator value as a threshold. When the difference between the updated current quality indicator value and the target quality indicator value is closest to the target quality indicator value, the balance between quality and bit rate in the application scenario is satisfied, and it is determined that the preset output condition is satisfied. Exemplarily, assuming that the current encoding parameter value is continuously increased, the preset output condition is "when the current quality indicator value is greater than the target quality indicator value, the updated current quality indicator value should be less than or equal to the target quality indicator value". In this example, when the current quality indicator value changes from greater than the target quality indicator value to less than or equal to the target quality indicator value, it is determined that the current quality indicator value is closest to the target quality indicator value.

[0070] It can be seen that the method combines the corresponding change relationship between the encoding parameter value, the bit rate and the quality index value to analyze the encoding parameter value of the multimedia data, and achieves the purpose of correspondingly adjusting the bit rate and quality by adjusting the current encoding parameter value, so as to find a better balance point between quality and bit rate as needed, and obtain the target encoding parameter value when the difference between the updated current quality index value and the target quality index value meets the preset output condition, and can determine the target encoding parameter value within a narrow range relative to the target quality index value, thereby ensuring the stability of the multimedia data quality and avoiding bit rate waste.

[0071] In one or more embodiments of the present specification, a rate-quality prediction model can be used to predict the rate-quality correspondence change relationship of multimedia data to obtain the rate-quality correspondence change relationship, in which the coding parameter value is a plurality of coding parameter values ​​within a preset coding parameter value range of the target coding parameter.

[0072] The rate quality prediction model is a model constructed using mathematical, statistical or machine learning methods. In the method provided in the embodiments of this specification, the rate quality prediction model is used to predict the rate and quality index values ​​of multimedia data under multiple encoding parameter values ​​within a preset encoding parameter value range. For example: using the rate quality prediction model, the corresponding change relationship of the rate quality of multimedia data can be predicted before actual encoding. The rate quality prediction model can be expressed as a model based on the human visual system; it can be expressed as a statistical model, such as linear regression, polynomial regression; or it can be expressed as a machine learning model, including support vector machine (SVM), random forest, neural network, etc.

[0073] The corresponding change relationship of the bit rate quality predicted by the bit rate quality prediction model can be expressed as a multi-point scatter plot with the bit rate as the horizontal axis information, the quality index as the vertical axis information, and one coding parameter value corresponding to one point. The scatter plot can further fit a bit rate quality prediction curve, which can also be called a bit rate-quality (bitrate-vmaf) curve. Therefore, in step 202, based on the current coding parameter value, the bit rate of the code stream encoded by the current coding parameter value of the multimedia data to be encoded can be determined, and then the current coding parameter value and the corresponding bit rate, and the current quality index value corresponding to the bit rate quality prediction curve can be found.

[0074] The following is a schematic illustration of the rate-quality prediction model and the process of the rate-quality prediction model predicting the corresponding change relationship between the rate and quality of multimedia data.

[0075] The implementation method of the rate quality prediction model is not limited. For example, in the field of deep learning, a variety of models can be used to construct a rate quality prediction model, such as a model based on a recurrent neural network, a model based on an attention mechanism, and the like. Before the model actually predicts, the rate quality prediction model needs to be trained. The training process may include: first, feature extraction of multimedia training data. The multimedia training data for feature extraction may be represented by video or segmented content. Feature extraction refers to extracting features related to the prediction of quality indicator values, bit rates, and encoding parameter values ​​from multimedia data. For example, features including but not limited to frame rate, resolution, content complexity (such as texture, motion information), scene change speed, etc. can be extracted. Secondly, the model is trained using a multimedia training data set. The multimedia training data set includes: features of multimedia training data, corresponding relationships between encoding parameter true value labels, bit rate true value labels, and quality indicator true value labels. During the training process, the model parameters are continuously iterated and optimized to obtain a trained rate quality prediction model. After obtaining the trained rate quality prediction model, the generalization ability and prediction accuracy of the rate quality prediction model can be further evaluated through a validation set.

[0076] After obtaining the trained bit rate quality prediction model, the trained bit rate quality prediction model can be used for actual prediction. Specifically, the features of the multimedia data to be encoded can be input into the bit rate quality prediction model to predict the corresponding change relationship between the encoding parameter value, bit rate and quality index value of the multimedia data.

[0077] For example, Figure 3a FIG. 1 shows a schematic diagram of the model structure of a bit rate quality prediction model provided by an embodiment of the present specification. Figure 3a The model structure shown in the figure is used to construct a bit rate quality prediction model. Figure 3aAs shown, the rate quality prediction model may include: a first basic network unit, a suspension operator and a second basic network unit. The basic network may include: a normalization layer, a self-attention unit, two layers of residual feature units and a fully connected layer. The input of the rate quality prediction model is the video features extracted by the feature extraction module from the source video or the video fragment of the source video. Specifically,

[0078] The feature extraction module can be configured to extract video features, including but not limited to content features, online complete coding features, and fast pre-coding features. Among them, content features may include video spatial texture features, quality assessment features, and spatiotemporal complexity features. Online complete coding features may include: anchor coding parameter features, anchor bit rate features, anchor quality features, etc. Fast pre-coding features may include: coding mode features, coding frame type / proportion features, etc. Anchor coding parameter features, anchor bit rate features, and anchor quality features are video spatiotemporal correlation features extracted based on traditional spatiotemporal domain algorithms. Coding mode features refer to using x264 to quickly encode the source video once and extract the corresponding intra-frame prediction / inter-frame prediction coding information and mode / partitioning information. In addition, basic information such as the width, height, and frame rate of the source video can also be extracted.

[0079] The first basic network unit may be configured to receive the video features output by the feature extraction module and predict an initial rate-quality corresponding change relationship curve. The rate-quality corresponding change relationship curve may include: a coding parameter rate corresponding change relationship curve and a coding parameter quality corresponding change relationship curve.

[0080] The suspension operator can be configured to use the suspension operator to calculate the initial rate-quality corresponding change relationship curve output by the first basic network unit to obtain the rate-quality corresponding change relationship suspension curve. The suspension operator can allow the curve after the suspension calculation to pass through the (anchor coding parameter, anchor code rate) point and the (anchor coding parameter, anchor quality) point.

[0081] The second basic network unit can be configured to receive the rate-quality corresponding change relationship suspension curve output by the suspension operator, and modify the curve to obtain the modified rate-quality corresponding change relationship curve. The second basic network unit can use skip connection for residual learning, which can have better performance.

[0082] More specifically, the first basic network unit and the second basic network unit both include a basic network. The input of the basic network is the video feature, and the output is the corresponding change relationship curve of the bit rate quality. The basic network first uses a normalization layer (such as a batch normalization layer) to normalize the video features of the input, and the normalized video features are input to the self-attention module to extract the correlation coefficients between the features and combine them into deep features. The deep features are passed through two layers of residual feature units and a fully connected layer to obtain the final curve output.

[0083] The suspension operator is to perform an offset operation on the overall y value of a curve. The offset is the y value corresponding to the anchor coding parameter in the curve minus the true value of y (where the y value in the coding parameter rate corresponding change relationship curve is the anchor code rate, and the y value in the coding parameter quality corresponding change relationship curve is the anchor quality). Therefore, after the calculation of the suspension operator, the curve after the suspension calculation passes through the (anchor coding parameter, anchor code rate) point and the (anchor CRF, anchor quality) point.

[0084] For example, Figure 3a Before training the rate quality prediction model shown, data preparation can be performed first. For example, the aforementioned video feature extraction can be performed on 1w online videos, and then the 1w videos can be transcoded using online transcoding commands. The value range of encoding parameters such as CRF can be 20-40, and encoding is performed once with a step size of 0.2. A total of 101w transcoded streams are generated, and the rate information and quality information of the transcoded streams are extracted at the same time. Therefore, the training data of the model includes: the extracted video features as input samples, the (rate, quality) point pairs corresponding to the corresponding encoding parameters such as CRF points of the corresponding videos as output labels, and each piece of data corresponds to 101 point pairs. During the training process, the difference between the predicted value and the label can be supervised by L1 loss, and the gradient difference can be supervised by L1 loss. Among them, the gradient can be defined as the quality difference value of the adjacent CRF points divided by the rate difference value.

[0085] according to Figure 3a The bitrate quality prediction model shown in the figure takes CRF encoding parameters as an example. The characteristics of multimedia data are input into the bitrate quality prediction model for inference, and the corresponding relationship of (bitrate, VMAF) with a CRF value range of 20 to 40 and a step size of 0.2 can be obtained. With bitrate as the horizontal axis information and VMAF as the vertical axis information, a scatter plot of 101 points can be drawn and a bitrate-vmaf curve can be fitted.

[0086] In one or more embodiments of the present specification, in order to more efficiently search for a target encoding parameter value with stable quality and quality that is compatible with bit rate changes, the method further includes:

[0087] Using the encoding parameter value of the target encoding parameter in the fixed encoding parameter mode as the initial current encoding parameter value;

[0088] The target quality index value is obtained by counting the mean value of the quality index of the reference multimedia data encoded in the fixed encoding parameter mode.

[0089] The fixed encoding parameter mode refers to a mode in which the encoding parameters are set to fixed values ​​during the multimedia data encoding process. For example, for the CRF parameter, its fixed value in the fixed encoding parameter mode can be used as the initial current encoding parameter value.

[0090] The quality index mean refers to the average value of the quality index (VMAF) achieved by the reference multimedia data encoded in the fixed coding parameter mode. Among them, the reference multimedia data refers to the multimedia data used as the quality index mean statistics. Reference multimedia data may refer to historical or online multimedia data in any data source. The data source may include any data source such as a multimedia platform and a multimedia database. The target quality index value is used as a constraint on the search range for analyzing and searching for the target coding parameter value. In the embodiment of this specification, the target quality index value can be determined based on experience or any strategy. In this embodiment, the target quality index value is obtained by statistically calculating the quality index mean of the reference multimedia data encoded in the fixed coding parameter mode, so that the search range can be constrained within a relatively narrow range relative to the quality average line, thereby achieving the purpose of ensuring the encoding quality of the multimedia data.

[0091] Exemplarily, taking the CRF encoding parameters as an example, the CRF parameter value in the fixed CRF mode can be used as the search starting value of the target CRF parameter value, that is, the initial current encoding parameter value, recorded as org_crf. Determine the bit rate of the code stream of the multimedia data encoded by org_crf, recorded as org_bitrate. According to the bitrate-vmaf prediction curve, find out the VMAF value corresponding to org_crf and org_bitrate, as the initial current quality index value, recorded as org_vmaf. In the initial state, curr_crf (indicating the current encoding parameter value) = org_crf, curr_vmaf (indicating the current quality index value) = org_vmaf. The VMAF mean value in the fixed CRF mode is counted to obtain the target quality index value, recorded as target_vmaf, which is used to make the encoded code stream of the target encoding parameter value finally searched out approach the value of target_vmaf.

[0092] In this embodiment, the initial current coding parameter value is determined based on the coding parameter value of the target coding parameter in the current fixed coding parameter mode online, which is equivalent to starting analysis and search from the coding parameter value that basically meets the quality requirements. It can more quickly search for the target coding parameter value that ensures the stability of multimedia data quality and avoids bit rate waste.

[0093] In one or more embodiments of the present specification, the gain or loss brought by adjusting the current encoding parameter value is measured by using the gradient in the adjustment direction on the curve corresponding to the relationship between the change in bit rate and quality, so that the encoding quality can achieve a balance between cost reduction or image quality improvement as needed according to the target adjustment strategy determined by the gradient. Specifically, step 204 adjusts the current encoding parameter value according to the target adjustment strategy, and before obtaining the updated current encoding parameter value, it also includes:

[0094] Determining an adjustment direction of a current encoding parameter value according to a difference between the current quality indicator value and the target quality indicator value;

[0095] A target adjustment strategy is determined according to a gradient of a current encoding parameter value in the adjustment direction on a curve corresponding to the bit rate quality change relationship.

[0096] Among them, the difference between the current quality index value and the target quality index value includes: greater than, less than or equal to three situations. In the case of greater than, it means that the bit rate is too high, and the adjustment direction of the current encoding parameter value should be increased, that is, by increasing the current encoding parameter value, the purpose of reducing the bit rate is achieved; in the case of less than or equal to, it means that the quality is insufficient, and the adjustment direction of the current encoding parameter value should be reduced, that is, by reducing the current encoding parameter value, the purpose of improving the quality is achieved. Based on this, in one or more embodiments of this specification, the adjustment direction is determined according to the difference between the current quality index value and the target quality index value, including:

[0097] When the current quality index value is greater than the target quality index value, the adjustment direction is an increase direction;

[0098] When the current quality indicator value is less than or equal to the target quality indicator value, the adjustment direction is a decreasing direction.

[0099] For example, assuming that the current quality index value is 40 and the target quality index value is 65, in this case, the current encoding parameter value needs to be increased; assuming that the current quality index value is 90 and the target quality index value is 65, in this case, the current encoding parameter value needs to be decreased.

[0100] According to the above embodiment, if the current quality index value is greater than the target quality index value, it means that the encoding quality of the current encoding parameter value is too high, and there is a waste of code rate. Therefore, by reducing the current encoding parameter value, the corresponding current quality index value can be reduced accordingly, so as to be closer to the target quality index value. In this case, the adjustment direction is to reduce, so as to achieve the purpose of saving code rate. Relatively speaking, if the current quality index value is less than or equal to the target quality index value, it means that the encoding quality of the current encoding parameter value is too low, and there is a situation of insufficient code rate. Therefore, by increasing the current encoding parameter value, the corresponding current quality index value can be increased accordingly, so as to be closer to the target quality index value. In this case, the adjustment direction is to increase, so as to achieve the purpose of improving quality.

[0101] In one or more embodiments of the present specification, determining a target adjustment strategy according to a gradient of a current encoding parameter value in the adjustment direction on a curve corresponding to the relationship between the bit rate and quality, includes:

[0102] In the case where the adjustment direction is an increasing direction, according to the first gradient in the increasing direction of the current encoding parameter value on the curve corresponding to the bit rate quality corresponding change relationship, it is determined whether to increase the current encoding parameter value, and if so, the target adjustment strategy is to increase the current encoding parameter value. Accordingly, in the case where it is determined according to the first gradient that the current encoding parameter value does not need to be increased, the target adjustment strategy is that no further adjustment is required.

[0103] In the case where the adjustment direction is a decreasing direction, according to the second gradient in the decreasing direction of the current encoding parameter value on the curve corresponding to the bit rate quality corresponding change relationship, it is determined whether to decrease the current encoding parameter value, and if so, the target adjustment strategy is determined to decrease the current encoding parameter value. Accordingly, in the case where it is determined according to the second gradient that the current encoding parameter value does not need to be decreased, the target adjustment strategy is that no further adjustment is required.

[0104] On the bit rate quality corresponding change relationship curve, whenever the current encoding parameter value is increased or decreased, a certain change rate will be presented in the corresponding adjustment direction, also called a gradient. The corresponding change rate when the current encoding parameter value is increased is called the first gradient, which means the ratio of the corresponding quality change value to the bit rate change value when the current encoding parameter value is increased under the bit rate quality corresponding change relationship. The corresponding change rate when the current encoding parameter value is decreased is called the second gradient, which means the ratio of the corresponding quality change value to the bit rate change value when the current encoding parameter value is decreased under the bit rate quality corresponding change relationship.

[0105] Exemplarily, taking the CRF encoding parameters as an example, the first gradient or the second gradient, denoted as curr_grad, can be calculated by the following formula:

[0106]

[0107] Among them, Δvmaf curr_crf , which represents the quality change value on the curve corresponding to the change relationship between the bit rate and quality of the multimedia data to be encoded, Δbitrate curr_crf , which represents the bit rate change value on the curve corresponding to the bit rate quality change relationship of the multimedia data to be encoded.

[0108] Therefore, the first gradient can represent the loss of VMAF caused by the reduction of unit bit rate. The lower the ratio, the smaller the loss of multimedia data quality caused by the reduction of each unit bit rate. The second gradient can represent the gain of VMAF caused by the increase of unit bit rate. The higher the ratio, the greater the improvement of multimedia data quality caused by the increase of each unit bit rate. Therefore, according to the size of the first gradient, it can be decided whether to increase the current encoding parameter value to achieve the purpose of saving more bit rate at the cost of less quality loss; according to the size of the second gradient, it can be decided whether to decrease the current encoding parameter value to achieve the purpose of improving more quality at the cost of less bit rate waste.

[0109] In practical applications, the judgment condition corresponding to the first gradient or the second gradient may be set according to the requirements of quality and bit rate, and this specification does not limit this.

[0110] For example, the first gradient or the second gradient can be compared with the gradient on the historical true value change moving average to determine whether the current encoding parameter value is worth continuing to adjust.

[0111] For another example, the first gradient or the second gradient may be compared with a gradient threshold value preset based on experience to determine whether the current encoding parameter value is worth continuing to adjust.

[0112] Through the above embodiments, based on the gradient of the current encoding parameter value in the adjustment direction on the curve corresponding to the relationship between the bit rate and the quality, it is possible to effectively evaluate the gain or loss brought by adjusting the current encoding parameter value, and then decide whether to adjust the current encoding parameter value. Specifically, when the first gradient is small, a smaller decrease in quality can be exchanged for more bit rate savings, the current encoding parameter value can be increased to reduce the bit rate of the multimedia data; when the second gradient is large, a smaller increase in bit rate can be exchanged for more quality improvement, the current encoding parameter value can be reduced to improve the quality of the multimedia data.

[0113] Furthermore, in order to more accurately evaluate whether the first gradient is small, or to accurately evaluate whether the second gradient is large, and to adjust the current encoding parameter based on the accurate evaluation, in one or more embodiments of the present specification, determining whether to increase the current encoding parameter value according to the first gradient in the increasing direction on the curve corresponding to the bit rate quality corresponding change relationship of the current encoding parameter value includes:

[0114] Determine a first gradient of the current encoding parameter value in an increasing direction on a curve corresponding to the bit rate quality corresponding change relationship;

[0115] When the first gradient is less than or equal to the first reference gradient, determining to increase the current encoding parameter value;

[0116] Among them, the first reference gradient is the gradient of the current coding parameter value on the historical true value change moving average in the adjustment direction, and the historical true value change moving average is the moving average of the corresponding change relationship among the coding parameter value, bit rate and quality index value determined based on historical multimedia data.

[0117] The historical true value change average line includes the corresponding change relationship between the encoding parameter value, bit rate and quality index value of the historical multimedia data. The gradient of the historical true value change average line can be used to evaluate whether the first gradient is small or the second gradient is large.

[0118] The historical multimedia data may be multimedia training data for training a bitrate quality prediction model, or may be existing multimedia data in any data source having a corresponding change relationship between a true value of a coding parameter, a true value of a bitrate, and a true value of a quality indicator, and this specification does not limit this. In one or more embodiments, the historical true value change moving average may be expressed as a bitrate-vmaf average curve corresponding to the true value in the multimedia training data of the bitrate quality prediction model.

[0119] In one or more other embodiments, determining whether to reduce the current encoding parameter value according to a second gradient in a decreasing direction on a curve corresponding to the bit rate quality corresponding change relationship of the current encoding parameter value includes:

[0120] Determine a second gradient of the current encoding parameter value in a decreasing direction on the curve corresponding to the bit rate quality change relationship;

[0121] When the second gradient is greater than or equal to a second reference gradient, determining to decrease the current encoding parameter value;

[0122] The second reference gradient is the gradient of the current encoding parameter value on the historical true value change moving average in the decreasing direction.

[0123] Exemplarily, taking the CRF encoding parameters as an example, the first reference gradient or the second reference gradient is recorded as online_grad, and the online_grad can be calculated by the following formula:

[0124]

[0125] Among them, Δavg_vmaf org_crf , represents the quality change value on the moving average of historical true value changes, Δavg_bitrate org_crf , which represents the bit rate change value on the historical true value change moving average.

[0126] It is understandable that since the above formula uses the ratio of the quality change value to the bit rate change value for calculation, and does not distinguish between the increasing direction and the decreasing direction, the above formula can be applied to both cases. For each current encoding parameter value, the corresponding gradient in the increasing direction or the decreasing direction, namely, curr_grad and online_grad, can be calculated.

[0127] Assume that the current encoding parameter value of the target encoding parameter is 25. On the bitrate-quality corresponding change relationship curve, the first gradient in the increasing direction is 0.005, and on the historical true value change moving average, the first reference gradient in the increasing direction is 0.01. The first reference gradient of 0.01 means that if the online video average bitrate drops by 100kb, the corresponding VMAF will drop by 1, and the first gradient of 0.005 means that if the current video bitrate drops by 200kb, the corresponding VMAF will drop by 1. At this time, increasing the current encoding parameter can obtain more bitrate savings than the online average line, while bringing less quality reduction. Therefore, the current encoding parameter value can be increased to save bitrate, and VMAF will not drop too much.

[0128] In the above embodiment, the first gradient is evaluated by the first reference gradient determined by the historical truth value change moving average, or the second gradient is evaluated by the second reference gradient determined by the historical truth value change moving average. It is possible to refer to the quality-bitrate benefit ratio of the historical truth value change moving average to effectively evaluate the size of the current quality-bitrate benefit ratio, and then decide whether to adjust the current encoding parameter value so that the multimedia data has a certain increase in the average quality index while the bitrate mean remains relatively unchanged and the quality distribution is within a reasonable range.

[0129] Accordingly, in one or more embodiments of this specification, the method further includes:

[0130] In a case where the first gradient is greater than a first reference gradient, determining the current encoding parameter value to be a target encoding parameter value for encoding multimedia data;

[0131] In a case where the second gradient is less than the second reference gradient, the current encoding parameter value is determined as a target encoding parameter value for encoding multimedia data.

[0132] It can be understood that, when the first gradient is greater than the first reference gradient, it means that if the current encoding parameter value is continued to be increased, it cannot save much bit rate relative to the moving average, the benefit is too small, and no further adjustment is required, and the current encoding parameter value should be output as the target encoding parameter value; when the second gradient is less than the second reference gradient, it means that if the current encoding parameter value is continued to be decreased, it cannot improve the quality more relative to the moving average while bringing less bit rate increase, the benefit is not large, and no further adjustment is required, and the current encoding parameter value should be output as the target encoding parameter value.

[0133] For example, assuming that the current encoding parameter value of the target encoding parameter is 25, on the bit rate quality corresponding change relationship curve, the second gradient in the decreasing direction is 0.005, and on the historical true value change moving average, the first reference gradient in the decreasing direction is 0.01. The first reference gradient of 0.01 means that the online video average bit rate increases by 100kb, and the corresponding VMAF will increase by 1, while the second gradient of 0.005 means that for the current video bit rate increases by 200kb, the corresponding VMAF will increase by 1. At this time, reducing the current encoding parameter requires more bit rate improvement than the online average line to bring less quality improvement. Therefore, there is no need to continue to reduce the current encoding parameter value to improve the quality and avoid excessive bit rate. Thus, the encoding parameter value of the target encoding parameter is obtained as 26.

[0134] In the above embodiment, by comparing the first gradient with the first reference gradient, and by comparing the second gradient with the second reference gradient, the gain or loss of quality and bit rate brought about by adjusting the current coding parameter value can be effectively evaluated, and then a decision is made whether to output the current coding parameter value as the target coding parameter value, thereby achieving the purpose of bringing more bit rate savings at the cost of smaller quality loss, or achieving the purpose of bringing more quality improvement at the cost of smaller bit rate waste.

[0135] In order to achieve higher application flexibility, in one or more embodiments of this specification, the size of the reference gradient is controlled by parameters, and by adjusting the size of the reference gradient, the decision strategy can be changed to reduce the bit rate (reduce costs) for high-quality content, or to increase the bit rate (improve image quality) for low-quality content. Specifically, the method also includes:

[0136] Acquire a first control parameter, and use the first control parameter to adjust the magnitude of the first reference gradient;

[0137] A second control parameter is obtained, and the magnitude of the second reference gradient is adjusted using the second control parameter.

[0138] Exemplarily, assuming that the first control parameter is recorded as factor1 and the second control parameter is recorded as factor2, the adjusted first reference gradient may be equal to the first reference gradient before adjustment*factor1; the adjusted second reference gradient may be equal to the second reference gradient before adjustment*factor2.

[0139] For example, when factor1 is set to 1, the decision strategy corresponding to the first gradient is: when VMAF decreases, the encoding parameter value is increased when more bitrate savings (compared to the online average) are achieved. This setting is used to reduce the bitrate of multimedia data with excess bitrate.

[0140] For another example, when factor2 is set to 1, the decision strategy corresponding to the second gradient is: when a smaller bit rate increase can be exchanged for a larger VMAF increase, the encoding parameter value is reduced. This setting is used to improve the quality of multimedia data with insufficient quality and avoid excessive bit rate increase.

[0141] For another example, when factor1 and / or factor2 are set to control parameter values ​​greater than 0 and less than 1, the decision strategy is expressed as: adjusting the balance point between cost reduction or image quality improvement according to the set control parameter values.

[0142] For example, factor1 and / or factor2 may be set to any value greater than 1 or less than 0 according to application scenario requirements, and this specification does not limit this.

[0143] In order to facilitate setting of the first control parameter and / or the second control parameter, one or more embodiments of the present specification also provide an embodiment of receiving the first control parameter and / or the second control parameter through a user interaction interface of a client device. Figure 3b is a schematic diagram of a user interaction interface provided by an embodiment of this specification, such as Figure 3b In the user interaction interface schematic diagram shown, the client receives the first control parameter and / or the second control parameter input by the user through the user interaction interface. For example: the client in this embodiment can be the management side of the multimedia platform, which is used to manage the transcoding process of multimedia data. The user can be understood as the administrator of the multimedia platform. Through the user interaction interface of the management side of the multimedia platform, the administrator can set the first control parameter and / or the second control parameter to adjust the decision-making strategy. The client sends the first control parameter and / or the second control parameter to the server, so that the server uses the first control parameter to adjust the size of the first reference gradient, and / or uses the second control parameter to adjust the size of the second reference gradient.

[0144] like Figure 3bAs shown, the user interaction interface may include a control parameter input box, "OK" and "Cancel" controls. Figure 3b In the user interaction interface shown, the user can input the first control parameter and / or the second control parameter through the control parameter input box, and click the "OK" control, so that the client sends the first control parameter and / or the second control parameter to the server. In addition, the user interaction interface can also include a video display area to display the video encoded based on the searched target encoding parameter value after the server adjusts the first reference gradient based on the first control parameter and / or adjusts the second reference gradient based on the second control parameter.

[0145] In actual applications, the user operates the control in any of the following ways: click, double-click, touch, mouse hover, slide, long press, voice control, or shake, etc., which is selected according to the actual situation, and the embodiments of this specification do not limit this. The user inputs the first control parameter to adjust the first reference gradient and the second control parameter to adjust the second reference gradient through the user interaction interface, and flexibly adjusts the decision strategy, which can achieve higher application flexibility.

[0146] It can be seen from the above embodiments that by adjusting factor1 and / or factor2, a flexible trade-off can be made between a higher overall VMAF mean value of multimedia data and a more stable overall VMAF value of multimedia data. In some application scenarios, it is also possible to improve the quality of low-quality content (improve image quality) or reduce the quality of high-quality content (reduce costs) by simply adjusting factor1 or factor2 as needed. By configuring different control parameter values ​​and statistically analyzing the quality distribution of the transcoded bitstream of online multimedia data, it is found that the quality can be more stable when transcoding according to the encoding parameter values ​​determined in the above embodiments.

[0147] In one or more embodiments of this specification, the preset output condition in step 208 includes:

[0148] In the case where the current quality indicator value is greater than the target quality indicator value, the updated current quality indicator value should be less than or equal to the target quality indicator value;

[0149] In the case where the current quality indicator value is less than or equal to the target quality indicator value, the updated current quality indicator value should be greater than or equal to the target quality indicator value.

[0150] For example: Assume that the encoding parameters have been adjusted up for multiple cycles, and the quality index value has dropped from the original 90 to the current quality index value of 70, the target quality index value is 65, and the updated current quality index value is 62. Since the updated current quality index value is less than the target quality index value, it can be determined that if it continues to be adjusted up, the difference between the current quality index value and the target quality index value will become increasingly larger. At this time, the updated current quality index value can be considered to be closest to the target quality index value, thereby meeting the preset output conditions.

[0151] For another example, assuming that the encoding parameters have been adjusted down through multiple cycles, the quality index value has increased from the original 40 to the current quality index value of 60, the target quality index value is 65, and the updated current quality index value is 66. Since the updated current quality index value is greater than the target quality index value, it can be determined that if the value is further reduced, the difference between the current quality index value and the target quality index value will become increasingly larger. At this time, the updated current quality index value can be considered to be closest to the target quality index value, thereby satisfying the preset output conditions.

[0152] Based on this, in order to search for a target encoding parameter value that better meets the requirements, the method may further include:

[0153] When the difference between the updated current quality indicator value and the target quality indicator value does not meet the preset output condition, the step of adjusting the current encoding parameter value according to the target adjustment strategy to obtain the updated current encoding parameter value is continued for the updated current quality indicator value and the updated current encoding parameter value.

[0154] In this embodiment, the encoding parameter value is adjusted through cyclic analysis until the difference between the updated current quality index value obtained in the last cycle and the target quality index value meets the preset output condition, then the loop is stopped and the updated current encoding parameter value obtained in the last cycle is used as the target encoding parameter value for encoding the multimedia data, ensuring that the target encoding parameter value can be obtained to ensure the stability of the multimedia data quality and avoid bit rate waste.

[0155] See also Figure 4 , Figure 4 A flowchart showing a processing process of a multimedia data encoding method provided by an embodiment of the present specification in a video application is shown, which specifically includes the following steps:

[0156] Step 402: extract features from the source video or the video segments of the source video to obtain video features.

[0157] The source video may be a video uploaded by a user, or a video obtained from any multimedia platform or multimedia database.

[0158] Step 404: Input the video features into the bit rate quality prediction model to obtain a corresponding change curve of the bit rate quality of the source video predicted by the bit rate quality prediction model.

[0159] The specific implementation of the bit rate quality prediction model can refer to the description of the above embodiment and will not be repeated here.

[0160] Step 406: The encoding parameter value in the fixed encoding parameter mode is used as the initial current encoding parameter value, and the mean value of the quality index of the reference video encoded in the fixed encoding parameter mode is counted to obtain the target quality index value.

[0161] Step 408: Determine the current quality index value based on the source video bit rate quality corresponding change curve and the current encoding parameter value.

[0162] Step 410: Determine whether the current quality index value is greater than the target quality index value.

[0163] If yes, then calculate the first gradient and the first reference gradient and proceed to step 412 .

[0164] If not, then calculate the second gradient and the second reference gradient and proceed to step 418 .

[0165] The calculation methods of the first gradient, the first reference gradient, the second gradient and the second reference gradient can refer to the description of the aforementioned embodiments and will not be described again here.

[0166] Step 412: Determine whether the first gradient is less than or equal to the first reference gradient.

[0167] If yes, proceed to step 414 .

[0168] If not, the current encoding parameter value is used as the target encoding parameter value, and the process ends to output the target encoding parameter value.

[0169] Step 414: According to a preset step size, the current encoding parameter value is increased to obtain an updated current encoding parameter value, and a corresponding updated current quality indicator value is determined.

[0170] Step 416: Determine whether the updated current quality indicator value is still greater than the target quality indicator value.

[0171] If yes, then for the updated current encoding parameter value and the updated current quality indicator value, the process returns to step 410, or, for the updated current encoding parameter value, the first gradient and the first reference gradient are recalculated, and the process returns to step 412. If no, the updated current encoding parameter value is used as the target encoding parameter value, and the process ends and the step of outputting the target encoding parameter value is entered.

[0172] Step 418: Determine whether the second gradient is greater than or equal to the second reference gradient.

[0173] If yes, proceed to step 420 .

[0174] If not, the current encoding parameter value is used as the target encoding parameter value, and the process ends to output the target encoding parameter value.

[0175] Step 420: According to a preset step size, the current encoding parameter value is adjusted down to obtain an updated current encoding parameter value, and a corresponding updated current quality indicator value is determined.

[0176] Step 422: Determine whether the updated current quality index value is still less than or equal to the target quality index value.

[0177] If yes, then for the updated current encoding parameter value and the updated current quality indicator value, the process returns to step 410, or, for the updated current encoding parameter value, the second gradient and the second reference gradient are recalculated, and the process returns to step 418. If no, the updated current encoding parameter value is used as the target encoding parameter value, and the process ends and the step of outputting the target encoding parameter value is entered.

[0178] In the above-mentioned embodiment, during the process of searching for the target encoding parameter value, the gain or loss of video quality and bit rate can be effectively evaluated, so as to achieve more video bit rate savings at the cost of less video quality loss, or more video quality improvement at the cost of less bit rate waste, thereby searching for the target encoding parameter value that achieves a better balance between video quality and video bit rate.

[0179] Corresponding to the above method embodiment, this specification also provides a multimedia data encoding device embodiment, Figure 5 FIG. 1 is a schematic diagram showing the structure of a multimedia data encoding device provided by an embodiment of the present specification. Figure 5 As shown, the device comprises:

[0180] The current value determination module 502 is configured to determine the current quality index value based on the corresponding change relationship between the bit rate and quality of the multimedia data and the current encoding parameter value of the target encoding parameter, wherein the corresponding change relationship between the bit rate and quality includes: multiple encoding parameter values ​​of the target encoding parameter, the corresponding change relationship between the bit rate and the quality index value, and the current encoding parameter value is any one of the multiple encoding parameter values.

[0181] The parameter value updating module 504 is configured to adjust the current encoding parameter value according to the target adjustment strategy to obtain an updated current encoding parameter value.

[0182] The quality indicator updating module 506 is configured to determine an updated current quality indicator value based on the bit rate quality corresponding change relationship and the updated current encoding parameter value.

[0183] The output module 508 is configured to use the updated current encoding parameter value as a target encoding parameter value for encoding the multimedia data when the difference between the updated current quality indicator value and the target quality indicator value meets a preset output condition.

[0184] Optionally, the device further comprises:

[0185] The initialization module is configured to use the encoding parameter value of the target encoding parameter in the fixed encoding parameter mode as the initial current encoding parameter value, calculate the mean value of the quality index of the reference multimedia data encoded in the fixed encoding parameter mode, and obtain the target quality index value.

[0186] Optionally, the device further comprises:

[0187] The loop trigger module is configured to continue to execute the step of adjusting the current encoding parameter value according to the target adjustment strategy to obtain an updated current encoding parameter value for the updated current quality indicator value and the updated current encoding parameter value when the difference between the updated current quality indicator value and the target quality indicator value does not meet the preset output condition.

[0188] Optionally, the device further comprises:

[0189] The adjustment direction determining module is configured to determine the adjustment direction of the current encoding parameter value according to the difference between the current quality indicator value and the target quality indicator value.

[0190] The adjustment strategy determination module is configured to determine a target adjustment strategy according to a gradient of a current encoding parameter value in the adjustment direction on a curve corresponding to the bit rate quality corresponding change relationship.

[0191] Optionally, the adjustment strategy determination module is configured to determine whether to increase the current encoding parameter value based on a first gradient in the increasing direction on the curve corresponding to the bit rate-quality change relationship of the current encoding parameter value when the adjustment direction is an increasing direction, and if so, determine that the target adjustment strategy is to increase the current encoding parameter value; and to determine whether to decrease the current encoding parameter value based on a second gradient in the decreasing direction on the curve corresponding to the bit rate-quality change relationship of the current encoding parameter value when the adjustment direction is a decreasing direction, and if so, determine that the target adjustment strategy is to decrease the current encoding parameter value.

[0192] Optionally, the adjustment strategy determination module is configured to determine the first gradient of the current encoding parameter value in the increasing direction on the curve corresponding to the corresponding change relationship of the bit rate quality, and determine to increase the current encoding parameter value when the first gradient is less than or equal to the first reference gradient. The first reference gradient is the gradient of the current encoding parameter value in the increasing direction on the historical true value change average line, and the historical true value change average line is the average line of the corresponding change relationship of the encoding parameter value, bit rate and quality index value determined based on historical multimedia data. The adjustment strategy determination module is configured to determine the second gradient of the current encoding parameter value in the decreasing direction on the curve corresponding to the corresponding change relationship of the bit rate quality, and determine to decrease the current encoding parameter value when the second gradient is greater than or equal to the second reference gradient. The second reference gradient is the gradient of the current encoding parameter value in the decreasing direction on the historical true value change average line.

[0193] Optionally, the output module is further configured to determine that the current encoding parameter value is a target encoding parameter value for encoding multimedia data when the first gradient is greater than a first reference gradient, and to determine that the current encoding parameter value is a target encoding parameter value for encoding multimedia data when the second gradient is less than the second reference gradient.

[0194] Optionally, the device also includes: a control parameter acquisition module, configured to acquire a first control parameter, and adjust the size of the first reference gradient using the first control parameter; and / or acquire a second control parameter, and adjust the size of the second reference gradient using the second control parameter.

[0195] Optionally, the device also includes: a prediction module, configured to use a rate-quality prediction model to predict the rate-quality corresponding change relationship of the multimedia data, and obtain the rate-quality corresponding change relationship, in which the coding parameter value is a plurality of coding parameter values ​​within a preset coding parameter value range of the target coding parameter.

[0196] The above is a schematic scheme of a multimedia data encoding device of this embodiment. It should be noted that the technical scheme of the multimedia data encoding device and the technical scheme of the multimedia data encoding method described above belong to the same concept, and the details not described in detail in the technical scheme of the multimedia data encoding device can be referred to the description of the technical scheme of the multimedia data encoding method described above.

[0197] Corresponding to the above method embodiment, this specification also provides a video data encoding method. Figure 6 , Figure 6A flowchart of a video data encoding method provided by an embodiment of the present specification is shown, which specifically includes the following steps:

[0198] Step 602: Determine the current quality index value based on the corresponding change relationship between the bit rate and quality of the video data and the current CRF parameter value of the CRF parameter, wherein the corresponding change relationship between the bit rate and quality includes: multiple CRF parameter values ​​of the CRF parameter, the corresponding change relationship between the bit rate and the quality index value, and the current CRF parameter value is any one of the multiple CRF parameter values.

[0199] Step 604: According to the target adjustment strategy, the current CRF parameter value is adjusted to obtain an updated current CRF parameter value.

[0200] Step 606: Determine an updated current quality indicator value based on the bit rate quality corresponding change relationship and the updated current CRF parameter value.

[0201] Step 608: When the difference between the updated current quality index value and the target quality index value meets a preset output condition, the updated current CRF parameter value is used as a target CRF parameter value for encoding the video data.

[0202] After obtaining the target CRF parameter value, the target CRF parameter value may be used to encode the video data to obtain encoded video data.

[0203] It should be noted that the specific implementation details of the above steps 602-608 can refer to the description of one or more of the above embodiments, such as the description of steps 202-208, and this embodiment will not be repeated in detail here.

[0204] See also Figure 7 , Figure 7 A flowchart showing a processing process of applying a video data encoding method provided by an embodiment of the present specification in an adaptive CRF encoding parameter strategy is shown, which specifically includes the following steps:

[0205] Step 702: Initialization.

[0206] Before the initialization step, the source video can be segmented to obtain video segments of the source video. Feature extraction is performed on the video segments to obtain video features. The video features are input into the bit rate quality prediction model to obtain the bit rate quality corresponding change curve predicted by the bit rate quality prediction model. In addition, the bit rate and quality true value of the multimedia training data used to train the bit rate quality prediction model can be statistically analyzed to obtain the true value bitrate-vmaf average line. The bit rate quality corresponding change curve and the true value bitrate-vmaf average line are input into the strategy module of the encoder, so that the strategy module of the encoder enters the initialization step according to the method provided in this embodiment.

[0207] The initialization in this step includes: initializing org_crf and org_vmaf. The current CRF parameter value curr_crf = org_crf, and the current quality index value curr_vmaf = org_vmaf. The VMAF mean value in the fixed CRF mode on the statistical line is used as the target quality index value target_vmaf. The true value bitrate-vmaf average line is used for analysis to obtain the initial reference gradient on line_grad corresponding to the org_crf value. On line_grad can refer to the first reference gradient or the second reference gradient according to the difference between the two directions of increasing or decreasing. The bit rate quality corresponding change curve of the current video or slice content to be encoded is used for analysis to obtain the first gradient corresponding to the current encoding parameter value curr_crf in the increasing direction or the second gradient corresponding to the decreasing direction, which is recorded as curr_grad.

[0208] Step 704: Determine whether the current quality indicator value curr_vmaf is greater than the target quality indicator value target_vmaf.

[0209] If yes, it means that the encoding quality using the current curr_crf is too high, there is a waste of bit rate, and curr_crf needs to be increased, and the process goes to step 706. If no, it means that the bit rate is insufficient, and curr_crf needs to be reduced, and the process goes to step 712.

[0210] Step 706: Determine whether curr_grad is less than or equal to online_grad*factor1.

[0211] In this step, curr_grad refers to the first gradient corresponding to curr_crf in the increasing direction on the bit rate quality corresponding change relationship curve; on line_grad refers to the first reference gradient corresponding to curr_crf in the increasing direction on the historical true value change moving average.

[0212] In this step, if the judgment is yes, it means that increasing curr_crf at this time can save more bit rate, and VMAF will not drop too much. Therefore, step 708 can be entered at this time; if the judgment is no, it means that there is no need to continue to increase it, and the current curr_crf is output as the target CRF, and the process ends.

[0213] Step 708: Increase the current curr_crf to obtain an updated curr_crf.

[0214] In this step, the current quality indicator value curr_vmaf and the first gradient curr_grad may also be updated accordingly, and then the process proceeds to step 710 .

[0215] Step 710: Determine whether the updated current quality indicator value curr_vmaf is greater than the target quality indicator value target_vmaf.

[0216] If the judgment is yes, it means that curr_vmaf needs to be further adjusted. If the judgment is no, it means that it does not need to be further increased. The updated curr_crf is used as the target CRF, and the process ends and the step of outputting the target CRF is entered.

[0217] Through the above steps 706-710, after calculating the curr_grad and online_grad corresponding to the current encoding parameter value in the direction of increasing, it is determined whether curr_grad is less than or equal to online_grad. If so, it means that if the current CRF parameter value is continued to be increased, more bitrate can be saved, and VMAF will not drop too much. If not, it means that if the current CRF parameter value is continued to be increased, it cannot save more bitrate relative to the average line, and the benefit is too small. No further adjustment is required, and the current CRF parameter value should be output as the target CRF parameter value. For example: assuming that online_grad = 0.01, it means that the average bitrate of online multimedia data decreases by 100kb and VMAF decreases by 1. At this time, if curr_grad = 0.005, it means that for the multimedia data to be encoded, the bitrate decreases by 200kb and VMAF will decrease by 1. At this time, the current CRF parameter value can be increased to save bitrate, and VMAF will not decrease too much. If it is determined to continue to increase the current CRF parameter value, the current CRF parameter value curr_crf can be increased by a step size (assuming it is 0.05) to obtain the updated current CRF parameter value, and according to the bitrate-vmaf curve, find the updated current quality index value corresponding to the updated current CRF parameter value. If the updated current quality index value is still greater than the target quality index value, the updated current quality index value is used as the updated curr_vmaf, and the updated current encoding parameter value is used as the updated curr_crf, and the current CRF parameter value is continued to be increased. Otherwise, the updated current encoding parameter value is used as the target encoding parameter value, and the search ends.

[0218] Step 712: Determine whether curr_grad is greater than or equal to online_grad*factor2.

[0219] In this step, curr_grad refers to the second gradient of curr_crf in the decreasing direction on the bit rate quality corresponding change relationship curve; online_grad refers to the first reference gradient of curr_crf in the decreasing direction on the historical true value change moving average.

[0220] In this step, if the judgment is yes, it means that reducing curr_crf at this time can improve the quality a lot without increasing the bit rate a lot. Therefore, step 714 can be entered at this time. If the judgment is no, it means that there is no need to continue to reduce it, and the current curr_crf is output as the target CRF, and the process ends.

[0221] Step 714: Decrease the current curr_crf to obtain an updated curr_crf.

[0222] For example, the current curr_crf may be reduced by a preset step size to obtain an updated curr_crf. In this step, the current quality indicator value curr_vmaf and the second gradient curr_grad may also be updated accordingly, and then step 716 is entered.

[0223] Step 716: Determine whether the updated current quality indicator value curr_vmaf is less than the target quality indicator value target_vmaf.

[0224] If the judgment is yes, it means that curr_vmaf needs to be further adjusted. If the judgment is no, it means that there is no need to continue to reduce it. The current curr_crf is output as the target CRF parameter value and the process ends.

[0225] Through the above steps 712-716, after calculating the curr_grad and online_grad corresponding to the current coding parameter value in the direction of decreasing, it is determined whether curr_grad is greater than or equal to online_grad. If yes, it means that if CRF is further decreased at this time, VMAF can be improved more, and the bit rate will not increase too much. If not, it means that if CRF is further decreased at this time, it is not possible to improve more quality relative to the moving average while bringing less bit rate increase, and the benefit is not large. No further adjustment is required, and the current coding parameter value should be output as the target coding parameter value. Thus, the current coding parameter value curr_crf can be decreased by one step to obtain the updated current coding parameter value, and according to the bitrate-vmaf curve, the updated current quality index value corresponding to the updated current coding parameter value is found. If the updated current quality index value is still less than the target quality index value, the updated current quality index value is used as the updated curr_vmaf, and the updated current coding parameter value is used as the updated curr_crf, and the current coding parameter value is continued to be decreased. Otherwise, the updated current coding parameter value is used as the target coding parameter value, and the search is terminated.

[0226] According to the above processing, the strategy module of the encoder outputs the target CRF parameter value corresponding to the video segment, so that the encoder encodes the video segment using the target CRF parameter value to obtain the encoded video segment.

[0227] It should be noted that, according to the method provided in the embodiments of this specification, the processing of the method can be performed on a video to determine a target encoding parameter value for the video, or the processing of the method can be performed on different video segments of a video respectively, so that different target encoding parameter values ​​can be determined by performing the processing of the method on multiple video segments of a video respectively. The specific data granularity for determining the target encoding parameter value can be set according to actual application needs, and this specification does not limit this.

[0228] Figure 8 A schematic diagram of mass distribution provided by one or more embodiments of the present specification is shown. Figure 8 The figure shows the quality distribution of a certain online video transcoded stream under three control parameter configurations, with VMAF as the horizontal axis and density as the vertical axis. Specifically, Figure 8 The solid line curve in (a) is when both factor1 and factor2 are 1. Figure 7 The target CRF parameter value obtained by the processing shown in the figure shows the quality distribution of the bitstream after video transcoding; Figure 8 The solid line curve in (b) is when both factor1 and factor2 are 1.6. Figure 7 The target CRF parameter value obtained by the processing shown in the figure shows the quality distribution of the bitstream after video transcoding; Figure 8 The solid curve in (c) is when both factor1 and factor2 are infinite. Figure 7 The target CRF parameter value obtained by the processing shown in the figure shows the quality distribution of the bitstream after video transcoding. Figure 8 The dashed curve in the figure is the quality distribution of the video transcoded online with a fixed CRF parameter value. Figure 7 The target CRF parameter values ​​obtained by the processing shown in the figure are used for transcoding, and the quality distribution is stable and improved.

[0229] The above-mentioned embodiment, based on the analysis of the corresponding change relationship curve of bit rate quality, can not only stabilize the VMAF value of the transcoded video within a relatively narrow range, but also use the gradient of the combined curve to decide the adjustment of the adaptive CRF, and can decide whether the current video or video slice needs to reduce the CRF value (corresponding to increase the VMAF value) or increase the CRF value (corresponding to reduce the VMAF value) according to the gradient (which can also be understood as the benefit ratio of quality-bit rate), and finally adaptively predict the target CRF value of each video or each video slice, and use the target CRF value for encoding to ensure that the VMAF distribution range of the transcoded video is narrowed while the VMAF mean is still improved, and the bit rate waste is avoided. Exemplarily, the method provided in the embodiment of this specification can be applied to the transcoding of narrowband high-definition code streams in audio and video architecture media transcoding, so that the transcoded content has stable quality.

[0230] Corresponding to the above method embodiment, this specification also provides a video data encoding device embodiment, Fig. 9 FIG. 2 shows a schematic diagram of the structure of a video data encoding device provided by an embodiment of the present specification. Fig. 9 As shown, the device comprises:

[0231] The current determination module 902 is configured to determine the current quality index value based on the corresponding change relationship between the bit rate and quality of the video data and the current CRF parameter value of the CRF parameter, wherein the corresponding change relationship between the bit rate and quality includes: multiple CRF parameter values ​​of the CRF parameter, the corresponding change relationship between the bit rate and the quality index value, and the current CRF parameter value is any one of the multiple CRF parameter values.

[0232] The CRF updating module 904 is configured to adjust the current CRF parameter value according to the target adjustment strategy to obtain an updated current CRF parameter value.

[0233] The quality updating module 906 is configured to determine an updated current quality indicator value based on the bit rate quality corresponding change relationship and the updated current CRF parameter value.

[0234] The target CRF output module 908 is configured to use the updated current CRF parameter value as the target CRF parameter value for encoding the video data when the difference between the updated current quality indicator value and the target quality indicator value meets a preset output condition.

[0235] The above is a schematic scheme of a video data encoding device of this embodiment. It should be noted that the technical scheme of the video data encoding device and the technical scheme of the above video data encoding method belong to the same concept, and the details not described in detail in the technical scheme of the video data encoding device can be referred to the description of the technical scheme of the above video data encoding method.

[0236] Fig.10 The block diagram of a computing device 1000 according to an embodiment of the present specification is shown. The components of the computing device 1000 include but are not limited to a memory 1010 and a processor 1020. The processor 1020 is connected to the memory 1010 via a bus 1030, and the database 1050 is used to store data.

[0237] The computing device 1000 also includes an access device 1040 that enables the computing device 1000 to communicate via one or more networks 1060. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1040 may include one or more of any type of network interface, wired or wireless (e.g., a network interface card (NIC)), such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, and a Near Field Communication (NFC).

[0238] In one embodiment of the present specification, the above components of the computing device 1000 and Fig.10 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig.10 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0239] The computing device 1000 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 1000 may also be a mobile or stationary server.

[0240] The memory 1010 is used to store computer programs / instructions, and the processor 1020 is used to execute the computer programs / instructions stored in the memory 1010. When the computer programs / instructions are executed by the processor, the steps of the method described in any of the above embodiments are implemented.

[0241] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the method described in any of the above embodiments belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the method described in any of the above embodiments.

[0242] An embodiment of the present specification further provides a computer-readable storage medium storing a computer program / instruction, which implements the steps of the method described in any of the above embodiments when executed by a processor.

[0243] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium belongs to the same concept as the technical scheme described in any of the above embodiments, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the method described in any of the above embodiments.

[0244] An embodiment of the present specification also provides a computer program product, including a computer program / instruction, which implements the steps of the method described in any of the above embodiments when executed by a processor.

[0245] The above is a schematic scheme of a computer program product of this embodiment. It should be noted that the technical scheme of the computer program product and the technical scheme of the method described in any of the above embodiments belong to the same concept, and the details not described in detail in the technical scheme of the computer program product can be referred to the description of the technical scheme of the method described in any of the above embodiments.

[0246] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0247] The computer program / instruction includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer readable media do not include electric carrier signals and telecommunication signals.

[0248] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0249] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0250] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A multimedia data encoding method, It is characterized in that include: Determine a current quality index value based on a corresponding change relationship between bit rate and quality of the multimedia data and a current encoding parameter value of the target encoding parameter, wherein the corresponding change relationship between bit rate and quality includes: a corresponding change relationship between multiple encoding parameter values ​​of the target encoding parameter, bit rate and quality index value, and the current encoding parameter value is any one of the multiple encoding parameter values; According to the target adjustment strategy, the current encoding parameter value is adjusted to obtain an updated current encoding parameter value; Determining an updated current quality indicator value based on the bit rate quality corresponding change relationship and the updated current encoding parameter value; When the difference between the updated current quality indicator value and the target quality indicator value meets a preset output condition, the updated current encoding parameter value is used as a target encoding parameter value for encoding the multimedia data.

2. The method according to claim 1, It is characterized in that Before adjusting the current encoding parameter value according to the target adjustment strategy to obtain the updated current encoding parameter value, the following steps are included: Determining an adjustment direction of a current encoding parameter value according to a difference between the current quality indicator value and the target quality indicator value; A target adjustment strategy is determined according to a gradient of a current encoding parameter value in the adjustment direction on a curve corresponding to the bit rate quality change relationship.

3. The method according to claim 2, It is characterized in that Determining a target adjustment strategy according to the gradient of the current encoding parameter value in the adjustment direction on the curve corresponding to the bit rate quality corresponding change relationship includes: In the case where the adjustment direction is an increasing direction, determining whether to increase the current encoding parameter value according to a first gradient in the increasing direction on the curve corresponding to the bit rate quality corresponding change relationship of the current encoding parameter value, and if so, determining that the target adjustment strategy is to increase the current encoding parameter value; In the case where the adjustment direction is a decreasing direction, according to the second gradient in the decreasing direction on the curve corresponding to the bit rate quality corresponding change relationship of the current encoding parameter value, it is determined whether to decrease the current encoding parameter value. If so, the target adjustment strategy is determined to decrease the current encoding parameter value.

4. The method according to claim 3, It is characterized in that Determining whether to increase the current encoding parameter value according to a first gradient in an increasing direction on a curve corresponding to the bit rate quality corresponding change relationship of the current encoding parameter value includes: Determine a first gradient of the current encoding parameter value in an increasing direction on a curve corresponding to the bit rate quality corresponding change relationship; When the first gradient is less than or equal to the first reference gradient, determining to increase the current encoding parameter value; The first reference gradient is the gradient of the current encoding parameter value on the historical true value change moving average in the adjustment direction, and the historical true value change moving average is the moving average of the corresponding change relationship between the encoding parameter value, the bit rate and the quality index value determined based on the historical multimedia data; Determining whether to reduce the current encoding parameter value according to a second gradient in a decreasing direction on a curve corresponding to the bit rate quality corresponding change relationship of the current encoding parameter value includes: Determine a second gradient of the current encoding parameter value in a decreasing direction on the curve corresponding to the bit rate quality change relationship; When the second gradient is greater than or equal to a second reference gradient, determining to decrease the current encoding parameter value; The second reference gradient is the gradient of the current encoding parameter value on the historical true value change moving average in the decreasing direction.

5. The method according to claim 4, It is characterized in that Also includes: In a case where the first gradient is greater than a first reference gradient, determining the current encoding parameter value to be a target encoding parameter value for encoding multimedia data; In a case where the second gradient is less than the second reference gradient, the current encoding parameter value is determined as a target encoding parameter value for encoding multimedia data.

6. The method according to claim 4, It is characterized in that Also includes: Acquire a first control parameter, and use the first control parameter to adjust the magnitude of the first reference gradient; and / or, A second control parameter is obtained, and the magnitude of the second reference gradient is adjusted using the second control parameter.

7. The method according to claim 1, It is characterized in that Also includes: When the difference between the updated current quality indicator value and the target quality indicator value does not meet the preset output condition, the step of adjusting the current encoding parameter value according to the target adjustment strategy to obtain the updated current encoding parameter value is continued for the updated current quality indicator value and the updated current encoding parameter value.

8. The method according to claim 1 or 7, It is characterized in that The preset output conditions include: In the case where the current quality indicator value is greater than the target quality indicator value, the updated current quality indicator value should be less than or equal to the target quality indicator value; In the case that the current quality indicator value is less than or equal to the target quality indicator value, the updated current quality indicator value should be greater than or equal to the target quality indicator value.

9. The method according to claim 1, It is characterized in that Also includes: The bit rate quality prediction model is used to predict the bit rate quality corresponding change relationship of the multimedia data to obtain the bit rate quality corresponding change relationship, in which the encoding parameter value is a plurality of encoding parameter values ​​within the preset encoding parameter value range of the target encoding parameter.

10. A method for encoding video data, It is characterized in that include: Determine a current quality index value based on a corresponding change relationship between bit rate and quality of the video data and a current CRF parameter value of the CRF parameter, wherein the corresponding change relationship between bit rate and quality includes: a corresponding change relationship between multiple CRF parameter values, bit rate and quality index values ​​of the CRF parameter, and the current CRF parameter value is any one of the multiple CRF parameter values; According to the target adjustment strategy, the current CRF parameter value is adjusted to obtain an updated current CRF parameter value; Determining an updated current quality indicator value based on the bit rate quality corresponding change relationship and the updated current CRF parameter value; When the difference between the updated current quality indicator value and the target quality indicator value meets a preset output condition, the updated current CRF parameter value is used as a target CRF parameter value for encoding the video data.

11. A multimedia data encoding device, It is characterized in that include: A current value determination module is configured to determine a current quality index value based on a corresponding change relationship between bit rate and quality of multimedia data and a current encoding parameter value of a target encoding parameter, wherein the corresponding change relationship between bit rate and quality includes: a corresponding change relationship between a plurality of encoding parameter values ​​of a target encoding parameter, a bit rate and a quality index value, and the current encoding parameter value is any one of the plurality of encoding parameter values; The parameter value updating module is configured to adjust the current encoding parameter value according to the target adjustment strategy to obtain an updated current encoding parameter value; A quality indicator updating module, configured to determine an updated current quality indicator value based on the bit rate quality corresponding change relationship and the updated current encoding parameter value; The output module is configured to use the updated current encoding parameter value as a target encoding parameter value for encoding the multimedia data when a difference between the updated current quality indicator value and the target quality indicator value meets a preset output condition.

12. A video data encoding device, It is characterized in that include: A current determination module is configured to determine a current quality index value based on a corresponding change relationship between bit rate and quality of the video data and a current CRF parameter value of the CRF parameter, wherein the corresponding change relationship between bit rate and quality includes: a corresponding change relationship between multiple CRF parameter values ​​of the CRF parameter, bit rate and quality index value, and the current CRF parameter value is any one of the multiple CRF parameter values; A CRF update module is configured to adjust the current CRF parameter value according to the target adjustment strategy to obtain an updated current CRF parameter value; A quality update module, configured to determine an updated current quality indicator value based on the bit rate quality corresponding change relationship and the updated current CRF parameter value; The target CRF output module is configured to use the updated current CRF parameter value as a target CRF parameter value for encoding the video data when a difference between the updated current quality indicator value and the target quality indicator value meets a preset output condition.

13. A computing device, It is characterized in that include: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, the steps of the method described in any one of claims 1 to 10 are implemented.

14. A computer-readable storage medium, It is characterized in that It stores a computer program / instruction, which implements the steps of the method described in any one of claims 1 to 10 when executed by a processor.

15. A computer program product, It is characterized in that The invention comprises a computer program / instruction, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 10.

Images