Video processing method and device, electronic equipment and storage medium

Abstract

The invention provides a video processing method and device, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: performing enhancement decision on a to-be-processed video based on an enhancement decision model to obtain a target enhancement operator, and performing enhancement processing by using the target enhancement operator; based on a preprocessing decision model, performing preprocessing decision on the to-be-processed video according to the target enhancement operator, obtaining a target preprocessing operator, and performing preprocessing by using the target preprocessing operator; and based on the coding decision model, according to the target enhancement operator and the target pre-processing operator, carrying out coding decision on the pre-processed video, obtaining a target coding parameter, carrying out coding by using the target coding parameter, and finally generating a video code stream. According to the method, the multi-stage pruning technology is adopted, the calculation complexity of full-link optimal parameter selection is reduced, a step-by-step guide decision mechanism is constructed, the accuracy and the collaboration of parameter prediction are improved, and video processing full-link self-adaptive parameter optimization is achieved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a video processing method, a video processing apparatus, an electronic device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] With the development of artificial intelligence and audio / video technology, video quality and viewing experience have become key focuses in the industry. From upload to consumption, videos undergo multiple processing stages, including quality enhancement, preprocessing, encoding, and distribution. Each stage involves multiple candidate operators or model parameters. Due to differences in video content characteristics and quality, different videos have different optimal processing parameters to achieve the best possible video quality and bitrate for the user. Directly finding the optimal parameter combination would require a full traversal of the Cartesian product of all parameters, potentially resulting in hundreds of millions of parameter combinations for a single video.

[0003] Related technologies rely on manually designed rules for selecting video enhancement and preprocessing parameters, which is time-consuming and has limited rules, and does not achieve adaptive parameter decision-making at the video granularity. Furthermore, due to the high search cost for the optimal parameter combination in the entire video processing and encoding chain, the solution space formed by the Cartesian product of parameters is unacceptable, and related technologies only predict some parameters in the encoding stage. Summary of the Invention

[0004] This disclosure provides a video processing method, video processing apparatus, electronic device, computer-readable storage medium, and computer program product to overcome or at least partially solve the above-mentioned problems.

[0005] This disclosure provides a video processing method, comprising: acquiring a video to be processed; performing enhancement decisions on the video to be processed based on an enhancement decision model to obtain a target enhancement operator corresponding to the video to be processed; performing enhancement processing on the video to be processed using the target enhancement operator to obtain an enhanced video; performing preprocessing decisions on the video to be processed based on a preprocessing decision model according to the target enhancement operator to obtain a target preprocessing operator corresponding to the video to be processed; performing preprocessing on the enhanced video using the target preprocessing operator to obtain a preprocessed video; performing encoding decisions on the preprocessed video based on an encoding decision model according to the target enhancement operator and the target preprocessing operator to obtain target encoding parameters corresponding to the video to be processed; and encoding the preprocessed video using the target encoding parameters to generate a video stream.

[0006] In one exemplary embodiment of this disclosure, the step of performing enhancement decision on the video to be processed based on the enhancement decision model to obtain the target enhancement operator corresponding to the video to be processed includes: precoding the first video segment of the video to be processed to obtain a first precoding feature; inputting the first precoding feature and the basic features of the video to be processed into the enhancement decision model to obtain the target enhancement decision result of the video to be processed; and determining the target enhancement operator according to the target enhancement decision result and preset enhancement constraint rules.

[0007] In one exemplary embodiment of this disclosure, the augmented decision model is trained using the following method: obtaining a first training video set; the first training video set includes multiple first training videos; precoding the first training video segment of the first training video to obtain first training encoded features; inputting the first training encoded features and the basic features of the first training video into a first initial classification model to obtain the predicted augmented decision result of the first training video; and training the first initial classification model based on the augmented label corresponding to the first training video and the predicted augmented decision result of the first training video to obtain the augmented decision model.

[0008] In one exemplary embodiment of this disclosure, the method further includes: obtaining at least one candidate enhancement operator; for each of the at least one candidate enhancement operator, performing enhancement processing on the first training video using the candidate enhancement operator to obtain an enhancement processing result corresponding to the candidate enhancement operator; encoding the enhancement processing result corresponding to the candidate enhancement operator based on at least one first constant rate factor to generate an encoding performance curve corresponding to the candidate enhancement operator; determining the performance gain corresponding to the candidate enhancement operator based on the encoding performance curve corresponding to the candidate enhancement operator; selecting the maximum value among the performance gains corresponding to the at least one candidate enhancement operator, and using the candidate enhancement operator corresponding to the maximum value as the enhancement label corresponding to the first training video.

[0009] In one exemplary embodiment of this disclosure, the target preprocessing operator includes segment preprocessing operators corresponding to each video segment to be processed in the video to be processed; wherein, the step of performing preprocessing decisions on the video to be processed based on the preprocessing decision model and according to the target enhancement operator to obtain the target preprocessing operator corresponding to the video to be processed includes: for each video segment to be processed in the video to be processed, precoding the video segment to be processed to obtain a second precoding feature; inputting the second precoding feature, the target enhancement operator, and the basic features of the video to be processed into the preprocessing decision model to obtain the segment preprocessing operator corresponding to the video segment to be processed.

[0010] In one exemplary embodiment of this disclosure, the preprocessing decision model is trained using the following method: obtaining a second training video set; the second training video set includes multiple second training videos, and each second training video includes multiple second training video segments; precoding the second training video segments to obtain second training encoded features; inputting the second training encoded features, the basic features of the second training videos, and the enhanced labels corresponding to the second training video segments into a second initial classification model to obtain the prediction preprocessing decision results for the second training video segments; and training the second initial classification model based on the preprocessing labels corresponding to the second training video segments and the prediction preprocessing decision results for the second training video segments to obtain the preprocessing decision model.

[0011] In one exemplary embodiment of this disclosure, the method further includes: obtaining at least one candidate preprocessing operator; for each of the at least one candidate preprocessing operator, performing preprocessing on the second training video segment using the candidate preprocessing operator to obtain a preprocessing result corresponding to the candidate preprocessing operator; encoding the preprocessing result corresponding to the candidate preprocessing operator based on at least one second constant rate factor to generate an encoding performance curve corresponding to the candidate preprocessing operator; determining the performance gain corresponding to the candidate preprocessing operator based on the encoding performance curve corresponding to the candidate preprocessing operator; selecting the maximum value among the performance gains corresponding to the at least one candidate preprocessing operator, and using the candidate enhancement operator corresponding to the maximum value as the preprocessing label corresponding to the second training video segment.

[0012] In one exemplary embodiment of this disclosure, the target encoding parameters include segment encoding parameters corresponding to each video segment to be processed in the video to be processed; wherein, the step of obtaining the target encoding parameters corresponding to the video to be processed by performing encoding decisions on the preprocessed video based on the encoding decision model, according to the target enhancement operator and the target preprocessing operator, includes: for each video segment to be processed in the video to be processed, obtaining the preprocessed video segment corresponding to the video segment to be processed; the preprocessed video segment is obtained by processing the video segment to be processed by the target enhancement operator and the target preprocessing operator. The video data obtained after processing by the segmentation preprocessing operator corresponding to the video segment is processed; the preprocessed video segment is pre-encoded to obtain a third pre-encoding feature; the third pre-encoding feature, the target enhancement operator, the segmentation preprocessing operator corresponding to the video segment to be processed, and the basic features of the video segment to be processed are input into the encoding decision model to obtain the initial encoding parameters corresponding to the video segment to be processed; based on the mapping relationship between the constant rate factor and the Lagrange multiplier, the initial encoding parameters corresponding to the video segment to be processed are optimized to obtain the segment encoding parameters corresponding to the video segment to be processed.

[0013] In one exemplary embodiment of this disclosure, the mapping relationship between the constant rate factor and the Lagrange multiplier is generated according to the following method: obtaining an offline video set; the offline video set includes multiple offline videos, and the offline videos include multiple offline video segments; for each of at least one third constant rate factor, encoding the offline video segment based on the third constant rate factor to obtain the encoding result of the offline video segment under the third constant rate factor; determining the optimal Lagrange multiplier corresponding to the third constant rate factor based on the encoding results of all offline video segments in the multiple offline videos under the third constant rate factor; and generating the mapping relationship between the constant rate factor and the Lagrange multiplier based on each of the at least one third constant rate factor and the corresponding optimal Lagrange multiplier.

[0014] This disclosure provides a video processing apparatus, comprising: a video acquisition module configured to acquire a video to be processed; an enhancement decision module configured to perform enhancement decisions on the video to be processed based on an enhancement decision model to obtain a target enhancement operator corresponding to the video to be processed; an enhancement processing model configured to perform enhancement processing on the video to be processed using the target enhancement operator to obtain an enhanced video; a preprocessing decision module configured to perform preprocessing decisions on the video to be processed based on the preprocessing decision model and the target enhancement operator to obtain a target preprocessing operator corresponding to the video to be processed; a preprocessing module configured to perform preprocessing on the enhanced video using the target preprocessing operator to obtain a preprocessed video; an encoding decision module configured to perform encoding decisions on the preprocessed video based on the encoding decision model and the target enhancement operator and the target preprocessing operator to obtain target encoding parameters corresponding to the video to be processed; and an encoding module configured to encode the preprocessed video using the target encoding parameters to generate a video stream.

[0015] In one exemplary embodiment of this disclosure, the enhancement decision module is further configured to: pre-encode the first video segment of the video to be processed to obtain a first pre-encoded feature; input the first pre-encoded feature and the basic features of the video to be processed into the enhancement decision model to obtain a target enhancement decision result for the video to be processed; and determine the target enhancement operator based on the target enhancement decision result and preset enhancement constraint rules.

[0016] In one exemplary embodiment of this disclosure, the apparatus further includes an enhanced decision model training module configured to train the enhanced decision model according to the following method: acquiring a first training video set; the first training video set including a plurality of first training videos; precoding the first training video segment of the first training videos to obtain first training encoded features; inputting the first training encoded features and the basic features of the first training videos into a first initial classification model to obtain the predicted enhanced decision result of the first training videos; and training the first initial classification model according to the enhanced labels corresponding to the first training videos and the predicted enhanced decision result of the first training videos to obtain the enhanced decision model.

[0017] In one exemplary embodiment of this disclosure, the augmentation decision model training module is further configured to: acquire at least one candidate augmentation operator; for each of the at least one candidate augmentation operator, perform augmentation processing on the first training video using the candidate augmentation operator to obtain an augmentation processing result corresponding to the candidate augmentation operator; encode the augmentation processing result corresponding to the candidate augmentation operator based on at least one first constant rate factor to generate an encoding performance curve corresponding to the candidate augmentation operator; determine the performance gain corresponding to the candidate augmentation operator based on the encoding performance curve corresponding to the candidate augmentation operator; select the maximum value among the performance gains corresponding to the at least one candidate augmentation operator, and use the candidate augmentation operator corresponding to the maximum value as the augmentation label corresponding to the first training video.

[0018] In one exemplary embodiment of this disclosure, the target preprocessing operator includes segment preprocessing operators corresponding to each video segment to be processed in the video to be processed; wherein, the preprocessing decision module is further configured to: for each video segment to be processed in the video to be processed, pre-encode the video segment to be processed to obtain a second pre-encoding feature; input the second pre-encoding feature, the target enhancement operator and the basic features of the video to be processed into the preprocessing decision model to obtain the segment preprocessing operator corresponding to the video segment to be processed.

[0019] In one exemplary embodiment of this disclosure, the apparatus further includes a preprocessing decision model training module, configured to train the preprocessing decision model according to the following method: acquiring a second training video set; the second training video set includes multiple second training videos, and each second training video includes multiple second training video segments; precoding the second training video segments to obtain second training encoded features; inputting the second training encoded features, the basic features of the second training videos, and the enhanced labels corresponding to the second training video segments into a second initial classification model to obtain the prediction preprocessing decision results of the second training video segments; and training the second initial classification model according to the preprocessing labels corresponding to the second training video segments and the prediction preprocessing decision results of the second training video segments to obtain the preprocessing decision model.

[0020] In one exemplary embodiment of this disclosure, the preprocessing decision model training module is further configured to: acquire at least one candidate preprocessing operator; for each of the at least one candidate preprocessing operator, preprocess the second training video segment using the candidate preprocessing operator to obtain a preprocessing result corresponding to the candidate preprocessing operator; encode the preprocessing result corresponding to the candidate preprocessing operator based on at least one second constant rate factor to generate an encoding performance curve corresponding to the candidate preprocessing operator; determine the performance gain corresponding to the candidate preprocessing operator based on the encoding performance curve corresponding to the candidate preprocessing operator; select the maximum value among the performance gains corresponding to the at least one candidate preprocessing operator, and use the candidate enhancement operator corresponding to the maximum value as the preprocessing label corresponding to the second training video segment.

[0021] In one exemplary embodiment of this disclosure, the target encoding parameters include segment encoding parameters corresponding to each video segment to be processed in the video to be processed; wherein, the encoding decision module is further configured to: for each video segment to be processed in the video to be processed, obtain a pre-processed video segment corresponding to the video segment to be processed; the pre-processed video segment is video data obtained after processing the video segment to be processed by the target enhancement operator and the segment pre-processing operator corresponding to the video segment to be processed; pre-encode the pre-processed video segment to obtain a third pre-coding feature; input the third pre-coding feature, the target enhancement operator, the segment pre-processing operator corresponding to the video segment to be processed, and the basic features of the video to be processed into the encoding decision model to obtain the initial encoding parameters corresponding to the video segment to be processed; optimize the initial encoding parameters corresponding to the video segment to be processed based on the mapping relationship between the constant rate factor and the Lagrange multiplier to obtain the segment encoding parameters corresponding to the video segment to be processed.

[0022] In one exemplary embodiment of this disclosure, the apparatus further includes a mapping relationship generation module configured to generate a mapping relationship between the constant rate factor and the Lagrange multiplier according to the following method: acquiring an offline video set; the offline video set includes multiple offline videos, and the offline videos include multiple offline video segments; for each of at least one third constant rate factor, encoding the offline video segment based on the third constant rate factor to obtain the encoding result of the offline video segment under the third constant rate factor; determining the optimal Lagrange multiplier corresponding to the third constant rate factor based on the encoding results of all offline video segments in the multiple offline videos under the third constant rate factor; and generating a mapping relationship between the constant rate factor and the Lagrange multiplier based on each of the at least one third constant rate factor and the corresponding optimal Lagrange multiplier.

[0023] This disclosure provides an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions to implement the video processing method described above.

[0024] This disclosure provides a computer-readable storage medium that, when the instructions in the computer-readable storage medium are executed by the processor of an electronic device, enables the electronic device to perform the video processing method described above.

[0025] This disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the video processing method described above.

[0026] The video processing method provided in this disclosure employs a multi-stage pruning technique to solve the problem of solution space explosion caused by the Cartesian product of multiple parameters, reducing the computational complexity of selecting optimal parameters across the entire process. Furthermore, it constructs a hierarchical guided decision-making mechanism. The preprocessing decision model takes the target enhancement operator obtained in the enhancement decision stage as input, while the encoding decision model integrates the target enhancement operator with the target preprocessing operator produced in the preprocessing stage to make a joint decision. In other words, the decision result of the previous stage is used as contextual information input to the decision model of the subsequent stage, realizing information transfer and state awareness between stages. This enhances the understanding of upstream processing results by each model, improves the accuracy and synergy of parameter prediction, and integrates the enhancement, preprocessing, and encoding stages into a unified whole. While ensuring video quality, it optimizes encoding efficiency and achieves adaptive parameter optimization across the entire video processing chain.

[0027] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0028] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0029] Figure 1 A schematic diagram of an exemplary system architecture to which the video processing method of the present disclosure can be applied is shown;

[0030] Figure 2 This is a flowchart illustrating a video processing method according to an exemplary embodiment of the present disclosure;

[0031] Figure 3 This is a flowchart illustrating, according to an exemplary embodiment of the present disclosure, a process for making enhancement decisions on a video to be processed based on an enhancement decision model to obtain a target enhancement operator;

[0032] Figure 4 This is a flowchart illustrating the determination of augmented labels corresponding to a first training video according to an exemplary embodiment of this disclosure;

[0033] Figure 5 This is a schematic diagram of the coding performance curves corresponding to the candidate enhancement operators shown in the exemplary embodiments of this disclosure;

[0034] Figure 6 This is a flowchart illustrating a preprocessing decision model based on an exemplary embodiment of the present disclosure, in which a target enhancement operator is used to make a preprocessing decision for the video segments to be processed to obtain the corresponding segment preprocessing operator.

[0035] Figure 7 This is a flowchart illustrating the determination of preprocessing labels corresponding to a second training video segment according to an exemplary embodiment of this disclosure;

[0036] Figure 8 This is a flowchart illustrating, according to an exemplary embodiment of the present disclosure, a preprocessing decision model that performs encoding decisions on video segments to be processed based on a target enhancement operator and a target preprocessing operator to obtain corresponding segment encoding parameters.

[0037] Figure 9 This is a flowchart illustrating the mapping relationship between the constant rate factor and the Lagrange multiplier according to an exemplary embodiment of this disclosure;

[0038] Figure 10 This is a system architecture diagram of a video processing method according to an exemplary embodiment of the present disclosure;

[0039] Figure 11 This is a block diagram of a video processing apparatus according to an exemplary embodiment of the present disclosure;

[0040] Figure 12 This is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0041] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0042] The features, structures, or characteristics described in this disclosure can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more specific details omitted, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0043] The collection, updating, analysis, processing, use, transmission, and storage of user personal information disclosed herein comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken to prevent unauthorized access to user personal information data and to safeguard user personal information security, network security, and national security.

[0044] The accompanying drawings are merely illustrative of this disclosure, and the same reference numerals in the drawings denote the same or similar parts, thus omitting repeated descriptions of them. Some block diagrams shown in the drawings do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in at least one hardware module or integrated circuit, or in different network and / or processor devices and / or microcontroller devices.

[0045] The flowchart shown in the accompanying drawings is merely illustrative and does not necessarily include all content and steps, nor does it require execution in the described order. For example, some steps may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0046] In this specification, the terms “a,” “one,” “the,” “the,” and “at least one” are used to indicate the presence of at least one element / component / etc.; the term “multiple” refers to two or more; the terms “comprising,” “including,” and “having” are used to indicate an open-ended inclusion meaning and that other elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms “first,” “second,” and “third,” etc., are used only as markings and are not a limitation on the number of objects.

[0047] Figure 1 A schematic diagram of an exemplary system architecture to which the video processing method of embodiments of this disclosure can be applied is shown. Figure 1 As shown, the system architecture may include server 101, network 102, and terminal device 103. Network 102 serves as the medium for providing a communication link between terminal device 103 and server 101. Network 102 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0048] In an exemplary embodiment, the terminal device 103 that transmits data with the server 101 may include, but is not limited to, mobile devices such as smartphones, tablets, and laptops, as well as terminal devices with specific functions or forms such as smart speakers, digital assistants, AR (Augmented Reality) devices, VR (Virtual Reality) devices, and smart wearable devices. Alternatively, the terminal device 103 may also be a personal computer, such as a laptop computer or a desktop computer. Optionally, the operating system running on the electronic device may include, but is not limited to, Android, iOS, Linux, and Windows.

[0049] Server 101 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. In some practical applications, server 101 can also be a server for a network platform, such as a trading platform, live streaming platform, social platform, or audio platform, etc., which is not limited in this disclosure. The server can be a single server or a cluster of multiple servers; the specific architecture of the server is not limited in this disclosure.

[0050] In some embodiments of this disclosure, the process by which server 101 performs video processing may include: acquiring a video to be processed; performing enhancement decisions on the video to be processed based on an enhancement decision model to obtain a target enhancement operator corresponding to the video to be processed; performing enhancement processing on the video to be processed using the target enhancement operator to obtain an enhanced video; performing preprocessing decisions on the video to be processed based on a preprocessing decision model according to the target enhancement operator to obtain a target preprocessing operator corresponding to the video to be processed; performing preprocessing on the enhanced video using the target preprocessing operator to obtain a preprocessed video; performing encoding decisions on the preprocessed video based on an encoding decision model according to the target enhancement operator and the target preprocessing operator to obtain target encoding parameters corresponding to the video to be processed; and encoding the preprocessed video using the target encoding parameters to generate a video stream.

[0051] In addition, it should be noted that, Figure 1 The example shown is merely one application environment of the video processing method provided in this disclosure. Figure 1 The number of terminal devices 103, networks 102 and servers 101 in the diagram is merely illustrative. Depending on actual needs, there can be any number of terminal devices, networks and servers.

[0052] Figure 2 This is a flowchart illustrating a video processing method according to an exemplary embodiment of the present disclosure. Figure 2 The subject executing the provided method can be any electronic device, such as... Figure 1 Terminal device 101, for example Figure 1 Terminal device 103, for example Figure 1 The server 101 and terminal device 103 jointly implement the video processing method, but this disclosure is not limited thereto. (See reference...) Figure 2 The video processing method includes the following steps.

[0053] Step S210: Obtain the video to be processed.

[0054] Step S220: Based on the augmentation decision model, perform augmentation decision on the video to be processed to obtain the target augmentation operator corresponding to the video to be processed.

[0055] Step S230: Use the target enhancement operator to enhance the video to be processed, and obtain the enhanced video.

[0056] Step S240: Based on the preprocessing decision model, perform preprocessing decisions on the video to be processed according to the target enhancement operator to obtain the target preprocessing operator corresponding to the video to be processed.

[0057] Step S250: Use the target preprocessing operator to preprocess the enhanced video to obtain the preprocessed video.

[0058] Step S260: Based on the coding decision model, the coding decision is made on the preprocessed video according to the target enhancement operator and the target preprocessing operator to obtain the target coding parameters corresponding to the video to be processed.

[0059] Step S270: Encode the preprocessed video using the target encoding parameters to generate a video stream.

[0060] In this embodiment, the video to be processed is first acquired. This video can originate from user uploads, local storage, network transmission, or real-time acquisition devices, serving as the input basis for the entire processing flow. Then, based on an enhancement decision model, the content of the video to be processed is analyzed to determine whether image enhancement is needed and, if so, which image enhancement method to use, such as super-resolution, deblurring, noise reduction, or face enhancement, thereby determining the most suitable target enhancement operator. After obtaining the target enhancement operator, the image enhancement operation is performed on the video to be processed using this operator, generating an enhanced video with superior visual quality, providing a high-quality source for subsequent processing.

[0061] Based on the preprocessing decision model, and combined with the analysis of the possible characteristic changes introduced after enhancement by the determined target enhancement operator, the most suitable target preprocessing operator, such as sharpening and texturing, is determined. After obtaining the target preprocessing operator, the enhanced video is further optimized using this target preprocessing operator to output the preprocessed video, ensuring that it meets the encoder's input requirements and reduces encoding complexity while maintaining image quality improvement.

[0062] During the encoding stage, based on the encoding decision model, the changes in video content brought about by the target enhancement operator and the target preprocessing operator are comprehensively considered to select the optimal combination of encoding parameters, including quantization parameters, GOP structure, and bitrate control mode, to form the target encoding parameters. Finally, the preprocessed video is encoded using the target encoding parameters to generate a video bitstream with high compression ratio, stable quality, and adaptability to the transmission environment, completing the entire intelligent and collaborative optimization process from raw video to a distributable bitstream.

[0063] It should be noted that after obtaining the target enhancement operator, the decision-making process for the target preprocessing operator and the image enhancement processing using that target enhancement operator are executed asynchronously and in parallel. That is, while the video is being enhanced in step S230, step S240 can simultaneously initiate the preprocessing decision-making process based on the determined target enhancement operator, analyzing the appropriate target preprocessing operator through the preprocessing decision model. Both processes can proceed independently without waiting for the other to complete; they only need to complete their respective tasks before entering the encoding decision stage, thus achieving a pipelined, highly efficient video processing architecture.

[0064] The video processing method provided in this disclosure employs a multi-stage pruning technique to solve the problem of solution space explosion caused by the Cartesian product of multiple parameters, reducing the computational complexity of selecting optimal parameters across the entire process. Furthermore, it constructs a hierarchical guided decision-making mechanism. The preprocessing decision model takes the target enhancement operator obtained in the enhancement decision stage as input, while the encoding decision model integrates the target enhancement operator with the target preprocessing operator produced in the preprocessing stage to make a joint decision. In other words, the decision result of the previous stage is used as contextual information input to the decision model of the subsequent stage, realizing information transfer and state awareness between stages. This enhances the understanding of upstream processing results by each model, improves the accuracy and synergy of parameter prediction, and integrates the enhancement, preprocessing, and encoding stages into a unified whole. While ensuring video quality, it optimizes encoding efficiency and achieves adaptive parameter optimization across the entire video processing chain.

[0065] The video processing method in this example embodiment will now be further explained.

[0066] In step S210, the video to be processed is acquired.

[0067] In this embodiment, the video to be processed can come from user uploads, local storage, network transmission, or real-time acquisition devices, and this video serves as the input basis for the entire processing flow.

[0068] In step S220, based on the augmentation decision model, augmentation decisions are made on the video to be processed to obtain the target augmentation operator corresponding to the video to be processed.

[0069] In this embodiment, the augmentation decision model is a classification model, which is used to analyze and understand the video content features of the video to be processed, and select an augmentation strategy that is suitable for the video to be processed, namely the target augmentation operator, or the optimal augmentation operator.

[0070] Figure 3 This is a flowchart illustrating, according to an exemplary embodiment of the present disclosure, a process for making enhancement decisions on a video to be processed based on an enhancement decision model to obtain a target enhancement operator. (Refer to...) Figure 3 This may include the following steps.

[0071] Step S310: Pre-encode the first video segment to be processed to obtain the first pre-encoded feature.

[0072] Step S320: Input the first pre-encoded features and the basic features of the video to be processed into the augmentation decision model to obtain the target augmentation decision result of the video to be processed.

[0073] Step S330: Determine the target enhancement operator based on the target enhancement decision results and the preset enhancement constraint rules.

[0074] The video to be processed includes at least one video segment. For example, the video to be processed is segmented into consecutive video segments at preset time intervals (e.g., 6s or 8s) to obtain at least one video segment to be processed. Of course, other methods can also be used to segment the video to be processed in this embodiment, such as segmenting according to indicators like inter-frame motion intensity, scene change degree, or visual saliency; this is not limited. The basic features of the video to be processed refer to features used to characterize the complexity of the video, and may include, but are not limited to, intra-frame complexity features, inter-frame complexity features, inter-frame prediction modes, intra-frame prediction modes, the number of I-frames, and the number of B-frames.

[0075] In this embodiment, the enhancement processing is video-level, meaning the same enhancement strategy is applied to the entire video to be processed. The first video segment to be processed is obtained and pre-encoded. For example, an encoder is used to encode the video segment with preset default parameters. Statistical information generated during the encoding process is extracted and used as the first pre-encoded feature corresponding to the first video segment. Then, the first pre-encoded feature and the basic features of the video to be processed are input into the enhancement decision model, and the output is the target enhancement decision result selected from the candidate enhancement operators. The candidate enhancement operators can include, but are not limited to, deep or generative models such as super-resolution models, deblurring models, denoising models, and face models. For example, the probability of using a super-resolution model is 0.6, the probability of using a denoising model is 0.3, and the probability of using a face enhancement model is 0.1.

[0076] In one possible implementation, in addition to inputting the first pre-encoded features and the basic features of the video to be processed into the augmented decision model, the complexity features of the video to be processed can be further introduced as model input to improve the accuracy of the augmented decision. Here, complexity features refer to statistical information extracted from the video to be processed through encoding using a preset encoder (such as H.264) that reflects the complexity of the video encoding.

[0077] After obtaining the target enhancement decision results, the target enhancement operator is determined by combining them with preset enhancement constraint rules. These enhancement constraint rules are pre-set and may include hardware resource limitations, such as disabling high-complexity super-resolution models when GPU memory is insufficient; processing latency requirements, such as prioritizing low-latency enhancement operators in real-time scenarios; content semantic matching, such as reducing the priority of face enhancement models if no face region is detected in the video; energy consumption constraints, such as prioritizing lightweight models on mobile devices; and overlay rules, such as allowing single enhancement or combining multiple operators according to priority. The target enhancement decision results output by the enhancement decision model are analyzed using the preset enhancement constraint rules to ultimately determine the target enhancement operator suitable for the video to be processed, and this target enhancement operator is then used to enhance the image quality of the entire video.

[0078] Through the above steps, the precoding features of the first video segment to be processed are used to reflect the overall complexity. The encoding results are also passed through to the forward link. The video-level enhancement strategy is decided by combining the basic features of the video to be processed, which reduces the computational overhead while ensuring the accuracy of the decision. Furthermore, by introducing enhancement constraint rules to post-process the output of the enhancement decision model, the feasibility and adaptability of the enhancement strategy in application are improved.

[0079] In one exemplary embodiment of this disclosure, the augmented decision model is trained using the following method: obtaining a first training video set; the first training video set includes multiple first training videos; precoding the first training video segment of the first training video to obtain first training encoded features; inputting the first training encoded features and the basic features of the first training video into a first initial classification model to obtain the predicted augmented decision result of the first training video; and training the first initial classification model based on the augmented label corresponding to the first training video and the predicted augmented decision result of the first training video to obtain the augmented decision model.

[0080] In this embodiment, the first initial classification model can be a machine learning model such as XGBoost or deep neural network. Of course, other deep learning networks or tree models, such as fully connected networks or Transformers, can also be used. The model can be flexibly selected according to the actual task requirements and data characteristics. This disclosure does not limit the choice.

[0081] During model training, a first training video set is constructed. This set contains multiple first training videos, covering different content types, resolutions, quality levels, and degradation types, to ensure that the model has good generalization ability. Each first training video has a pre-generated augmentation label, which represents the optimal augmentation strategy determined through offline evaluation, such as categories like "super-resolution," "denoising," or "no augmentation required."

[0082] Figure 4 This is a flowchart illustrating the determination of augmented labels corresponding to a first training video according to an exemplary embodiment of this disclosure. (Refer to...) Figure 4 This may include the following steps.

[0083] Step S410: Obtain at least one candidate enhancement operator.

[0084] Step S420: For each candidate enhancement operator among at least one candidate enhancement operator, the first training video is enhanced using the candidate enhancement operator to obtain the enhancement result corresponding to the candidate enhancement operator.

[0085] Step S430: Based on at least one first constant rate factor, encode the enhancement processing result corresponding to the candidate enhancement operator to generate the coding performance curve corresponding to the candidate enhancement operator; determine the performance gain corresponding to the candidate enhancement operator based on the coding performance curve corresponding to the candidate enhancement operator.

[0086] Step S440: Select the maximum value among the performance gains corresponding to at least one candidate enhancement operator, and use the candidate enhancement operator corresponding to the maximum value as the enhancement label corresponding to the first training video.

[0087] Candidate enhancement operators can include, but are not limited to, deep or generative models such as super-resolution models, deblurring models, denoising models, and face models. For each candidate enhancement operator, the first training video is enhanced using that operator to generate the corresponding enhanced video sequence, i.e., the enhancement result. Each of the first constant rate factors (CRFs) is used as an encoding parameter to encode the enhancement result corresponding to the first training video, recording the bitrate and image quality under each first CRF, thereby plotting the encoding performance curve for each candidate enhancement operator. For example, a CRF value is selected every 2 intervals between 25 and 40, i.e., {25, 27, 29, 31, 33, 35, 37, 39}, and each CRF value is used as an encoding parameter to encode the enhancement result corresponding to the first training video, obtaining the bitrate and image quality under these 8 CRF values.

[0088] Figure 5 This is a schematic diagram of the coding performance curves corresponding to candidate enhancement operators according to an exemplary embodiment of this disclosure. It shows the coding performance curves corresponding to three candidate operators, with the horizontal axis representing bitrate and the vertical axis representing image quality. After obtaining the coding performance curves corresponding to the candidate enhancement operators, the coverage area of ​​these curves on the coordinate axes is calculated as BDBR. The area represents the performance gain corresponding to the candidate enhancement operator; the larger the area, the better the performance. Finally, among all candidate enhancement operators, the operator with the largest area under the encoding performance curve is selected as the enhancement label for the first training video. This label serves as the supervised target for training the enhancement decision model, ensuring that the model learns the optimal enhancement strategy that truly improves image quality and encoding efficiency.

[0089] By introducing multiple candidate enhancement operators and combining them with multi-CRF point coding tests, the effects of different enhancement strategies can be covered. The performance gain of each candidate enhancement operator is evaluated by coding performance curves and area quantization, achieving objective comparability of enhancement effects. Finally, the operator with the largest gain is selected as the training label, which ensures that the enhancement decision model can learn the optimal enhancement strategy that can improve both image quality and coding efficiency, providing a reliable training basis for subsequent enhancement decisions for different video content.

[0090] In this embodiment, during the training of the enhanced decision model, for each first training video, its first training video segment is extracted and pre-encoded under preset encoding parameters consistent with those in the online inference stage. Statistical information during the encoding process is extracted to form the first training encoded features. Simultaneously, basic features of the first training video are extracted, such as intra-frame complexity features, inter-frame complexity features, inter-frame prediction mode, intra-frame prediction mode, number of I-frames, and number of B-frames.

[0091] The first training encoded features and the basic features of the first training video are input into the first initial classification model, which outputs the predicted augmentation decision result. By calculating the classification loss (such as cross-entropy loss) between the predicted augmentation decision result of the first training video and the augmentation label, the model parameters are optimized using the backpropagation algorithm, and finally the trained augmentation decision model is obtained.

[0092] In one possible implementation, to ensure consistency of input features between the training and inference phases, in addition to inputting the first training encoded features and the basic features of the first training video into the first initial classification model, complexity features extracted by encoding the first training video using a preset encoder (such as H.264) are also introduced as model input. Thus, by using the same type of feature input during both training and inference, consistency between the model's learning distribution and the actual application scenario is ensured, improving the generalization ability and prediction accuracy of the enhanced decision-making model.

[0093] The training method for the augmented decision-making model disclosed herein achieves efficient supervised learning of the augmented decision-making model by precoding and analyzing the first training video segment of the first training video and combining the basic features of the first training video with the real augmentation labels. This method can significantly reduce training overhead while ensuring decision accuracy and enable the model to have good generalization ability, making it suitable for intelligent augmentation strategy selection for diverse video content.

[0094] In step S230, the target enhancement operator is used to enhance the video to be processed, and the enhanced video is obtained.

[0095] After obtaining the target enhancement operator, the target enhancement operator is used to perform image quality enhancement operations on the video to be processed, generating an enhanced video with better visual quality, providing a high-quality source for subsequent processing.

[0096] In step S240, based on the preprocessing decision model, a preprocessing decision is made on the video to be processed according to the target enhancement operator to obtain the target preprocessing operator corresponding to the video to be processed.

[0097] In this embodiment, the preprocessing decision model is a classification model, which is used to determine the optimal preprocessing strategy, i.e., the target preprocessing operator, or the optimal preprocessing operator, based on the determined target enhancement operator and the video content features of the video to be processed.

[0098] In the embodiments of this disclosure, the video to be processed includes at least one video segment to be processed, and the target preprocessing operator includes segment preprocessing operators corresponding to each video segment to be processed. Figure 6 This is a flowchart illustrating a preprocessing decision model based on an exemplary embodiment of this disclosure, in which preprocessing decisions are made on video segments to be processed according to a target enhancement operator to obtain the corresponding segment preprocessing operator. (Refer to...) Figure 6 This may include the following steps.

[0099] Step S610: For each video segment to be processed in the video to be processed, pre-encode the video segment to be processed to obtain the second pre-encoded feature.

[0100] Step S620: Input the second precoding feature, the target enhancement operator, and the basic features of the video to be processed into the preprocessing decision model to obtain the segment preprocessing operator corresponding to the segment of the video to be processed.

[0101] In this embodiment, the preprocessing granularity is at the slice level, meaning a preprocessing strategy is selected for each video slice to be processed. For each video slice, pre-coding is performed, such as using an encoder to encode the video slice under preset default parameters. Statistical information generated during the encoding process is extracted as a second pre-encoded feature. Then, the second pre-encoded feature, the target enhancement operator, and the basic features of the video to be processed are input into the preprocessing decision model, which outputs the probability distribution of candidate preprocessing operators. The optimal preprocessing operator is then selected based on this probability distribution and used as the slice preprocessing operator corresponding to that video slice. The candidate preprocessing operators are combinations of image processing algorithms and deep learning operators, including but not limited to sharpening filters, texture operators, and bilateral filtering based on image processing algorithms, as well as learnable sharpening networks and neural texture generation modules based on deep learning.

[0102] In one possible implementation, in addition to inputting the second precoding features, the target enhancement operator, and the basic features of the video to be processed into the preprocessing decision model, the complexity features of the video to be processed can be further introduced as model input to improve the accuracy of the enhancement decision. Here, complexity features refer to statistical information extracted from the video to be processed through encoding using a pre-defined encoder (such as H.264) that reflects the complexity of the video encoding.

[0103] Through the above steps, precoding is performed on each video segment to be processed to extract the second precoding feature. Combined with the target enhancement operator and the basic video features, adaptive preprocessing decision is achieved for each video segment to be processed, which fully considers the local changes in content and coding characteristics while ensuring processing accuracy.

[0104] In one exemplary embodiment of this disclosure, the preprocessing decision model is trained using the following method: obtaining a second training video set; the second training video set includes multiple second training videos, and each second training video includes multiple second training video segments; precoding the second training video segments to obtain second training encoded features; inputting the second training encoded features, the basic features of the second training videos, and the enhanced labels corresponding to the second training video segments into a second initial classification model to obtain the prediction preprocessing decision results for the second training video segments; and training the second initial classification model based on the preprocessing labels corresponding to the second training video segments and the prediction preprocessing decision results for the second training video segments to obtain the preprocessing decision model.

[0105] In this embodiment, the second initial classification model can be a machine learning model such as XGBoost or deep neural network. Of course, other deep learning networks or tree models, such as fully connected networks or Transformers, can also be used. The model can be flexibly selected according to the actual task requirements and data characteristics. This disclosure does not limit the choice.

[0106] During model training, a second training video set is first constructed. This set contains multiple second training videos, covering different content types, resolutions, quality levels, and degradation types, to ensure that the model has good generalization ability. Each second training video includes multiple second training video segments, and each second training video segment has a pre-generated preprocessing label. This label represents the optimal preprocessing strategy determined through offline evaluation, such as categories like "sharpening filter" and "texture operator".

[0107] Figure 7 This is a flowchart illustrating the determination of preprocessing labels corresponding to a second training video segment, according to an exemplary embodiment of this disclosure. (Refer to...) Figure 7 This may include the following steps.

[0108] Step S710: Obtain at least one candidate preprocessing operator.

[0109] Step S720: For each candidate preprocessing operator in at least one candidate preprocessing operator, preprocess the second training video segment using the candidate preprocessing operator to obtain the preprocessing result corresponding to the candidate preprocessing operator.

[0110] Step S730: Based on at least one second constant rate factor, encode the preprocessing result corresponding to the candidate preprocessing operator to generate the coding performance curve corresponding to the candidate preprocessing operator; determine the performance gain corresponding to the candidate preprocessing operator based on the coding performance curve corresponding to the candidate preprocessing operator.

[0111] Step S740: Select the maximum value among the performance gains corresponding to at least one candidate preprocessing operator, and use the candidate enhancement operator corresponding to the maximum value as the preprocessing label corresponding to the second training video segment.

[0112] The candidate preprocessing operators are combinations of image processing algorithms and deep learning operators, including but not limited to sharpening filters, texture operators, and bilateral filtering based on image processing algorithms, as well as learnable sharpening networks and neural texture generation modules based on deep learning. For each candidate preprocessing operator, the second training video segment is preprocessed using that operator to generate the corresponding preprocessing result. Each of the at least one second CRF is used as an encoding parameter to encode the preprocessing result corresponding to the second training video segment, and the bitrate and image quality under each second CRF are recorded, thereby plotting the encoding performance curve corresponding to each candidate preprocessing operator. For example, a CRF value is selected every 2 intervals between 25 and 40, i.e., {25, 27, 29, 31, 33, 35, 37, 39}, and each CRF value is used as an encoding parameter to encode the preprocessing result corresponding to the second training video segment, obtaining the bitrate and image quality under these 8 CRF values.

[0113] In this model, the x-axis of the coding performance curve corresponding to the candidate preprocessing operators represents bitrate, and the y-axis represents image quality. After obtaining the coding performance curve corresponding to the candidate preprocessing operators, the area covered by this curve under the coordinate axis is calculated as the BDBR area. This area represents the performance gain corresponding to the candidate preprocessing operator; the larger the area, the better the performance. Finally, among all candidate preprocessing operators, the operator with the largest area under its coding performance curve is selected as the preprocessing label for the second training video segment. This label serves as the supervised target for training the preprocessing decision model, ensuring that the model learns the optimal preprocessing strategy that truly improves image quality and coding efficiency.

[0114] By introducing multiple candidate preprocessing operators and combining them with multi-CRF point coding tests, the effects of different preprocessing strategies can be covered. The performance gain of each candidate preprocessing operator is evaluated by coding performance curves and area quantization, achieving objective comparability of preprocessing effects. Finally, the operator with the largest gain is selected as the training label, which ensures that the preprocessing decision model can learn the optimal enhancement strategy that can improve both image quality and coding efficiency, providing a reliable training basis for subsequent enhancement decisions for different video content.

[0115] In this embodiment, during the preprocessing decision model training process, each second training video segment is pre-encoded under preset encoding parameters consistent with the online inference stage. Statistical information from the encoding process is extracted to form second training encoding features. Simultaneously, basic features of the second training video to which the second training video segment belongs are extracted, such as intra-frame complexity features, inter-frame complexity features, inter-frame prediction mode, intra-frame prediction mode, number of I-frames, and number of B-frames. Furthermore, the enhancement tag corresponding to the second training video segment is obtained. Since the granularity of enhancement processing is at the video level, all segments within the same video share the same enhancement tag; that is, the enhancement tag corresponding to the second training video to which the second training video segment belongs is obtained.

[0116] The enhanced label corresponding to the second training video segment, the second training encoded features, and the basic features of the second training video to which the second training video segment belongs are input into the second initial classification model, and the prediction preprocessing decision result is output. By calculating the classification loss (such as cross-entropy loss) between the prediction preprocessing decision result of the second training video segment and the preprocessing label, the model parameters are optimized using the backpropagation algorithm, and finally the trained preprocessing decision model is obtained.

[0117] In one possible implementation, to ensure consistency of input features between the training and inference phases, in addition to inputting the enhanced labels corresponding to the second training video segments, the second training encoded features, and the basic features of the second training video to which the second training video segments belong into the second initial classification model, complexity features extracted by encoding the second training video using a preset encoder (such as H.264) are also introduced as model input. Thus, by using the same type of feature input during training and inference, consistency between the model's learning distribution and the actual application scenario is ensured, improving the generalization ability and prediction accuracy of the preprocessing decision model.

[0118] The training method of the preprocessing decision model in this embodiment introduces segment-level precoding features, basic video features, and corresponding enhancement labels from a second training video set to construct fine-grained supervision signals, enabling the preprocessing decision model to learn the synergistic relationship between enhancement processing and preprocessing strategies. By combining training with segment-level labeled preprocessing labels, the model's adaptability to local content changes is improved, achieving accurate preprocessing decisions based on context awareness.

[0119] In step S250, the enhanced video is preprocessed using the target preprocessing operator to obtain the preprocessed video.

[0120] The target preprocessing operators include segment preprocessing operators corresponding to each video segment in the video to be processed. For each video segment of the enhanced video, its corresponding segment preprocessing operator is applied to generate corresponding preprocessed segments, which can then be merged into the preprocessed video.

[0121] In step S260, based on the coding decision model, coding decisions are made on the preprocessed video according to the target enhancement operator and the target preprocessing operator to obtain the target coding parameters corresponding to the video to be processed.

[0122] In this embodiment, the coding decision model is a regression model, which is used to determine the target coding parameters that match the determined target enhancement operator, target preprocessing operator, and video content features of the video to be processed.

[0123] In the embodiments of this disclosure, the video to be processed includes at least one video segment to be processed, and the target encoding parameters include the segment encoding parameters corresponding to each video segment to be processed. Figure 8 This is a flowchart illustrating an exemplary embodiment of the present disclosure, showing how a preprocessing decision model is used to make encoding decisions for video segments to be processed based on a target enhancement operator and a target preprocessing operator to obtain corresponding segment encoding parameters. (Refer to...) Figure 8 This may include the following steps.

[0124] Step S810: For each video segment to be processed in the video to be processed, obtain the pre-processed video segment corresponding to the video segment to be processed; the pre-processed video segment is the video data obtained after the video segment to be processed is processed by the target enhancement operator and the segment pre-processing operator corresponding to the video segment to be processed.

[0125] Step S820: Pre-encode the pre-processed video segments to obtain the third pre-encoding feature.

[0126] Step S830: Input the third precoding feature, the target enhancement operator, the segmentation preprocessing operator corresponding to the video segment to be processed, and the basic features of the video to be processed into the coding decision model to obtain the initial coding parameters corresponding to the video segment to be processed.

[0127] Step S840: Based on the mapping relationship between the constant rate factor and the Lagrange multiplier, optimize the initial coding parameters corresponding to the video segment to be processed to obtain the segment coding parameters corresponding to the video segment to be processed.

[0128] For each video segment to be processed, a corresponding pre-processed video segment is obtained. This involves sequentially enhancing and pre-processing the video segment using the target enhancement operator and the corresponding segment pre-processing operator. Then, the pre-processed video segment is pre-encoded, for example, using a codec encoder with preset default parameters. Statistical information generated during encoding is extracted as a third pre-encoding feature. Finally, the third pre-encoding feature, the target enhancement operator, the corresponding segment pre-processing operator, and the basic features of the video to be processed are input into the encoding decision model. By combining the context information from the upstream processing chain with the current content features, initial encoding parameters suitable for the video segment to be processed are predicted. These parameters may include, but are not limited to, CRF, quantization parameters (QP), encoding tools, frame structure size, and frame rate.

[0129] After outputting the initial coding parameters through the coding decision model, the obtained initial coding parameters are optimized and adjusted based on the mapping relationship between the constant rate factor (CRF) and the Lagrange multiplier (λ). This mapping relationship is used to balance the bit rate and distortion terms in the rate-distortion cost. By dynamically adjusting the value of λ, the encoder can more accurately control the bit allocation while maintaining the target visual quality. Finally, the optimized segment coding parameters are output to guide the coding process of the video segment to be processed, so as to achieve high-quality and high-efficiency adaptive coding.

[0130] Figure 9 This is a flowchart illustrating the mapping relationship between the constant rate factor and Lagrange multipliers according to an exemplary embodiment of this disclosure. (Refer to...) Figure 9 This may include the following steps.

[0131] Step S910: Obtain the offline video set; the offline video set includes multiple offline videos, and each offline video includes multiple offline video segments.

[0132] In the process of generating the mapping relationship, an offline video set is first obtained. This offline video set includes multiple offline videos, covering different content types, resolutions, quality levels and degradation types. Each offline video is divided into multiple offline video segments.

[0133] Step S920: For each of the at least one third constant rate factors, the offline video segment is encoded based on the third constant rate factor to obtain the encoding result of the offline video segment under the third constant rate factor.

[0134] For each offline video segment, each of the at least one third CRF is used as an encoding parameter to encode the offline video segment, and the bitrate and image quality under each third CRF are recorded. For example, 16 third CRF values ​​are preset, and the offline video segment is encoded using a standard video encoder. During the encoding process, other encoding parameters are fixed, and only the CRF is used as a control variable to obtain the bitrate and image quality of the offline video segment under different CRFs.

[0135] Step S930: Based on the encoding results of all offline video segments in multiple offline videos under the third constant rate factor, determine the optimal Lagrange multiplier corresponding to the third constant rate factor.

[0136] For each third CRF, the bitrate and image quality of all offline video segments under that CRF are statistically analyzed to construct a rate-distortion performance metric. Then, a Lagrange optimization algorithm is used to search for the optimal Lagrange multiplier λ that minimizes the overall rate-distortion cost. For example, this is achieved by minimizing the objective function J = D + λ·R, where D is the distortion (estimated based on image quality; higher image quality results in higher distortion), and R is the bitrate, to find the λ value that minimizes J. Finally, a weighted average or quantile aggregation is performed on the results of all offline video segments to determine the optimal λ corresponding to that third CRF.

[0137] Step S940: Generate a mapping relationship between constant rate factors and Lagrange multipliers based on each of the at least one third constant rate factors and its corresponding optimal Lagrange multiplier.

[0138] After obtaining the optimal λ corresponding to each third CRF, each third CRF value and its corresponding optimal λ are paired to form a (CRF,λ) mapping relationship.

[0139] By using the above steps and an offline video set covering multiple content types, statistical coding results are obtained under at least one third CRF, and the optimal λ is searched using a rate-distortion optimization algorithm. This allows for the offline construction of a mapping relationship between the constant rate factor (CRF) and the Lagrange multiplier (λ). This mapping relationship can accurately reflect the optimal λ under different CRF settings, providing a verifiable optimization basis for the online phase.

[0140] In one exemplary embodiment of this disclosure, an initial regression model is obtained and trained to obtain an encoded decision model. The initial regression model can be a machine learning model such as XGBoost or a deep neural network. Alternatively, other deep learning networks or tree models, such as fully connected networks or Transformers, can also be used. The choice can be flexible and depends on the specific task requirements and data characteristics; this disclosure does not impose any limitations on this.

[0141] For example, the encoding decision model is trained as follows: A third training video set is obtained, comprising multiple third training videos, each comprising multiple third training video segments; for each third training video segment, enhancement processing is performed using the enhanced labels of the corresponding third training video; then, preprocessing is performed using the preprocessing labels corresponding to that third training video segment to obtain processed segments; finally, pre-coding is performed on the processed segments using preset encoding parameters consistent with those used in the online inference stage to obtain third training encoding features. Simultaneously, basic features of the third training video to which the segment belongs are extracted, such as intra-frame complexity features, inter-frame complexity features, inter-frame prediction mode, intra-frame prediction mode, number of I-frames, and number of B-frames.

[0142] Then, the third training video segment's third training coding features and preprocessed labels, along with the enhanced labels and basic features of the third training video to which the third training video segment belongs, are input to the initial regression model. The model outputs the predicted coding parameters corresponding to the third training video segment. Based on the loss between the predicted coding parameters and the true coding parameters (such as cross-entropy loss or mean squared error), the initial regression model is trained and its parameters are optimized through supervised learning, ultimately yielding the trained coding decision model. The true coding parameters of the third training video segment can be obtained by performing coding tests on multiple candidate coding parameter combinations and selecting the parameter combination with the optimal rate-distortion performance under the target bitrate or quality constraints as the true coding parameters. Of course, other methods can also be used to determine the true coding parameters, and this disclosure does not limit this approach.

[0143] The training method of the coding decision model in this disclosure introduces segment-level precoding features and corresponding preprocessing labels from a third training video set, as well as video basic features and corresponding enhancement labels, to construct fine-grained supervision signals. This enables the coding decision model to learn the synergistic relationship between enhancement processing, preprocessing, and coding decisions. By combining video-level labeled enhancement labels and segment-level labeled preprocessing labels for training, the model's adaptability to local content changes is improved, achieving context-aware and accurate coding decisions.

[0144] In step S270, the preprocessed video is encoded using the target encoding parameters to generate a video bitstream.

[0145] The target encoding parameters include the segment encoding parameters corresponding to each video segment in the video to be processed. Each video segment of the preprocessed video is processed using its corresponding segment encoding parameters to generate a corresponding encoded bitstream. The encoded bitstreams of all the video segments to be processed are then concatenated in chronological order to form a complete video bitstream.

[0146] Figure 10 This is a system architecture diagram illustrating a video processing method according to an exemplary embodiment of this disclosure. Figure 10 As shown, after a user uploads a video, the system obtains the video's basic features and segmentation features, and makes segmentation decisions to divide the video into multiple video segments. Then, the GPU and CPU apply enhancement and preprocessing strategies to process the video. During this process, an enhancement strategy (i.e., the target enhancement operator) is selected from more than 10 candidate enhancement operators based on an enhancement decision model, and a corresponding preprocessing strategy (i.e., the target preprocessing operator, including segmentation preprocessing operators corresponding to each video segment) is obtained based on a preprocessing decision model. The preprocessing strategies include preprocessing techniques related to sharpening and texture, such as base-weight, non-roi-weight, and scale-threshold, for video texture enhancement or region adaptive processing. The parameters or techniques are as follows: After enhancing and preprocessing the video, enhanced and preprocessed video segments are obtained, and they are encoded to generate corresponding encoded bitstreams. The encoded bitstreams of all video segments to be processed are concatenated in chronological order to form a complete video bitstream. In this process, the corresponding initial encoding parameters are obtained based on the encoding decision model, and the optimal λ is obtained based on the mapping relationship between the constant rate factor (CRF) and the Lagrange multiplier (λ). The initial encoding parameters are optimized using the optimal λ, and finally, the optimized segment encoding parameters are output. The enhanced and preprocessed video segments are then encoded using these segment encoding parameters. The encoding parameters may include QP, encoding tool, frame structure size, and frame rate, such as QP being specific parameters like crf, max-qp, saliencyQP, qpScaleFactor, etc., encoding tool being cutrele strength, frame structure size being gop-size, and frame rate being fps.

[0147] The video enhancement method provided in this disclosure employs multi-stage pruning technology to solve the problem of solution space explosion caused by the Cartesian product of multiple parameters, reducing the computational complexity of selecting optimal parameters across the entire process. Furthermore, it constructs a hierarchical guided decision-making mechanism. The preprocessing decision model takes the target enhancement operator obtained in the enhancement decision stage as input, while the encoding decision model integrates the target enhancement operator with the target preprocessing operator produced in the preprocessing stage to make a joint decision. In other words, the decision result of the previous stage is used as contextual information input to the decision model of the subsequent stage, realizing information transfer and state awareness between stages. This enhances the understanding of upstream processing results by each model, improves the accuracy and synergy of parameter prediction, and integrates the enhancement, preprocessing, and encoding stages into a whole. While ensuring video quality, it optimizes encoding efficiency and achieves adaptive parameter optimization across the entire video processing chain.

[0148] It is understood that the same / similar parts between the various embodiments of the above methods in this specification can be referred to each other. Each embodiment focuses on the differences from other embodiments, and relevant parts can be referred to the descriptions of other method embodiments.

[0149] Figure 11 This is a block diagram illustrating a video processing apparatus according to an exemplary embodiment of the present disclosure. Figure 11 As shown, the device 1100 includes a video acquisition module 1101, an enhancement decision module 1102, an enhancement processing module 1103, a preprocessing decision module 1104, a preprocessing module 1105, an encoding decision module 1106, and an encoding module 1107.

[0150] The video acquisition module 1101 is configured to acquire the video to be processed. The enhancement decision module 1102 is configured to perform enhancement decisions on the video to be processed based on the enhancement decision model, obtaining the target enhancement operator corresponding to the video. The enhancement processing model 1103 is configured to perform enhancement processing on the video to be processed using the target enhancement operator, obtaining the enhanced video. The preprocessing decision module 1104 is configured to perform preprocessing decisions on the video to be processed based on the preprocessing decision model and the target enhancement operator, obtaining the target preprocessing operator corresponding to the video. The preprocessing module 1105 is configured to perform preprocessing on the enhanced video using the target preprocessing operator, obtaining the preprocessed video. The encoding decision module 1106 is configured to perform encoding decisions on the preprocessed video based on the encoding decision model, the target enhancement operator, and the target preprocessing operator, obtaining the target encoding parameters corresponding to the video. The encoding module 1107 is configured to encode the preprocessed video using the target encoding parameters, generating a video stream.

[0151] In one exemplary embodiment of this disclosure, the enhancement decision module 1102 is further configured to: pre-encode the first video segment to be processed to obtain a first pre-encoded feature; input the first pre-encoded feature and the basic features of the video to be processed into the enhancement decision model to obtain the target enhancement decision result of the video to be processed; and determine the target enhancement operator according to the target enhancement decision result and the preset enhancement constraint rules.

[0152] In one exemplary embodiment of this disclosure, the apparatus 1100 further includes an enhanced decision model training module 1108, configured to train an enhanced decision model according to the following method: acquiring a first training video set; the first training video set includes multiple first training videos; pre-coding the first training video segment of the first training video to obtain first training encoded features; inputting the first training encoded features and the basic features of the first training video into a first initial classification model to obtain the predicted enhanced decision result of the first training video; and training the first initial classification model according to the enhanced label corresponding to the first training video and the predicted enhanced decision result of the first training video to obtain the enhanced decision model.

[0153] In one exemplary embodiment of this disclosure, the augmentation decision model training module 1108 is further configured to: acquire at least one candidate augmentation operator; for each of the at least one candidate augmentation operator, perform augmentation processing on the first training video using the candidate augmentation operator to obtain the augmentation processing result corresponding to the candidate augmentation operator; encode the augmentation processing result corresponding to the candidate augmentation operator based on at least one first constant rate factor to generate an encoding performance curve corresponding to the candidate augmentation operator; determine the performance gain corresponding to the candidate augmentation operator based on the encoding performance curve corresponding to the candidate augmentation operator; select the maximum value among the performance gains corresponding to the at least one candidate augmentation operator, and use the candidate augmentation operator corresponding to the maximum value as the augmentation label corresponding to the first training video.

[0154] In one exemplary embodiment of this disclosure, the target preprocessing operator includes segment preprocessing operators corresponding to each video segment to be processed in the video to be processed. The preprocessing decision module 1104 is further configured to: for each video segment to be processed in the video to be processed, pre-encode the video segment to be processed to obtain a second pre-encoded feature; input the second pre-encoded feature, the target enhancement operator, and the basic features of the video to be processed into the preprocessing decision model to obtain the segment preprocessing operator corresponding to the video segment to be processed.

[0155] In one exemplary embodiment of this disclosure, the apparatus 1100 further includes a preprocessing decision model training module 1109, configured to train a preprocessing decision model according to the following method: acquiring a second training video set; the second training video set includes multiple second training videos, and each second training video includes multiple second training video segments; precoding the second training video segments to obtain second training encoded features; inputting the second training encoded features, the basic features of the second training videos, and the enhanced labels corresponding to the second training video segments into a second initial classification model to obtain the prediction preprocessing decision results of the second training video segments; and training the second initial classification model according to the preprocessing labels corresponding to the second training video segments and the prediction preprocessing decision results of the second training video segments to obtain the preprocessing decision model.

[0156] In one exemplary embodiment of this disclosure, the preprocessing decision model training module 1109 is further configured to: acquire at least one candidate preprocessing operator; for each candidate preprocessing operator, preprocess the second training video segment using the candidate preprocessing operator to obtain the preprocessing result corresponding to the candidate preprocessing operator; encode the preprocessing result corresponding to the candidate preprocessing operator based on at least one second constant rate factor to generate the encoding performance curve corresponding to the candidate preprocessing operator; determine the performance gain corresponding to the candidate preprocessing operator based on the encoding performance curve corresponding to the candidate preprocessing operator; select the maximum value among the performance gains corresponding to at least one candidate preprocessing operator, and use the candidate enhancement operator corresponding to the maximum value as the preprocessing label corresponding to the second training video segment.

[0157] In one exemplary embodiment of this disclosure, the target encoding parameters include the segment encoding parameters corresponding to each video segment to be processed in the video to be processed. The encoding decision module 1106 is further configured to: for each video segment to be processed in the video to be processed, obtain the pre-processed video segment corresponding to the video segment to be processed; the pre-processed video segment is the video data obtained after processing the video segment to be processed by the target enhancement operator and the segment pre-processing operator corresponding to the video segment to be processed; pre-encode the pre-processed video segment to obtain a third pre-encoding feature; input the third pre-encoding feature, the target enhancement operator, the segment pre-processing operator corresponding to the video segment to be processed, and the basic features of the video to be processed into the encoding decision model to obtain the initial encoding parameters corresponding to the video segment to be processed; and optimize the initial encoding parameters corresponding to the video segment to be processed based on the mapping relationship between the constant rate factor and the Lagrange multiplier to obtain the segment encoding parameters corresponding to the video segment to be processed.

[0158] In one exemplary embodiment of this disclosure, the apparatus 1100 further includes a mapping relationship generation module 1110, configured to generate a mapping relationship between a constant rate factor and a Lagrange multiplier according to the following method: acquiring an offline video set; the offline video set includes multiple offline videos, and each offline video includes multiple offline video segments; for each of at least one third constant rate factor, encoding the offline video segments based on the third constant rate factor to obtain the encoding result of the offline video segments under the third constant rate factor; determining the optimal Lagrange multiplier corresponding to the third constant rate factor based on the encoding results of all offline video segments in the multiple offline videos under the third constant rate factor; and generating a mapping relationship between the constant rate factor and the Lagrange multiplier based on each of the at least one third constant rate factor and the corresponding optimal Lagrange multiplier.

[0159] Regarding the apparatus in the above embodiments, the specific manner in which each unit performs its operation has been described in detail in the embodiments of the method, and will not be elaborated upon here.

[0160] Figure 12 This is a schematic diagram illustrating the structure of an electronic device according to an exemplary embodiment of this disclosure. It should be noted that... Figure 12 The electronic device 1200 shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0161] like Figure 12 As shown, the electronic device 1200 is manifested in the form of a general-purpose computing device. The components of the electronic device 1200 may include, but are not limited to: at least one processing unit 1210, at least one storage unit 1220, and a bus 1230 connecting different system components (including storage unit 1220 and processing unit 1210).

[0162] The storage unit stores program code that can be executed by the processing unit 1210, causing the processing unit 1210 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 1210 can perform, as follows: Figure 2 The steps are shown in the figure.

[0163] Storage unit 1220 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 12201 and / or cache memory 12202, and may further include a read-only memory (ROM) 12203.

[0164] Storage unit 1220 may also include a program / utility 12204 having a set (at least one) of program modules 12205, such program modules 12205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0165] Bus 1230 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0166] Electronic device 1200 can also communicate with one or more external devices 1300 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 1200, and / or with any device that enables electronic device 1200 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1250. Furthermore, electronic device 1200 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1240. As shown, network adapter 1240 communicates with other modules of electronic device 1200 via bus 1230. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0167] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of the invention may also be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of the invention described in the "Exemplary Methods" section of this specification.

[0168] According to embodiments of the present invention, a program product for implementing the above-described method may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0169] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0170] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0171] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0172] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0173] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0174] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0175] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0176] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

[0177] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A video processing method, characterized in that, The method includes: Get the video to be processed; Based on the augmentation decision model, an augmentation decision is made on the video to be processed to obtain the target augmentation operator corresponding to the video to be processed; the target augmentation operator is used to augment the video to be processed to obtain the augmented video; Based on the preprocessing decision model, the video to be processed is preprocessed according to the target enhancement operator to obtain the target preprocessing operator corresponding to the video to be processed; the enhanced video is preprocessed using the target preprocessing operator to obtain the preprocessed video. Based on the coding decision model, coding decisions are made on the preprocessed video according to the target enhancement operator and the target preprocessing operator to obtain the target coding parameters corresponding to the video to be processed; the preprocessed video is encoded using the target coding parameters to generate a video stream.

2. The method according to claim 1, characterized in that, The step of performing enhancement decisions on the video to be processed based on the enhancement decision model to obtain the target enhancement operator corresponding to the video to be processed includes: The first video segment to be processed is pre-encoded to obtain a first pre-encoded feature; The first pre-encoded feature and the basic features of the video to be processed are input into the augmentation decision model to obtain the target augmentation decision result of the video to be processed. The target enhancement operator is determined based on the target enhancement decision result and the preset enhancement constraint rules.

3. The method according to claim 1 or 2, characterized in that, The enhanced decision model was trained using the following method: Obtain the first training video set; the first training video set includes multiple first training videos; The first training video segment of the first training video is pre-encoded to obtain the first training encoding feature; The first training encoded features and the basic features of the first training video are input into the first initial classification model to obtain the prediction enhancement decision result of the first training video; The first initial classification model is trained based on the augmented labels corresponding to the first training video and the predicted augmentation decision results of the first training video to obtain the augmentation decision model.

4. The method according to claim 3, characterized in that, The method further includes: Obtain at least one candidate enhancement operator; For each of the at least one candidate enhancement operators, the first training video is enhanced using the candidate enhancement operator to obtain the enhancement result corresponding to the candidate enhancement operator; the enhancement result corresponding to the candidate enhancement operator is encoded based on at least one first constant rate factor to generate the encoding performance curve corresponding to the candidate enhancement operator; the performance gain corresponding to the candidate enhancement operator is determined based on the encoding performance curve corresponding to the candidate enhancement operator. The maximum value among the performance gains corresponding to the at least one candidate enhancement operator is selected, and the candidate enhancement operator corresponding to the maximum value is used as the enhancement label corresponding to the first training video.

5. The method according to claim 1, characterized in that, The target preprocessing operator includes the segment preprocessing operator corresponding to each video segment to be processed in the video to be processed; The step of performing preprocessing decisions on the video to be processed based on the preprocessing decision model and the target enhancement operator to obtain the target preprocessing operator corresponding to the video to be processed includes: For each video segment to be processed in the video to be processed, the video segment to be processed is pre-encoded to obtain a second pre-encoded feature; The second precoding feature, the target enhancement operator, and the basic features of the video to be processed are input into the preprocessing decision model to obtain the segment preprocessing operator corresponding to the video segment to be processed.

6. The method according to claim 1 or 5, characterized in that, The preprocessing decision model was trained using the following method: Obtain a second training video set; the second training video set includes multiple second training videos, and each second training video includes multiple second training video segments. The second training video segment is pre-encoded to obtain the second training encoding features; The second training encoded features, the basic features of the second training video, and the enhanced labels corresponding to the second training video segments are input into the second initial classification model to obtain the prediction preprocessing decision results of the second training video segments. Based on the preprocessing labels corresponding to the second training video segment and the prediction preprocessing decision results of the second training video segment, the second initial classification model is trained to obtain the preprocessing decision model.

7. The method according to claim 6, characterized in that, The method further includes: Obtain at least one candidate preprocessing operator; For each of the at least one candidate preprocessing operators, the second training video segment is preprocessed using the candidate preprocessing operator to obtain the preprocessing result corresponding to the candidate preprocessing operator; the preprocessing result corresponding to the candidate preprocessing operator is encoded based on at least one second constant rate factor to generate the encoding performance curve corresponding to the candidate preprocessing operator; the performance gain corresponding to the candidate preprocessing operator is determined based on the encoding performance curve corresponding to the candidate preprocessing operator. The maximum value among the performance gains corresponding to the at least one candidate preprocessing operator is selected, and the candidate enhancement operator corresponding to the maximum value is used as the preprocessing label corresponding to the second training video segment.

8. The method according to claim 1, characterized in that, The target encoding parameters include the segment encoding parameters corresponding to each video segment to be processed in the video to be processed. The step of using a coding decision model to perform coding decisions on the preprocessed video based on the target enhancement operator and the target preprocessing operator to obtain the target coding parameters corresponding to the video to be processed includes: For each video segment to be processed in the video to be processed, a pre-processed video segment corresponding to the video segment to be processed is obtained; the pre-processed video segment is the video data obtained after processing the video segment to be processed by the target enhancement operator and the segment pre-processing operator corresponding to the video segment to be processed. The preprocessed video segments are pre-encoded to obtain a third pre-encoding feature; The third precoding feature, the target enhancement operator, the segment preprocessing operator corresponding to the video segment to be processed, and the basic features of the video to be processed are input into the coding decision model to obtain the initial coding parameters corresponding to the video segment to be processed. Based on the mapping relationship between the constant rate factor and the Lagrange multiplier, the initial coding parameters corresponding to the video segment to be processed are optimized to obtain the segment coding parameters corresponding to the video segment to be processed.

9. The method according to claim 8, characterized in that, The mapping relationship between the constant rate factor and the Lagrange multipliers is generated according to the following method: Obtain an offline video set; the offline video set includes multiple offline videos, and the offline videos include multiple offline video segments; For each of the at least one third constant rate factor, the offline video segment is encoded based on the third constant rate factor to obtain the encoding result of the offline video segment under the third constant rate factor; Based on the encoding results of all offline video segments in the plurality of offline videos under the third constant rate factor, determine the optimal Lagrange multiplier corresponding to the third constant rate factor; Based on each of the at least one third constant rate factor and its corresponding optimal Lagrange multiplier, a mapping relationship between the constant rate factor and the Lagrange multiplier is generated.

10. A video processing apparatus, characterized in that, The device includes: The video acquisition module is configured to acquire the video to be processed; The enhancement decision module is configured to perform enhancement decisions on the video to be processed based on the enhancement decision model, and obtain the target enhancement operator corresponding to the video to be processed. An enhancement processing model is configured to enhance the video to be processed using the target enhancement operator to obtain an enhanced video. The preprocessing decision module is configured to perform preprocessing decisions on the video to be processed based on the preprocessing decision model and according to the target enhancement operator, so as to obtain the target preprocessing operator corresponding to the video to be processed. The preprocessing module is configured to preprocess the enhanced video using the target preprocessing operator to obtain the preprocessed video; The encoding decision module is configured to make encoding decisions on the preprocessed video based on the encoding decision model, according to the target enhancement operator and the target preprocessing operator, to obtain the target encoding parameters corresponding to the video to be processed. The encoding module is configured to encode the preprocessed video using the target encoding parameters to generate a video stream.

11. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the executable instructions to implement the video processing method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, wherein instructions in the computer-readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the video processing method as described in any one of claims 1 to 9.

13. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the video processing method as described in any one of claims 1 to 9.

Images