Video transmission method, electronic device, storage medium and computer program product

Abstract

The application provides a video transmission method, an electronic device, a storage medium and a computer program product, wherein the method comprises: generating one or more candidate output parameter groups of a first video stream based on video frames in the first video stream, and network state parameters and terminal state parameters; the candidate output parameter groups comprise a code rate, a frame rate and a quantization parameter; evaluating the one or more candidate output parameter groups to obtain a score of each candidate output parameter group in the one or more candidate output parameter groups; determining a video output parameter group from the one or more candidate output parameter groups based on the score of each candidate output parameter group; and outputting the first video stream based on the video output parameter group.

Description

Technical Field

[0001] This application relates to video transmission technology, and more particularly to a video transmission method, electronic device, storage medium, and computer program product. Background Technology

[0002] With the increasing popularity of high-bandwidth services such as Ultra High Definition (UHD) (e.g., 4K), Super Hi-Vision (SHV) (e.g., 8K), Virtual Reality (VR), and Augmented Reality (AR), users have higher and higher requirements for video output quality. Traditional video output methods are difficult to maintain high-quality video output in dynamically changing network environments. Summary of the Invention

[0003] To address the related technical issues, embodiments of this application provide a video transmission method, an electronic device, a storage medium, and a computer program product.

[0004] The technical solution of this application embodiment is implemented as follows: This application provides a video transmission method, the method comprising: Based on the video frames in the first video stream, as well as network status parameters and terminal status parameters, one or more candidate output parameter groups for the first video stream are generated; the candidate output parameter groups include bitrate, frame rate, and quantization parameters. The one or more candidate output parameter groups are evaluated to obtain a score for each candidate output parameter group in the one or more candidate output parameter groups; Based on the score of each candidate output parameter group, a video output parameter group is determined from the one or more candidate output parameter groups; Based on the video output parameter group, the first video stream is output.

[0005] In the above scheme, generating one or more candidate output parameter groups for the first video stream based on video frames in the first video stream, as well as network state parameters and terminal state parameters, includes: The video frame is semantically segmented to obtain a semantic segmentation result; the semantic segmentation result is used to indicate one or more semantic regions of the video frame. Based on the semantic segmentation results, the content criticality of each semantic region in the one or more semantic regions is calculated; Based on the content criticality of each semantic region, the network state parameters, and the terminal state parameters, one or more candidate output parameter groups are generated.

[0006] In the above scheme, generating one or more candidate output parameter groups based on the content criticality of each semantic region, the network state parameters, and the terminal state parameters includes: A three-dimensional weight matrix is ​​generated based on the content criticality of each semantic region, as well as the network state parameters and terminal state parameters. The bitrate of each semantic region is calculated based on the three-dimensional weight matrix, the total available bitrate of the video frame, and the area of ​​each semantic region. The frame rate of the first video stream is calculated based on the set frame rate upper limit, the terminal state parameters, and the motion speed of each semantic region; the motion speed of the semantic region is obtained by tracking the corresponding semantic region in the most recent few video frames. Based on the set basic quantization parameters, the set maximum offset of the quantization parameters, and the content criticality of each semantic region, the quantization parameters of each semantic region are calculated.

[0007] In the above scheme, the step of evaluating the one or more candidate output parameter groups to obtain a score for each candidate output parameter group includes: For each candidate output parameter group, based on the candidate output parameter group, network status parameters, and terminal status parameters, multiple scoring items for the candidate output parameter group are calculated; the multiple scoring items include multiple items from text clarity, transmission delay, terminal overload risk, network congestion risk, and content mutation risk. The score of the candidate output parameter group is obtained based on the multiple scoring items and the set weight of each of the multiple scoring items.

[0008] In the above scheme, the calculation of multiple scoring items for the candidate output parameter group based on the candidate output parameter group, network status parameters, and terminal status parameters includes the following: Based on this candidate output parameter group, as well as the bitrate-clarity mapping model, frame rate-clarity mapping model, and quantization parameter-clarity mapping model, the text clarity sub-item is calculated; Based on the candidate output parameter group, the network status parameters, and the terminal status parameters, the transmission delay component is calculated. Based on the terminal state parameters, the candidate output parameter set, and the coding complexity model, the terminal overload risk sub-item is calculated; the coding complexity model is used to predict the terminal resources occupied by the video output according to the input candidate output parameter set; Based on the code rate in the candidate output parameter group and the network state parameters, the network congestion risk component is calculated. Based on the semantic segmentation results of the video frames, the content mutation risk component is calculated.

[0009] In the above scheme, determining the video output parameter group from the one or more candidate output parameter groups based on the score of each candidate output parameter group includes: Based on one or more of the following: the score of each candidate output parameter group, the maximum risk of each candidate output parameter group, the change amount corresponding to each candidate output parameter group, and the historical stability score of each candidate output parameter group, the priority score of each candidate output parameter group is calculated accordingly; the maximum risk is the maximum value among the terminal overload risk item, the network congestion risk item, and the content mutation risk item. Select the candidate output parameter group corresponding to the highest priority score as the video output parameter group.

[0010] The method in the above scheme further includes: During the output of the first video stream, network status parameters and terminal status parameters are monitored and acquired in real time. Based on the real-time acquired network status parameters and the bitrate in the video output parameter group, the target bitrate adjustment value is calculated. Based on the target bitrate adjustment value and the current bitrate of the first video stream, the bitrate of the first video stream is adjusted using a progressive adjustment strategy. Based on the real-time acquired terminal status parameters and network status parameters, as well as the frame rate in the video output parameter group, a frame rate adjustment target value is calculated, and the frame rate of the first video stream is adjusted to the frame rate adjustment target value.

[0011] This application also provides an electronic device, including a processor and a memory for storing a computer program that can run on the processor, wherein the processor, when running the computer program, performs the steps of any of the methods described above.

[0012] This application also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of any of the above methods.

[0013] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0014] The video transmission method, electronic device, storage medium, and computer program product provided in this application embodiment generate one or more candidate output parameter groups for the first video stream based on video frames in the first video stream, as well as network status parameters and terminal status parameters. The candidate output parameter groups include bitrate, frame rate, and quantization parameters. The one or more candidate output parameter groups are evaluated to obtain a score for each candidate output parameter group. Based on the scores of each candidate output parameter group, a video output parameter group is determined from the one or more candidate output parameter groups. Based on the video output parameter group, the first video stream is output. This scheme, by combining the current video frames with the current network status parameters and current terminal status parameters, determines the optimal bitrate, optimal frame rate, and optimal quantization parameters for video output under the current network environment and device status, enabling the video output strategy to maintain high-quality video output under the current network and terminal status, thereby improving the user's video viewing experience. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the video transmission system architecture provided in an embodiment of this application; Figure 2 A flowchart illustrating a video transmission method provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a video transmission device provided in an embodiment of this application; Figure 4 This is a schematic diagram of the electronic device structure provided in the embodiments of this application. Detailed Implementation

[0016] Currently, in the field of video transmission, with the widespread adoption of high-bandwidth demanding services such as Ultra High Definition (UHD) (e.g., 4K), Super Hi-Vision (SHV) (e.g., 8K), Virtual Reality (VR), and Augmented Reality (AR), the complex network environment and diverse terminal devices have brought enormous challenges to ensuring stable video quality. Users' requirements for the smoothness and clarity of video calls, live streaming, and on-demand playback are increasing, and traditional transmission optimization methods are struggling to continuously provide a high-quality experience under dynamically changing network conditions.

[0017] To address the issue of optimizing video transmission quality, current solutions mainly include adaptive bitrate adjustment algorithms based on network conditions and resource scheduling strategies based on terminal load. Among them, the adaptive bitrate adjustment algorithm based on network conditions dynamically switches bitrate levels according to parameters such as real-time bandwidth and latency; the resource scheduling strategy based on terminal load adjusts encoding parameters by monitoring the central processing unit (CPU) and memory usage of the device; some solutions are also attempting to perform simple partitioning of video content and assign fixed weights to allocate resources.

[0018] The main drawbacks of these video transmission quality optimization solutions are that they typically consider the network, terminal, and content dimensions separately, lacking deep integration; their content perception capabilities are weak, failing to intelligently distinguish the semantic importance of different areas in the image and dynamically allocate resources; at the same time, the strategy generation lacks forward-looking risk prediction, which can easily lead to device overload or network congestion, ultimately affecting the stability of the user experience.

[0019] Based on this, in various embodiments of this application, one or more candidate output parameter groups for the first video stream are generated based on video frames in the first video stream, as well as network state parameters and terminal state parameters; the candidate output parameter groups include bitrate, frame rate, and quantization parameters; the one or more candidate output parameter groups are evaluated to obtain a score for each candidate output parameter group; based on the score of each candidate output parameter group, a video output parameter group is determined from the one or more candidate output parameter groups; and based on the video output parameter group, the first video stream is output. The above scheme, by combining the current video frames with the current network state parameters and the current terminal state parameters, determines the optimal bitrate, optimal frame rate, and optimal quantization parameters for video output under the current network environment and device state, enabling the video output strategy to maintain high-quality video output under the current network state and current terminal state, thereby improving the user's video viewing experience.

[0020] The present application will now be described in further detail with reference to the accompanying drawings and embodiments.

[0021] This application provides a video transmission method, applicable to electronic devices or video transmission systems. When applied to electronic devices, it is an integrated deployment; when applied to video transmission systems, it is a distributed deployment, for example... Figure 1As shown, the video transmission system includes a semantic perception engine deployed at edge nodes, a decision optimization center deployed in the cloud, a prediction and evaluation module deployed at edge nodes with AI (Artificial Intelligence) acceleration chips, and a dynamic executor acting as a terminal agent, forming a closed-loop optimization system across network, terminal, and content. The semantic perception engine analyzes video frames, for example, performing real-time semantic segmentation and multi-dimensional data fusion on video frames in the first video stream. It extracts precise masks for regions such as faces and moving objects using a deep learning model, synchronously collects network and terminal state parameters, dynamically calculates the content criticality of each semantic region in the video frame, and finally generates a semantic graph which is uploaded to the decision optimization center. The decision optimization center generates one or more candidate output parameter sets for the first video stream based on the video frame analysis results, network and terminal state parameters, and outputs these candidate output parameter sets to the prediction and evaluation module. For example, it constructs a dynamic three-dimensional weight matrix based on the received semantic graph, with the three dimensions including content criticality, network state, and terminal state dimensions, generating a matrix that includes differentiated bitrate allocation and adaptive... The system sets a set of candidate output parameter groups with frame rate and quantization parameter (QP) settings, and outputs this set to the prediction and evaluation module. The prediction and evaluation module evaluates one or more candidate output parameter groups, obtaining a score for each group. For example, it predicts text clarity and end-to-end latency for each candidate output parameter group, and comprehensively evaluates terminal overload risk, network congestion risk, and content mutation risk. It outputs the sorted strategy and evaluation results to the dynamic executor, with the evaluation results including scores. The dynamic executor determines the video output parameter group from the candidate output parameter groups based on their scores, and outputs the first video stream based on the video output parameter group. For example, it selects the optimal output parameter group based on the evaluation results, performs smooth fine-tuning of bitrate and frame rate in conjunction with real-time network conditions, and continuously monitors the execution effect, forming a feedback optimization loop to ensure stable video transmission quality. Figure 2 As shown, the method includes: Step 201: Based on the video frames in the first video stream, as well as the network status parameters and terminal status parameters, generate one or more candidate output parameter groups for the first video stream.

[0022] The candidate output parameter group includes bit rate, frame rate, and quantization parameter.

[0023] Here, when a first video stream needs to be output to a terminal, the video frames in the first video stream are analyzed. For example, semantic segmentation is performed on the video frames to analyze the content criticality of semantic regions of different categories or instances of the video frames. Based on the analysis results of the video frames, as well as the current network state parameters and the current terminal state parameters, the bitrate, frame rate, and quantization parameters of the first video stream are generated. The network state parameters include bandwidth (B). t ), delay (D) t Packet loss rate (L) t ) and jitter (J t One or more of the following: terminal status parameters include CPU utilization (U cpu ), memory usage (U mem ) and chip temperature (T) current One or more of the following can be considered as candidate output parameters. A candidate output parameter set can also be understood as a candidate video output strategy. One or more generated candidate output parameter sets can form a candidate strategy set P = {P1, P2, ..., P...} L},P1,P2,…,P L Characterize each candidate output parameter group.

[0024] It should be noted that the video frame in the first video stream can be any video frame in the first video stream. For example, it can be the first video frame in the first video stream, or the video frame that triggers the re-determination of the video output parameter group. The re-determination of the video output parameter group can be triggered periodically, or it can be triggered by detecting a sudden change in network status, a sudden change in terminal status, or a sudden change in video frame content.

[0025] Step 202: Evaluate the one or more candidate output parameter groups to obtain a score for each candidate output parameter group in the one or more candidate output parameter groups.

[0026] Here, one or more candidate output parameter groups can be evaluated from one or more dimensions, including text clarity, end-to-end latency, terminal overload risk, network congestion risk, and video frame content mutation risk.

[0027] Step 203: Based on the score of each candidate output parameter group, determine the video output parameter group from the one or more candidate output parameter groups.

[0028] Here, the candidate output parameter group corresponding to the highest score can be directly determined as the video output parameter group; or further analysis can be carried out based on the score of each candidate output parameter group. For example, each candidate output parameter group can be further analyzed from one or more dimensions such as score, risk control, parameter smoothness and system stability, and the optimal candidate output parameter group can be selected as the video output parameter group from one or more candidate output parameter groups.

[0029] Step 204: Output the first video stream based on the video output parameter group.

[0030] Here, a first video stream can be output based on the determined video output parameter set. During the output of the first video stream, the video output parameter set can also be smoothly fine-tuned based on real-time network status parameters and terminal status parameters.

[0031] To differentiate the importance of different semantic regions on video frames, enabling precise resource allocation based on the real-time importance of the image content rather than a fixed template, and to maximize resource utilization and optimize video output under current network and terminal resource conditions, thus ensuring the transmission quality of critical information and user experience, in one embodiment, generating one or more candidate output parameter groups for the first video stream based on video frames in the first video stream, as well as network state parameters and terminal state parameters, includes: The video frame is semantically segmented to obtain a semantic segmentation result; the semantic segmentation result is used to indicate one or more semantic regions of the video frame. Based on the semantic segmentation results, the content criticality of each semantic region in the one or more semantic regions is calculated; Based on the content criticality of each semantic region, the network state parameters, and the terminal state parameters, one or more candidate output parameter groups are generated.

[0032] Here, the semantic segmentation result can be a two-dimensional matrix with the same size as the video frame. The elements of the two-dimensional matrix are the mask values ​​of the corresponding pixels, and the mask values ​​correspond to semantic categories; different semantic categories have different mask values. Based on the semantic segmentation result, the content criticality of each semantic region in one or more semantic regions is calculated, including: calculating the content criticality of each semantic region based on one or more of the following: the motion velocity of each semantic region, the set category weight corresponding to each semantic region, and an environment adjustment factor. Specifically, a motion enhancement coefficient is calculated based on the motion velocity of each semantic region. For each semantic region, the content criticality is calculated based on the product of one or more of the set category weight corresponding to that semantic region, the motion enhancement coefficient of that semantic region, and an environment adjustment factor. For example, this can be achieved using a formula... The content criticality of this semantic region is calculated, where, The content criticality of the c-th semantic region in the video frame at time t. The representation restricts the content criticality to the range [0,1]. The set category weights represent the semantic regions of type c. The motion enhancement coefficient characterizing the c-th semantic region The environmental adjustment factor at time t; the motion enhancement coefficient of the c-th semantic region can be expressed by the formula The calculation yielded that, The velocity or displacement vector representing the c-th semantic region. The maximum set motion speed is represented by k, which represents the preset amplification factor; for example, k can take the value of 1 or 2. The environmental adjustment factor can be based on the formula. Calculations show that, when resources are sufficient, it is not necessary to reduce the content criticality of each semantic region through environmental adjustment factors; that is, in and In this case, In other cases, ,in, The bandwidth representing time t, Characterizes the bandwidth penalty coefficient. Characterizes the set bandwidth threshold, The CPU utilization at time t is represented. Characterizes the setting of CPU utilization thresholds. This setting represents the maximum CPU utilization, typically set to 100% to prevent CPU overload. The CPU utilization penalty coefficient is set, and the penalty coefficient ranges from [0,1]. The calculation formula for the environmental adjustment factor can also be determined by establishing a correlation model between semantic content changes and fluctuations in network state parameters and terminal state parameters, such as calculating the Pearson correlation coefficient between semantic content changes and network state parameters and terminal state parameters.

[0033] It should be noted that content criticality can also be used to determine critical and non-critical regions. For example, if the content criticality of a semantic region is below a set criticality threshold, that semantic region is considered non-critical. In situations of resource scarcity, this can be mitigated by... Further reduce the content criticality of this semantic region, thereby reducing the bitrate allocation of this semantic region, increasing the QP value of this semantic region, and prioritizing the video output quality of critical regions.

[0034] It should be noted that real-time semantic segmentation can be performed on the video frames of the first video stream, and the segmented semantic regions can be tracked simultaneously. For example, semantic regions of a specific category can be tracked, or all segmented semantic regions can be tracked to obtain the motion velocity of each semantic region. Semantic segmentation can be performed by category to obtain the semantic region of each category, or by instance to obtain the semantic region of each instance.

[0035] Having calculated the content criticality of each semantic region, the bitrate, frame rate of the first video stream, and quantization parameters of each semantic region are calculated by combining the content criticality of each semantic region with the current network state parameters and the current terminal state parameters, thereby generating candidate output parameter sets. Multiple candidate output parameter sets are generated by fine-tuning one or more of the calculation factors.

[0036] To combine the content criticality of different semantic regions with the current network and terminal states to determine a video output strategy suitable for the current video frame and environmental state, and to achieve refined and personalized scheduling of encoding and transmission resources under multi-dimensional constraints, thereby improving video output quality, in one embodiment, generating one or more candidate output parameter groups based on the content criticality of each semantic region, the network state parameters, and the terminal state parameters includes: A three-dimensional weight matrix is ​​generated based on the content criticality of each semantic region, as well as the network state parameters and terminal state parameters; The bitrate of each semantic region is calculated based on the three-dimensional weight matrix, the total available bitrate of the video frame, and the area of ​​each semantic region. The frame rate of the first video stream is calculated based on the set frame rate upper limit, the terminal state parameters, and the motion speed of each semantic region; the motion speed of the semantic region is obtained by tracking the corresponding semantic region in the most recent few video frames. Based on the set basic quantization parameters, the set maximum offset of the quantization parameters, and the content criticality of each semantic region, the quantization parameters of each semantic region are calculated.

[0037] Here, a three-dimensional weight matrix of content keyness, network state, and terminal state can be constructed based on the content keyness, network state parameters, and terminal state parameters of each semantic region. The dimensions of the three-dimensional weight matrix can be C×N×M, where C represents the number of semantic region categories in the video frame, N represents the number of network state parameters, and M represents the number of terminal state parameters. The elements in the three-dimensional weight matrix are calculated based on the content keyness-content keyness weight mapping function, the network state parameter-network state weight mapping function, and the terminal state-terminal weight mapping function for the corresponding semantic region category. Specifically, the corresponding elements in the three-dimensional weight matrix are obtained by calculating the product of the weights of the three dimensions. The weights of the dimensions can be obtained by substituting the corresponding values ​​into the corresponding weight mapping functions; for example, it can be obtained through the formula... The elements in the three-dimensional weight matrix are calculated, where, The elements representing the c-th semantic region, the nth network state parameter, and the mth terminal state parameter are calculated. The content keyness weight mapping function can be used as follows: ; The network state weight mapping function, taking bandwidth as an example, can be: , Characterizes the maximum set bandwidth; The terminal state weight mapping function can be: N represents the network state parameter set, and M represents the terminal state parameter set.

[0038] When generating a 3D weight matrix, the bitrate of each semantic region is calculated based on the 3D weight matrix, the total available bitrate of the video frames, and the area of ​​each semantic region. Specifically, for each semantic region, a 3D comprehensive weight can be calculated based on the 3D weight matrix. For example, the 3D weight matrix can be reduced to a one-dimensional matrix with a content criticality dimension by calculating the average value. The first value of the semantic region is obtained by multiplying the 3D comprehensive weight of the semantic region by the pixel area of ​​the semantic region. The bitrate weight of the semantic region is obtained by calculating the ratio of the first value of the semantic region to the sum of the first values ​​of all semantic regions in the video frame. The bitrate of the semantic region is obtained by multiplying the bitrate weight of the semantic region by the total available bitrate of the video frames. For example, this can be achieved using a formula. The bitrate of the semantic region k is calculated, where the semantic region is the instance region, and... Characterizes the total available bitrate of video frames. The element representing the category of this semantic region in a one-dimensional matrix corresponding to the content keyness dimension. The pixel area of ​​the semantic region k is represented; K represents the number of independent semantic regions in the video frame.

[0039] Based on the set frame rate upper limit, terminal state parameters, and motion speed of each semantic region, the frame rate of the first video stream is calculated. Specifically, the average motion speed of all semantic regions with non-zero motion speed is calculated to obtain the average motion speed. The ratio of the average motion speed to the set maximum motion speed is calculated to obtain the motion speed component. The ratio of the difference between the set maximum CPU utilization and the CPU utilization to the set maximum CPU utilization is calculated to obtain the CPU utilization component. The product of the motion speed component and the CPU utilization component is calculated to obtain the candidate frame rate weight. The minimum value of 1 and the candidate frame rate weight is used as the final frame rate weight. The product of the set frame rate upper limit and the final frame rate weight is calculated to obtain the frame rate of the first video stream. For example, it can be obtained through the formula... The frame rate of the first video stream is calculated, where, Characterizes the maximum frame rate. Characterizing the average velocity of motion, Characterizes the maximum set motion speed. This characterizes the maximum CPU utilization setting. It represents CPU utilization. When the CPU utilization is close to the set maximum CPU utilization, the CPU utilization component approaches 0, the frame rate is suppressed to ensure terminal stability; the faster the movement speed, the closer the frame rate is to the set frame rate limit, ensuring the smoothness of moving objects.

[0040] It should be noted that, compared to directly using the motion speed of semantic regions, using the average motion speed of semantic regions can smooth out instantaneous disturbances, match human visual perception, so as to align computational statistics, better represent the intensity of motion in the picture, avoid frequent frame rate jumps, and ensure the smoothness of video playback.

[0041] Based on the set base quantization parameters, the set maximum offset of the quantization parameters, and the content criticality of each semantic region, the quantization parameters for each semantic region are calculated. Specifically, for each semantic region, the difference between 1 and the content criticality of that semantic region is calculated to obtain the weight of the quantization parameter offset. The product of the weight of the quantization parameter offset and the set maximum offset of the quantization parameters is calculated to obtain the quantization parameter offset of that semantic region. Finally, the sum of the quantization parameter offset and the set base quantization parameters is calculated to obtain the quantization parameters for that semantic region. For example, this can be achieved through a formula. The quantization parameters of this semantic region are calculated, where, Quantization parameters characterizing semantic region k, Characterization sets the basic quantization parameters, Characterizes the maximum offset of the set quantization parameters. The content criticality corresponding to the category of semantic region k is represented. The maximum offset of the quantization parameter is set to control the range of image quality fluctuation between video frames with different image complexities within the same semantic region. The basic quantization parameter is negatively correlated with the target bitrate, positively correlated with the video resolution, and positively correlated with the content complexity. That is, for videos with complex textures and violent motion, the basic quantization parameter is usually set higher to avoid bitrate runaway. The QP value range varies for different encoder standards.

[0042] It should be noted that different candidate output parameter sets can be generated by adjusting one or more parameters in the calculation process; for example, different candidate output parameter sets can be generated by adjusting one or more of the following: setting class weights. Preset magnification factor k Set bandwidth penalty coefficient, Set CPU usage penalty coefficient and content criticality weight mapping function. Network state weight mapping function Terminal state weight mapping function This includes adjusting category weights, such as changing the face weight from 1.0 to 0.8, or the background weight from 0.2 to 0.4; adjusting the preset magnification factor, setting the bandwidth penalty factor, and setting the CPU utilization penalty factor to change the weight changes when moving objects are present or resources are strained; and adjusting the weight mapping function, which can be understood as adjusting the non-linear function form of the weight mapping function, for example, changing the linear function to a logarithmic or exponential function. Different scene preference parameters can also be preset for different scenarios, for example, prioritizing text clarity in meeting scenarios and prioritizing the smoothness of faces or moving objects in live streaming scenarios.

[0043] To comprehensively evaluate candidate output parameter groups and improve the selection accuracy of video output parameter groups, in one embodiment, the evaluation of the one or more candidate output parameter groups to obtain a score for each candidate output parameter group includes: For each candidate output parameter group, based on the candidate output parameter group, network status parameters, and terminal status parameters, multiple scoring items for the candidate output parameter group are calculated; the multiple scoring items include multiple items from text clarity, transmission delay, terminal overload risk, network congestion risk, and content mutation risk. The score of the candidate output parameter group is obtained based on the multiple scoring items and the set weight of each of the multiple scoring items.

[0044] Here, the output parameters in the candidate output parameter group can be parsed and / or normalized first. The parsing includes bitrate distribution feature analysis, frame rate verification, QP value volatility evaluation, etc., to ensure that the subsequent evaluation can be based on accurate and reliable policy parameters.

[0045] When a semantic region of type text exists in a video frame, multiple scoring items may include multiple items from the following categories: text clarity, transmission delay, terminal overload risk, network congestion risk, and content mutation risk. When a semantic region of type text does not exist in a video frame, multiple scoring items may include multiple items from the following categories: transmission delay, terminal overload risk, network congestion risk, and content mutation risk.

[0046] Given multiple scoring items for the candidate output parameter group, the score can be obtained by calculating the product of text clarity and its corresponding weight, subtracting the product of transmission delay and its corresponding weight, the product of terminal overload risk and its corresponding weight, the product of network congestion risk and its corresponding weight, and the product of content mutation risk and its corresponding weight. For example, this can be achieved using the formula... The score for the candidate output parameter group is calculated; where, The score representing the l-th candidate output parameter group The weights assigned to represent text clarity are as follows: The sub-items representing text clarity, The weights assigned to characterize the transmission delay component. Characterizing the transmission delay component, The weights assigned to characterize the terminal overload risk components are as follows: Characterizing the terminal overload risk components, The weights assigned to characterize the network congestion risk components are... Characterizing network congestion risk components, The weights are assigned to the components representing the risk of content mutation. The scoring items represent the risk of sudden changes in content. The weight of each scoring item can be dynamically adjusted according to business needs. For example, in video conferencing scenarios, latency and text clarity are emphasized, while in on-demand scenarios, picture quality is emphasized.

[0047] The candidate output parameter groups are ranked based on their scores.

[0048] To comprehensively evaluate candidate output parameter sets and improve the accuracy of video output parameter set selection, in one embodiment, the process calculates multiple scoring items for the candidate output parameter set based on the candidate output parameter set, network status parameters, and terminal status parameters, including the following: Based on this candidate output parameter group, as well as the bitrate-clarity mapping model, frame rate-clarity mapping model, and quantization parameter-clarity mapping model, the text clarity sub-item is calculated; Based on the candidate output parameter group, the network status parameters, and the terminal status parameters, the transmission delay component is calculated. Based on the terminal state parameters, the candidate output parameter set, and the coding complexity model, the terminal overload risk sub-item is calculated; the coding complexity model is used to predict the terminal resources occupied by the video output according to the input candidate output parameter set; Based on the code rate in the candidate output parameter group and the network state parameters, the network congestion risk component is calculated. Based on the semantic segmentation results of the video frames, the content mutation risk component is calculated.

[0049] Here, based on the candidate output parameter set, as well as the bitrate-clarity mapping model, frame rate-clarity mapping model, and quantization parameter-clarity mapping model, the text clarity sub-item is calculated. Specifically, based on the bitrate in the candidate output parameter set, the sum of bitrates for all semantic regions of the text category can be calculated. Based on the sum of bitrates and the bitrate-clarity mapping model, the influence of bitrate on text clarity is calculated. Based on the quantization parameters in the candidate output parameter set, the sum of quantization parameters for all semantic regions of the text category can be calculated. Based on the sum of quantization parameters and the quantization parameter-clarity mapping model, the influence of quantization parameters on text clarity is calculated. Based on the frame rate and the frame rate-clarity mapping model in the candidate output parameter set, the influence of frame rate on text clarity is calculated. Based on the influence of bitrate on text clarity and the set bitrate influence weight, the influence of frame rate on text clarity and the set frame rate influence weight, and the influence of quantization parameters on text clarity and the set quantization parameter influence weight, a weighted sum is calculated to obtain the text clarity sub-item. Bitrate-to-clarity mapping models, frame rate-to-clarity mapping models, and quantization parameter-to-clarity mapping models can be trained in advance based on a large amount of experimental data.

[0050] For example, the bitrate-clarity mapping model can be This reflects the fundamental impact of bitrate on image clarity, using a logarithmic function to illustrate the law of diminishing marginal returns. The characterization category is the sum of the code rates of all semantic regions of the text. This can be understood as the total available bitrate of video frames. Frame rate resolution mapping model can be This addresses the negative impact of motion blur caused by insufficient frame rate on text sharpness; the quantized parameter sharpness mapping model can... The effect of quantifying QP values ​​on text quality, among which, The characterization category is the sum of quantized parameters for all semantic regions of the text. Text clarity can be represented by a formula. The calculation yielded that, The weights that represent the influence of the set bit rate are represented. The weights that represent the impact of setting the frame rate are represented. The quantization parameters used to characterize the weights are affected.

[0051] Based on the candidate output parameter set, network status parameters, and terminal status parameters, the transmission delay component is calculated. Specifically, the transmission delay can be understood as end-to-end delay, which can be divided into three stages: encoding, transmission, and decoding.

[0052] Encoding time can be calculated based on the frame rate and quantization parameters in the candidate output parameter group, as well as the terminal status parameters. The encoding time mainly depends on the complexity of the video content and the terminal's processing power. For example, the encoding latency can be calculated by summing the reciprocal of the frame rate and the second value in the candidate output parameter group. The second value represents the ratio of the third value to the fourth value. The third value represents the sum of the fifth values ​​of all semantic regions of the video frame. The fifth value represents the product of the pixel area of ​​the corresponding semantic region, the encoding complexity coefficient of the corresponding semantic region, and the encoding efficiency of the corresponding semantic region. The fourth value represents the product of the available CPU utilization and the CPU encoding performance coefficient. The encoding complexity coefficient is calculated based on the regional texture features, and the encoding efficiency is calculated based on the encoding efficiency function. The higher the QP value, the lower the encoding efficiency. The CPU encoding performance coefficient is a fixed coefficient obtained through offline testing based on terminal performance parameters such as terminal model, number of CPU cores, and CPU frequency. It represents the upper limit of the terminal's encoding processing capability. The stronger the encoding capability, the larger the CPU encoding performance coefficient.

[0053] For example, through formula The encoding time is calculated; where F represents the frame rate in the candidate output parameter group. The pixel area representing semantic region i; The encoding complexity coefficient representing semantic region i can be understood as the image gradient variance of semantic region i. The more complex the texture, the larger the variance. This can be based on... Calculations show that Texture features that characterize semantic region i; Characterizing coding efficiency function The quantization parameters characterizing semantic region i, for example, the coding efficiency function can be... ; Characterizes available CPU utilization; Characterizing CPU coding performance coefficients; Characterizes the second value; Represents the third numerical value; Characterizes the fourth value; It represents the fifth numerical value.

[0054] Transmission time can be calculated based on the bitrate and frame rate in the candidate output parameter group, as well as network state parameters. For example, transmission time can be estimated using a packet-level transmission model. Specifically, the transmission time is calculated based on the sum of the bandwidth impact factor, latency impact factor, packet loss rate impact factor, and jitter impact factor. The bandwidth impact factor is calculated based on the ratio of the sum of the bitrates of all semantic regions of the video frame to the sixth value. The sixth value represents the product of bandwidth and network protocol efficiency coefficient. The network protocol efficiency coefficient is a fixed coefficient preset according to the transmission protocol. For example, the network protocol efficiency coefficient of Quick UDP Internet Connections (QUIC) is higher than that of Transmission Control Protocol (TCP). Protocol); The impact of packet loss rate is the product of the seventh value and the average retransmission delay. The seventh value represents the ratio of the product of packet loss rate and the total available bitrate of video frames to the bandwidth. The average retransmission delay is obtained through statistics of historical packet loss rate and average round-trip delay. For example, the average retransmission delay is calculated based on the product of historical packet loss rate and average round-trip delay. The impact of jitter is the product of jitter and jitter coefficient. The jitter coefficient is calculated through the standard deviation of historical network jitter data and represents the degree of fluctuation in network latency. The greater the jitter, the greater the jitter coefficient, and the longer the transmission delay.

[0055] For example, through formula The transmission time was calculated; where, The sum of bitrates representing all semantic regions of a video frame. Characterizing the efficiency coefficient of a network protocol, Characterize the component affecting bandwidth. Represents the sixth numerical value; Characterization delay; Characterizing jitter, Characterizing the jitter coefficient, Characterize the components affected by jitter; Characterizing packet loss rate Characterizing the average delay of packet loss retransmission, It represents the seventh numerical value.

[0056] Decoding time can be obtained offline by fitting the terminal performance coefficient, bitrate, and resolution. The decoding time is implicit in the processing latency of the terminal device. For example, decoding time can be calculated based on the ratio of the total available bitrate of the video frames to the terminal's decoding performance coefficient, expressed by the formula: ,in, Characterizes the decoding performance coefficient of the terminal.

[0057] Transmission delay is the sum of encoding time, transmission time, and decoding time. For example, it can be expressed by the formula... The transmission delay component is calculated. When calculating the transmission delay component, the delay overlap effect caused by parallel processing can also be considered.

[0058] Based on terminal status parameters, the candidate output parameter set, and the coding complexity model, terminal overload risk components are calculated. Terminal overload risk assessment is based on the terminal's current load state and the computing resources required to execute the output parameter set. Specifically, terminal overload risk components can be calculated based on the product of the computing power load factor, memory load factor, and temperature control factor. The computing power load factor reflects the CPU's computing power saturation level after the coding task adds load; the memory load factor amplifies the terminal overload risk when memory is scarce; and the temperature control factor reflects the characteristic that the risk increases exponentially when the chip temperature approaches a threshold. The computing power load factor is the ratio of the sum of the predicted CPU utilization increment value corresponding to the candidate output parameter set and the current CPU utilization rate to the set maximum CPU utilization rate. The set maximum CPU utilization rate can be understood as the chip specification parameter or the full-load computing power value calibrated by stress testing. The predicted CPU utilization increment value corresponding to the candidate output parameter set is the predicted CPU utilization increment value obtained by inputting the candidate output parameter set into the coding complexity model. The incremental utilization prediction value can be understood as the increase in CPU computing power occupied by the subsequent coding process, used to predict the increase in CPU load after executing the candidate output parameter group; the memory load factor is the sum of 1 and the eighth value, where the eighth value represents the product of the memory pressure weight coefficient and the ninth value, and the ninth value represents the ratio of the current memory usage to the maximum available memory capacity. The memory pressure weight coefficient can be understood as the weight of the impact of memory load on the overall risk. It is obtained by fitting the terminal overload failure probability under different memory usage through offline experimental testing. It is a preset fixed hyperparameter that can be adjusted according to the business scenario. For example, it can be set to 0.3 for video conferencing scenarios and 0.1 for on-demand scenarios; the temperature control factor is an S-shaped sigmoid function, where the exponent of e is the product of the negative of the terminal temperature sensitivity coefficient and the tenth value, where the tenth value is the difference between the terminal temperature control protection threshold and the terminal chip temperature. The terminal temperature sensitivity coefficient is obtained by fitting through offline stress testing and is used to describe the steep growth rate of equipment failure or equipment overload risk when the chip temperature approaches the threshold.

[0059] For example, through formula The terminal overload risk components were calculated; among them, Characterizing the computing load factor, This represents the predicted CPU utilization increment value corresponding to the candidate output parameter group. Characterizing memory load factor, Representing the eighth value, Characterizing memory weight coefficients, Indicates terminal memory usage. Indicates the maximum available memory capacity. Represents the ninth numerical value; Characterizing the temperature control factor, Characterizing the terminal temperature sensitivity coefficient, Representing the tenth value, Characterizing the terminal temperature control protection threshold, Characterizes the current chip temperature of the terminal.

[0060] Based on the bitrate in the candidate output parameter group and the network state parameters, the network congestion risk component is calculated. The network congestion risk comprehensively considers bandwidth utilization, packet loss rate, and jitter. Specifically, the network congestion risk component is calculated based on the product of the tenth, eleventh, twelfth, and thirteenth values. The tenth value represents the square of the ratio of the sum of the bitrates of all semantic regions of the video frame to the fourteenth value. The fourteenth value represents the product of the total physical bandwidth and the network bandwidth security utilization coefficient. The eleventh value represents the ratio of the packet loss rate to the maximum packet loss rate. The twelfth value is obtained by adding 1 to the ratio of jitter to the maximum jitter value. The thirteenth value represents the sum of 1 and the fifteenth value. The fifteenth value represents the product of the bandwidth trend sensitivity coefficient and the negative rate of change of bandwidth. The network bandwidth security utilization factor can also be understood as a security bandwidth utilization threshold, used as a benchmark parameter to define the bandwidth security operating range. The network bandwidth security factor is typically less than 1, for example, between 0.6 and 0.8. The smaller the network bandwidth security factor, the more redundant bandwidth is reserved, and the less likely the network is to experience congestion, packet loss, or attack amplification. The network bandwidth security utilization factor can be determined by collecting historical peak traffic, packet loss rate, and latency data to fit the maximum long-term bandwidth utilization rate that does not trigger significant congestion, lag, or packet loss degradation. This value is then defined as the network bandwidth security utilization factor. In highly real-time services such as live streaming or live video, the network bandwidth security utilization factor is set even lower to reserve more burst margin. For ordinary file transfer services, the network bandwidth security utilization factor can be appropriately increased. The network bandwidth security utilization factor can also be adjusted according to network security level and protection requirements. Utilizing high-security links can lower the network bandwidth security utilization factor, increasing redundant bandwidth and preventing attacks from being carried out by Distributed Denial of Service (DDoS) attacks. For services and traffic surge risks, standard values ​​can be set for ordinary access links based on routine operation and maintenance experience. The network bandwidth security utilization coefficient can also be fine-tuned by long-term monitoring of congestion frequency, stuttering rate, and number of failures under different network bandwidth security utilization coefficients. For example, if failures frequently occur close to the bandwidth limit, the network bandwidth security utilization coefficient should be reduced; if the bandwidth idle rate is too high for a long period, the network bandwidth security utilization coefficient should be appropriately increased. The bandwidth trend sensitivity coefficient is a preset fixed coefficient, usually ranging from [0,1]. It represents the penalty for a bandwidth decline trend; the larger the bandwidth trend sensitivity coefficient, the faster the bandwidth declines, and the higher the risk of network congestion.

[0061] For example, through formula The network congestion risk component is calculated; where B represents the total physical bandwidth of the link. Characterizes the network bandwidth security utilization coefficient. This represents the preset maximum available secure bandwidth for the link, reserving redundant bandwidth to cope with traffic surges, network jitter, and security protection overhead. This indicates the degree of congestion in terms of the current total bandwidth usage relative to the safe bandwidth limit. Represents the tenth numerical value; Characterizes the maximum packet loss rate. Representing the eleventh value, Characterizes the maximum value of jitter. Represents the twelfth numerical value; Characterizing the sensitivity coefficient to bandwidth trends; The negative rate of change characterizing bandwidth, where, It can be calculated using the difference approximation method, i.e. , Represents the bandwidth at the previous time step. The sampling interval (e.g., 100 milliseconds) is represented. A negative result indicates a decrease in bandwidth; the larger the absolute value, the faster the decrease and the higher the risk of network congestion. This represents the thirteenth numerical value. .

[0062] Based on the semantic segmentation results of video frames, a content mutation risk component is calculated. The content mutation risk is predicted by analyzing the temporal variation patterns of semantic features. Specifically, the content mutation risk component is obtained by calculating the average content mutation probability of all semantic regions in the video frame. For each semantic region, the content mutation probability of that semantic region is calculated by multiplying the absolute value of the change rate of the content keyness of that semantic region, the ratio of the motion speed of that semantic region to the set maximum motion speed, and the sum of the texture complexity coefficient of that semantic region and 1. Among these, the texture complexity coefficient of the semantic region is the product of the texture weight coefficient and the encoding complexity coefficient of that semantic region.

[0063] For example, through formula The calculated content risk items are as follows: The rate of change representing the content criticality of this semantic region can be calculated using the difference approximation method, i.e. The larger the absolute value of the rate of change of content keyness, the more drastic the change in keyness representing the semantic region (such as the sudden appearance of a face or text), and the higher the risk of content mutation. Content keyness is calculated based on the semantic segmentation results of video frames. For the specific calculation method, please refer to the relevant description above. The motion velocity of semantic region i is characterized and determined based on the semantic segmentation results of video frames; Characterizes the texture complexity coefficient; Texture weight coefficients are characterized by offline acquisition of various semantic scenes (such as text, faces, and landscapes) to test the impact of different texture richness on subjective image quality and encoding bitrate. A baseline texture weight is then fitted; for example, high-texture regions (such as text and facial features) have a larger baseline texture weight, while low-texture regions (such as solid-color backgrounds) have a smaller baseline texture weight. During the first video stream output, the texture weights can be fine-tuned based on the motion intensity and network congestion risk of each semantic region to obtain real-time texture weights. For example, when the motion speed is too high, the texture weight can be appropriately increased to avoid motion texture blurring; when the network congestion risk is high, the texture weight can be slightly decreased to prioritize semantic contours over subtle textures. Real-time texture weights can be calculated using the formula... Calculated; The encoding complexity coefficient representing semantic region i can be understood as the image gradient variance of semantic region i. The more complex the texture, the larger the variance. This can be based on... Calculations show that Texture features that characterize semantic region i.

[0064] To improve the scientific rigor of output strategy selection and enhance the stability, reliability, and success rate of the output strategy, in one embodiment, determining the video output parameter set from one or more candidate output parameter sets based on the score of each candidate output parameter set includes: Based on one or more of the following: the score of each candidate output parameter group, the maximum risk of each candidate output parameter group, the change amount corresponding to each candidate output parameter group, and the historical stability score of each candidate output parameter group, the priority score of each candidate output parameter group is calculated accordingly; the maximum risk is the maximum value among the terminal overload risk item, the network congestion risk item, and the content mutation risk item. Select the candidate output parameter group corresponding to the highest priority score as the video output parameter group.

[0065] Here, the priority score for each candidate output parameter group is calculated, including: for each candidate output parameter group, the product of the candidate output parameter group's score and its corresponding set weight is calculated, minus the product of the candidate output parameter group's maximum risk and its corresponding set weight, minus the product of the candidate output parameter group's change and its corresponding set weight, and adding the product of the candidate output parameter group's historical stability score and its corresponding set weight. One or more of these factors can be selected based on the actual application. The candidate output parameter group with the highest priority score is selected as the video output parameter group. Further selection of candidate output parameter groups is conducted from four aspects: strategy score, risk control, parameter smoothness, and system stability. This ensures that the final selected video output parameter group can improve video output performance in the short term and maintain system stability in the long term, considering not only the immediate effect of the strategy but also the long-term stability of the system. A higher priority score indicates better overall adaptability of the corresponding candidate output parameter group in the current scenario.

[0066] For example, through formula Select the candidate output parameter group with the highest priority score; among them... Characterizes the video output parameter set; The maximum value among the terminal overload risk, network congestion risk, and content mutation risk items of the candidate output parameter group represents the risk level of the entire candidate output parameter group as long as any one of the content mutation, network congestion, or terminal overload risks is high. In other words, the maximum risk value is used as a conservative constraint to prevent the risk from being missed in a single dimension. The change in the corresponding candidate output parameter set can be understood as the candidate output parameter set. The degree of difference from the currently used set of output parameters; Characterizes the historical stability of this candidate output parameter set; , , , It can be configured according to business objectives and the characteristics of the business scenario, such as in a live streaming scenario. high, Secondly, in weak network or conference scenarios, and higher.

[0067] The change corresponding to the candidate output parameter set It can be calculated based on the weighted sum of the frame rate change, bit rate change, and quantization parameter change in the candidate output parameter group. For example, it can be calculated using the formula... The change corresponding to the candidate output parameter set is calculated, where, The set weights that characterize the changes in frame rate. Characterizes the amount of frame rate change. The set weights that represent the changes in bit rate. Characterizes the amount of change in bit rate. The set weights that characterize the changes in the quantization parameters Characterizes the change in quantification parameters; The closer the parameter set is to 0, the less change it represents compared to the currently used parameter set, indicating no risk of oscillation during switching. The closer it is to 1, the more drastic the change in performance. Switching is very likely to cause drastic fluctuations in bitrate / image quality and decoding stuttering. The lower the smoothness, the lower the priority score is, and the lower the probability of being selected.

[0068] Here, the frame rate change represents the magnitude of the overall smoothness change. It can be calculated by the ratio of the absolute value of the difference between the frame rate in the candidate output parameter group and the currently used frame rate to the maximum allowable frame rate change limit. The maximum allowable frame rate change limit is determined based on the difference between the hardware frame rate ranges, expressed by the formula: ,in, Indicates the currently used frame rate. This represents the maximum allowable variation in frame rate. Since a constant total available bitrate but a redistribution of bitrate between semantic regions can still cause abrupt changes in text clarity, the bitrate variation for each semantic region must be considered. Therefore, the bitrate variation can be calculated by averaging the bitrate variations across all semantic regions of the video frame. For each semantic region, the bitrate variation can be calculated by the ratio of the absolute value of the difference between the bitrate of that semantic region in the candidate output parameter set and the bitrate of the currently used corresponding position region to the bitrate of the currently used corresponding position region. The formula is expressed as follows: ,in, The code rate change value representing semantic region i. The frame rate currently used in semantic region i is represented. To reflect the magnitude of changes in global image quality distribution, the change in quantization parameters can be calculated by averaging the quantization parameter changes across all semantic regions of the video frame. For each semantic region, the change in quantization parameters can be calculated by the ratio of the absolute value of the difference between the quantization parameters of that semantic region in the candidate output parameter group and the quantization parameters of the corresponding currently used region, to the length of the encoder's QP value range. The formula is expressed as follows: ,in, The change in the quantization parameter representing the semantic region i, The quantization parameters currently used to represent semantic region i Characterizes the length of the QP value range of the encoder.

[0069] Historical stability of the candidate output parameter set This can be obtained by scoring the historical stability of the candidate output parameter group by technical personnel, or by calculating performance data from multiple execution cycles. Performance data includes one or more of the following: bitrate fluctuation amplitude, parameter group switching frequency, and execution success rate. Bitrate fluctuation amplitude is calculated using the standard deviation of bitrate changes between adjacent execution cycles. The parameter group switching frequency reflects whether the candidate output parameter group is frequently replaced by other output parameter strategies. The execution success rate is determined by the number of playback stutters or decoding failures for the candidate output parameter group. For example, it can be obtained using a formula... Calculate the historical stability of the candidate output parameter set, where, The normalized standard deviation representing bit rate fluctuation, Characterizes the switching frequency of parameter groups. Characterizes the switching frequency penalty coefficient.

[0070] Furthermore, a priority matrix can be constructed for one or more of the following: the score of each candidate output parameter group, the maximum risk of each candidate output parameter group, the change corresponding to each candidate output parameter group, and the historical stability score of each candidate output parameter group. The rows of the priority matrix can represent candidate output parameter groups, and the columns can represent priority scoring dimensions, including one or more optimization objectives among the score, maximum risk, corresponding change, and historical stability score of the candidate output parameter group. That is, the i-th row and j-th column of the priority matrix represents the priority or performance value of the i-th candidate output parameter group on the j-th optimization objective. Then, a weighted sum is calculated for each row of the priority matrix to obtain the priority score of the corresponding candidate output parameter group.

[0071] It should be noted that the calculation of the priority score for each candidate output parameter group may also include: for each candidate output parameter group, the priority score for that candidate output parameter group is calculated based on the product of the sixteenth and seventeenth values; the sixteenth value represents the ratio of the score of the candidate output parameter group to the sum of the maximum risk and the minimum constant of the candidate output parameter group; the seventeenth value represents the difference between 1 and the change corresponding to the candidate output parameter group.

[0072] For example, through formula The priority score of the candidate output parameter group is calculated; where, The priority score characterizes the candidate output parameter group l. It represents a minimal constant used to ensure numerical stability.

[0073] To ensure smooth transmission and overall system stability, in one embodiment, the method further includes: During the output of the first video stream, network status parameters and terminal status parameters are monitored and acquired in real time. Based on the real-time acquired network status parameters and the bitrate in the video output parameter group, the target bitrate adjustment value is calculated. Based on the target bitrate adjustment value and the current bitrate of the first video stream, the bitrate of the first video stream is adjusted using a progressive adjustment strategy. Based on the real-time acquired terminal status parameters and network status parameters, as well as the frame rate in the video output parameter group, a frame rate adjustment target value is calculated, and the frame rate of the first video stream is adjusted to the frame rate adjustment target value.

[0074] Here, during the process of outputting the first video stream based on the video output parameter group, the network status and terminal status can be monitored in real time, and the network status parameters and terminal status parameters can be obtained in real time. Based on the real-time obtained network status parameters and the bitrate in the video output parameter group, an adaptive algorithm based on a sliding window is used to calculate the target bitrate adjustment value for the current time window. For example, based on the real-time obtained network status parameters, the bitrate adjustment coefficient is calculated, and the target bitrate adjustment value is calculated based on the product of the bitrate adjustment coefficient and the bitrate in the video output parameter group. Based on real-time acquired network state parameters, the bitrate adjustment coefficient is calculated, including: calculating 1 plus a bandwidth impact adjustment item, subtracting a packet loss rate impact adjustment item, and adding a jitter impact adjustment item to obtain the bitrate adjustment coefficient; the bandwidth impact adjustment item is calculated based on the product of the bandwidth sensitivity coefficient and the bandwidth impact coefficient, which is calculated based on the ratio of the difference between the real-time acquired bandwidth and the average bandwidth to the average bandwidth, where the average bandwidth represents the average bandwidth within the most recent time window; the packet loss rate impact adjustment item is calculated based on the product of the packet loss rate sensitivity coefficient and the packet loss rate impact coefficient, which is calculated based on the ratio of the difference between the real-time acquired packet loss rate and the target packet loss rate to the target packet loss rate; the jitter impact adjustment item is calculated based on the product of the jitter sensitivity coefficient and the jitter impact coefficient, which is calculated based on the ratio of the difference between the real-time acquired jitter and the set jitter threshold to the set jitter threshold; the bandwidth sensitivity coefficient, packet loss rate sensitivity coefficient, and jitter sensitivity coefficient can be dynamically adjusted using machine learning algorithms.

[0075] For example, through formula Calculate the target bitrate adjustment value for the current time window, where, It can characterize the bitrate of each semantic region in the video output parameter set. Sensitivity coefficient characterizing bandwidth Characterizing the bandwidth influence coefficient, Characterizes the bandwidth acquired in real time. Characterizing average bandwidth, Sensitivity coefficient characterizing packet loss rate Characterizing the impact coefficient of packet loss rate, Characterizes the real-time packet loss rate. Characterizes the target packet loss rate. Sensitivity coefficient characterizing jitter, Characterizing the impact coefficient of jitter, Characterizes the jitter acquired in real time. The jitter threshold is set to represent the target packet loss rate. Both the target packet loss rate and the set jitter threshold can be pre-defined or dynamically configured based on the business scenario, user experience goals, network environment, and device encoding / decoding capabilities.

[0076] Based on the target bitrate adjustment value and the current bitrate of the first video stream, a progressive adjustment strategy is used to adjust the bitrate of the first video stream. This progressive adjustment strategy involves calculating the bitrate value for the next time step based on the sum of the current bitrate, the bitrate smoothing adjustment value, and the bandwidth trend prediction adjustment value. The bitrate is then adjusted at the next time step based on this next bitrate value until it reaches the target bitrate adjustment value within the current time window. The bitrate smoothing adjustment value is calculated based on the product of a smoothing factor and the difference between the current bitrate and the target bitrate adjustment value. The bandwidth trend prediction adjustment value is calculated based on the product of a bandwidth change trend sensitivity coefficient and a bandwidth change rate. This progressive adjustment strategy ensures the smoothness of bitrate changes during the output of the first video stream. The current bitrate can be understood as the bitrate currently used by the first video stream.

[0077] For example, through formula Calculate the bitrate value at the next time step, where, The bitrate value representing the next time step. Indicates the currently used bitrate. Characterizes the bit rate smoothing adjustment value. Characterizing smoothness factor, Indicates the target value for bit rate adjustment. Characterizing the bandwidth trend prediction adjustment value, Sensitivity coefficient characterizing bandwidth change trend The rate of change of bandwidth can be calculated using the difference approximation method.

[0078] Based on real-time acquired terminal and network status parameters, as well as the frame rate in the video output parameter set, a frame rate smoothing control mechanism is adopted. Taking into account both terminal processing capabilities and latency requirements, a target frame rate adjustment value is calculated. Specifically, based on real-time acquired terminal status parameters, a frame rate adjustment coefficient is calculated. For example, the minimum value between the product of the latency constraint factor, CPU load constraint factor, and battery life constraint factor and 1 is determined as the frame rate adjustment coefficient. The latency constraint factor is calculated based on the ratio of the target end-to-end latency to the currently calculated real-time transmission latency. The target end-to-end latency is preset according to the business scenario (e.g., the target end-to-end latency can be set to 150 milliseconds in a video conferencing scenario and 300 milliseconds in a live streaming scenario). The currently calculated real-time transmission latency... The transmission latency component is calculated based on the video output parameter set, the currently acquired network status parameters, and the terminal status parameters. The specific calculation method is described above. A smaller transmission latency component results in a smaller latency constraint factor, reducing the frame rate and preventing further latency exceedances. The CPU load constraint factor is calculated as the ratio of the difference between the set maximum CPU utilization and the currently acquired CPU utilization, to the difference between the set maximum CPU utilization and the CPU idle baseline utilization. Higher CPU utilization results in a smaller CPU load constraint factor and a larger frame rate reduction, preventing terminal overload. The battery life constraint factor is calculated based on the ratio of the currently acquired terminal battery level to the terminal's full battery level. Lower current terminal battery level results in a smaller battery life constraint factor, smoothly reducing frame rate to lower power consumption. Frame rate adjustment uses a comparator algorithm with hysteresis to avoid frequent oscillations near adjacent points and incorporates a battery factor to ensure the mobile terminal device's battery life.

[0079] For example, through formula The frame rate adjustment target value is calculated; where, Characterizes the target value for frame rate adjustment. Characterizes the frame rate in the video output parameter set. Characterizing the time delay constraint factor, Characterizing the target end-to-end delay, Characterizing the transmission delay component of current real-time computing, Characterizing CPU load constraint factors, This characterizes the maximum CPU utilization setting. Characterizes the CPU utilization obtained in real time. Characterizes CPU idle baseline utilization. Characterizing the battery life constraint factor, This represents the real-time battery level of the terminal. This indicates that the terminal is fully charged.

[0080] The following section provides a more detailed description of this application with reference to application examples.

[0081] The video output method includes the following steps: Step 1: Perform semantic segmentation on the video frames in the first video stream to obtain the semantic segmentation results.

[0082] Here, network state parameters and terminal state parameters can be collected simultaneously and in parallel. Semantic segmentation results can also include pixel-level masks for faces, text, moving objects, and backgrounds. Furthermore, the segmented semantic regions can be tracked; for example, the displacement vector or velocity of semantic regions categorized as moving objects can be calculated.

[0083] Step 2: Based on the semantic segmentation results, calculate the content criticality of each semantic region in one or more semantic regions.

[0084] Here, a semantic graph can be constructed based on the semantic segmentation results, the content criticality of each semantic region, and the collected network state parameters and terminal state parameters. The semantic graph can include: the mask coordinates and category of each semantic region, the content criticality and segmentation confidence of each semantic region, network state parameters, and terminal state parameters. When using the semantic graph to generate candidate output parameter sets, the data in the semantic graph can be preprocessed first, including data alignment and smoothing filtering to eliminate interference from instantaneous fluctuations. The method of generating candidate output parameter sets based on the semantic graph is the same as the method of generating one or more candidate output parameter sets based on the content criticality of each semantic region, network state parameters, and terminal state parameters.

[0085] Step 3: Generate a three-dimensional weight matrix based on the content criticality of each semantic region, as well as network state parameters and terminal state parameters.

[0086] The specific implementation of steps 3-8 is described above and will not be repeated here.

[0087] Step 4: Calculate the bitrate of each semantic region based on the three-dimensional weight matrix, the total available bitrate of the video frames, and the area of ​​each semantic region.

[0088] Step 5: Calculate the frame rate of the first video stream based on the set frame rate upper limit, terminal status parameters, and motion speed of each semantic region.

[0089] Step 6: Based on the set basic quantization parameters, the set maximum offset of the quantization parameters, and the content criticality of each semantic region, calculate the quantization parameters for each semantic region.

[0090] Here, steps 3-6 describe the generation method of a candidate output parameter group. Different candidate output parameter groups can be generated by adjusting one or more parameters used in the process of generating the candidate output parameter group, that is, multiple candidate output parameter groups can be generated.

[0091] Step 7: For each candidate output parameter group, calculate multiple scoring items based on the candidate output parameter group, network status parameters, and terminal status parameters. Based on the multiple scoring items and the set weight of each scoring item, obtain the score of the candidate output parameter group.

[0092] Step 8: Based on one or more of the following: the score of each candidate output parameter group, the maximum risk of each candidate output parameter group, the change amount corresponding to each candidate output parameter group, and the historical stability score of each candidate output parameter group, calculate the priority score of each candidate output parameter group, and select the candidate output parameter group corresponding to the highest priority score as the video output parameter group.

[0093] Step 9: Output the first video stream based on the video output parameter group.

[0094] Here, once the video output parameter group is determined, the output parameters of the first video stream are set according to the video output parameter group, and the first video stream is output to the corresponding terminal.

[0095] Step 10: During the first video stream output process, monitor and acquire network status parameters and terminal status parameters in real time, and fine-tune the video output parameter group based on the acquired network status parameters and terminal status parameters in real time.

[0096] Here, a target bitrate adjustment value can be calculated based on real-time acquired network status parameters and the bitrate in the video output parameter group. Based on the target bitrate adjustment value and the current bitrate of the first video stream, the bitrate of the first video stream is adjusted using a gradual adjustment strategy. A target frame rate adjustment value is calculated based on real-time acquired terminal status parameters, network status parameters, and the frame rate in the video output parameter group, and the frame rate of the first video stream is adjusted to the target frame rate adjustment value. The specific implementation method is described above.

[0097] In this application embodiment, the video transmission method can be applied to electronic devices, or to... Figure 1The video transmission system shown constructs a cloud-based collaborative closed-loop optimization system based on semantic awareness. By deeply coupling intelligent semantic understanding of video content with network transmission control, it builds a complete technology chain from semantic awareness, decision optimization, prediction and evaluation to dynamic execution, realizing global intelligent control over the video transmission process. This fundamentally improves the decision-making ability and overall efficiency of adaptive transmission in complex network environments. This application embodiment can also dynamically calculate and update the content criticality of each semantic region in real time, enabling the system to accurately guide resource allocation based on the real-time importance of the screen content rather than a fixed template, ensuring the transmission quality of key information and user experience. A three-dimensional dynamic weight matrix integrating content criticality, network status, and terminal load status serves as the core decision-making mechanism, driving the generation of differentiated bitrate allocation, adaptive frame rate, and QP parameter sets, achieving refined and personalized scheduling of encoding and transmission resources under multi-dimensional constraints. During strategy evaluation, by predicting text clarity and end-to-end latency, and comprehensively assessing three risk levels—device overload, network congestion, and content mutation—the system simulates, verifies, and sorts candidate strategies, significantly improving strategy reliability and execution success rate, and effectively mitigating potential risks. During video stream output, a dynamic execution and fine-tuning mechanism based on multi-objective optimization and real-time feedback selects the optimal strategy based on risk assessment results and smoothly adjusts parameters such as bitrate and frame rate according to real-time network conditions, ensuring smooth transmission and overall system stability.

[0098] To implement the method of the embodiments of this application, the embodiments of this application also provide a video transmission device, such as... Figure 3 As shown, the device includes: The generation unit 301 is configured to generate one or more candidate output parameter groups for the first video stream based on video frames in the first video stream, as well as network status parameters and terminal status parameters; the candidate output parameter groups include bitrate, frame rate and quantization parameters; Evaluation unit 302 is used to evaluate the one or more candidate output parameter groups and obtain a score for each candidate output parameter group in the one or more candidate output parameter groups; The determining unit 303 is used to determine a video output parameter group from the one or more candidate output parameter groups based on the score of each candidate output parameter group; The output unit 304 is used to output the first video stream based on the video output parameter group.

[0099] In one embodiment, the generation unit 301 is specifically used to perform semantic segmentation on the video frame to obtain a semantic segmentation result; the semantic segmentation result is used to indicate one or more semantic regions of the video frame; Based on the semantic segmentation results, the content criticality of each semantic region in the one or more semantic regions is calculated; Based on the content criticality of each semantic region, the network state parameters, and the terminal state parameters, one or more candidate output parameter groups are generated.

[0100] In one embodiment, the generation unit 301 is specifically used to generate a three-dimensional weight matrix based on the content criticality of each semantic region and the network state parameters and terminal state parameters; The bitrate of each semantic region is calculated based on the three-dimensional weight matrix, the total available bitrate of the video frame, and the area of ​​each semantic region. The frame rate of the first video stream is calculated based on the set frame rate upper limit, the terminal state parameters, and the motion speed of each semantic region; the motion speed of the semantic region is obtained by tracking the corresponding semantic region in the most recent few video frames. Based on the set basic quantization parameters, the set maximum offset of the quantization parameters, and the content criticality of each semantic region, the quantization parameters of each semantic region are calculated.

[0101] In one embodiment, the evaluation unit 302 is specifically used to calculate multiple scoring items for each candidate output parameter group based on the candidate output parameter group, network status parameters, and terminal status parameters; the multiple scoring items include multiple items from text clarity, transmission delay, terminal overload risk, network congestion risk, and content mutation risk. The score of the candidate output parameter group is obtained based on the multiple scoring items and the set weight of each of the multiple scoring items.

[0102] In one embodiment, the evaluation unit 302 is specifically used for the following purposes: Based on this candidate output parameter group, as well as the bitrate-clarity mapping model, frame rate-clarity mapping model, and quantization parameter-clarity mapping model, the text clarity sub-item is calculated; Based on the candidate output parameter group, the network status parameters, and the terminal status parameters, the transmission delay component is calculated. Based on the terminal state parameters, the candidate output parameter set, and the coding complexity model, the terminal overload risk sub-item is calculated; the coding complexity model is used to predict the terminal resources occupied by the video output according to the input candidate output parameter set; Based on the code rate in the candidate output parameter group and the network state parameters, the network congestion risk component is calculated. Based on the semantic segmentation results of the video frames, the content mutation risk component is calculated.

[0103] In one embodiment, the determining unit 303 is specifically used to calculate the priority score of each candidate output parameter group based on one or more of the following: the score of each candidate output parameter group, the maximum risk value of each candidate output parameter group, the change amount corresponding to each candidate output parameter group, and the historical stability score of each candidate output parameter group; the maximum risk value is the maximum value among the terminal overload risk item, the network congestion risk item, and the content mutation risk item. Select the candidate output parameter group corresponding to the highest priority score as the video output parameter group.

[0104] In one embodiment, the device further includes: The acquisition unit is used to monitor and acquire network status parameters and terminal status parameters in real time during the output of the first video stream. The first calculation unit is used to calculate the target value of bitrate adjustment based on the network status parameters acquired in real time and the bitrate in the video output parameter group. The first adjustment unit is used to adjust the bitrate of the first video stream based on the bitrate adjustment target value and the current bitrate of the first video stream, using a progressive adjustment strategy. The second calculation unit is used to calculate the frame rate adjustment target value based on the real-time acquired terminal status parameters and network status parameters, as well as the frame rate in the video output parameter group. The second adjustment unit is used to adjust the frame rate of the first video stream to the frame rate adjustment target value.

[0105] In practical applications, the generation unit 301, the evaluation unit 302, the determination unit 303, the output unit 304, the acquisition unit, the first calculation unit, the first adjustment unit, the second calculation unit, and the second adjustment unit can be implemented by a processor in the video transmission device combined with a communication interface.

[0106] It should be noted that the video transmission device provided in the above embodiments is only illustrated by the division of the above-described program modules. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the video transmission device and the video transmission method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0107] Based on the hardware implementation of the above program modules, embodiments of this application also provide an electronic device, such as... Figure 4 As shown, the electronic device 400 includes: Communication interface 401 allows for information exchange with other devices or network nodes.

[0108] The processor 402 is connected to the communication interface 401 to enable information interaction with other devices or network nodes, and to execute the methods provided by one or more technical solutions in the above embodiments when running computer programs.

[0109] Memory 403 is used to store computer programs that can run on processor 402.

[0110] Specifically, the processor 402 is configured to generate one or more candidate output parameter groups for the first video stream based on video frames in the first video stream, as well as network status parameters and terminal status parameters; the candidate output parameter groups include bitrate, frame rate, and quantization parameters; The one or more candidate output parameter groups are evaluated to obtain a score for each candidate output parameter group in the one or more candidate output parameter groups; Based on the score of each candidate output parameter group, a video output parameter group is determined from the one or more candidate output parameter groups; Based on the video output parameter group, the first video stream is output.

[0111] In one embodiment, the processor 402 is specifically configured to perform semantic segmentation on the video frame to obtain a semantic segmentation result; the semantic segmentation result is used to indicate one or more semantic regions of the video frame; Based on the semantic segmentation results, the content criticality of each semantic region in the one or more semantic regions is calculated; Based on the content criticality of each semantic region, the network state parameters, and the terminal state parameters, one or more candidate output parameter groups are generated.

[0112] In one embodiment, the processor 402 is specifically configured to generate a three-dimensional weight matrix based on the content criticality of each semantic region, as well as the network state parameters and terminal state parameters; The bitrate of each semantic region is calculated based on the three-dimensional weight matrix, the total available bitrate of the video frame, and the area of ​​each semantic region. The frame rate of the first video stream is calculated based on the set frame rate upper limit, the terminal state parameters, and the motion speed of each semantic region; the motion speed of the semantic region is obtained by tracking the corresponding semantic region in the most recent few video frames. Based on the set basic quantization parameters, the set maximum offset of the quantization parameters, and the content criticality of each semantic region, the quantization parameters of each semantic region are calculated.

[0113] In one embodiment, the processor 402 is specifically configured to calculate multiple scoring items for each candidate output parameter group based on the candidate output parameter group, network status parameters, and terminal status parameters; the multiple scoring items include multiple items from text clarity, transmission delay, terminal overload risk, network congestion risk, and content mutation risk. The score of the candidate output parameter group is obtained based on the multiple scoring items and the set weight of each of the multiple scoring items.

[0114] In one embodiment, the processor 402 is specifically used for the following purposes: Based on this candidate output parameter group, as well as the bitrate-clarity mapping model, frame rate-clarity mapping model, and quantization parameter-clarity mapping model, the text clarity sub-item is calculated; Based on the candidate output parameter group, the network status parameters, and the terminal status parameters, the transmission delay component is calculated. Based on the terminal state parameters, the candidate output parameter set, and the coding complexity model, the terminal overload risk sub-item is calculated; the coding complexity model is used to predict the terminal resources occupied by the video output according to the input candidate output parameter set; Based on the code rate in the candidate output parameter group and the network state parameters, the network congestion risk component is calculated. Based on the semantic segmentation results of the video frames, the content mutation risk component is calculated.

[0115] In one embodiment, the processor 402 is specifically configured to calculate a priority score for each candidate output parameter group based on one or more of the following: the score of each candidate output parameter group, the maximum risk value of each candidate output parameter group, the change amount corresponding to each candidate output parameter group, and the historical stability score of each candidate output parameter group; the maximum risk value is the maximum value among the terminal overload risk item, the network congestion risk item, and the content mutation risk item. Select the candidate output parameter group corresponding to the highest priority score as the video output parameter group.

[0116] In one embodiment, the processor 402 is further configured to monitor and acquire network status parameters and terminal status parameters in real time during the output of the first video stream. Based on the real-time acquired network status parameters and the bitrate in the video output parameter group, the target bitrate adjustment value is calculated. Based on the target bitrate adjustment value and the current bitrate of the first video stream, the bitrate of the first video stream is adjusted using a progressive adjustment strategy. Based on the real-time acquired terminal status parameters and network status parameters, as well as the frame rate in the video output parameter group, the frame rate adjustment target value is calculated. The frame rate of the first video stream is adjusted to the frame rate adjustment target value.

[0117] In practical applications, the generation unit 301, the evaluation unit 302, the determination unit 303, the output unit 304, the acquisition unit, the first calculation unit, the first adjustment unit, the second calculation unit, and the second adjustment unit can be implemented by a processor in the video transmission device combined with a communication interface.

[0118] It should be noted that the specific processing procedure of processor 402 can be understood by referring to the above method. Of course, in practical applications, the various components in electronic device 400 are coupled together through bus system 404. It can be understood that bus system 404 is used to realize the connection and communication between these components. In addition to the data bus, bus system 404 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in... Figure 4 The general designated all buses as Bus System 404.

[0119] The memory 403 in this embodiment is used to store various types of data to support the operation of the electronic device 400. Examples of such data include any computer program used to operate on the electronic device 400.

[0120] The methods disclosed in the embodiments of this application can be applied to the processor 402, or implemented by the processor 402. The processor 402 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor 402 or by instructions in the form of software. The processor 402 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 402 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in the memory 403. The processor 402 reads the information in the memory 403 and completes the steps of the aforementioned method in combination with its hardware.

[0121] In an exemplary embodiment, the electronic device 400 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0122] It is understood that the memory (memory 403) in this embodiment of the application can be volatile memory or non-volatile memory, or it can include both volatile memory and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); the magnetic surface memory can be disk storage or magnetic tape storage. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.

[0123] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 403 storing a computer program, which can be executed by the processor 402 of the electronic device 400 to complete the steps described in any of the aforementioned methods. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0124] For example, embodiments of this application also provide a computer program product, including a computer program that can be executed by a processor 402 of an electronic device 400 to perform the steps described in any of the foregoing methods.

[0125] It should be noted that terms such as "first" and "second" are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. The term "and / or" in this document merely describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The terms "one or more" and "one or more" in this document are exemplary expressions and can be replaced with any possible expressions, such as one or more, at least one, or at least one. Furthermore, the technical solutions described in the embodiments of this application can be arbitrarily combined without conflict. The above descriptions are merely preferred embodiments of this application and are not intended to limit the scope of protection of this application.

Claims

1. A video transmission method, characterized in that, The method includes: Based on the video frames in the first video stream, as well as network status parameters and terminal status parameters, one or more candidate output parameter groups for the first video stream are generated; the candidate output parameter groups include bitrate, frame rate, and quantization parameters. The one or more candidate output parameter groups are evaluated to obtain a score for each candidate output parameter group in the one or more candidate output parameter groups; Based on the score of each candidate output parameter group, a video output parameter group is determined from the one or more candidate output parameter groups; Based on the video output parameter group, the first video stream is output.

2. The method according to claim 1, characterized in that, The step of generating one or more candidate output parameter groups for the first video stream based on video frames in the first video stream, as well as network state parameters and terminal state parameters, includes: The video frame is semantically segmented to obtain a semantic segmentation result; the semantic segmentation result is used to indicate one or more semantic regions of the video frame. Based on the semantic segmentation results, the content criticality of each semantic region in the one or more semantic regions is calculated; Based on the content criticality of each semantic region, the network state parameters, and the terminal state parameters, one or more candidate output parameter groups are generated.

3. The method according to claim 2, characterized in that, The generation of one or more candidate output parameter groups based on the content criticality of each semantic region, the network state parameters, and the terminal state parameters includes: A three-dimensional weight matrix is ​​generated based on the content criticality of each semantic region, as well as the network state parameters and terminal state parameters. The bitrate of each semantic region is calculated based on the three-dimensional weight matrix, the total available bitrate of the video frame, and the area of ​​each semantic region. The frame rate of the first video stream is calculated based on the set frame rate upper limit, the terminal state parameters, and the motion speed of each semantic region; the motion speed of the semantic region is obtained by tracking the corresponding semantic region in the most recent few video frames. Based on the set basic quantization parameters, the set maximum offset of the quantization parameters, and the content criticality of each semantic region, the quantization parameters of each semantic region are calculated.

4. The method according to any one of claims 1 to 3, characterized in that, The evaluation of the one or more candidate output parameter groups to obtain a score for each candidate output parameter group includes: For each candidate output parameter group, based on the candidate output parameter group, network status parameters, and terminal status parameters, multiple scoring items for the candidate output parameter group are calculated; the multiple scoring items include multiple items from text clarity, transmission delay, terminal overload risk, network congestion risk, and content mutation risk. The score of the candidate output parameter group is obtained based on the multiple scoring items and the set weight of each of the multiple scoring items.

5. The method according to claim 4, characterized in that, Based on the candidate output parameter group, network status parameters, and terminal status parameters, multiple scoring items for the candidate output parameter group are calculated, including the following: Based on this candidate output parameter group, as well as the bitrate-clarity mapping model, frame rate-clarity mapping model, and quantization parameter-clarity mapping model, the text clarity sub-item is calculated; Based on the candidate output parameter group, the network status parameters, and the terminal status parameters, the transmission delay component is calculated. Based on the terminal state parameters, the candidate output parameter group, and the coding complexity model, the terminal overload risk sub-item is calculated. The coding complexity model is used to predict the terminal resources occupied by the output video according to the input candidate output parameter set; Based on the code rate in the candidate output parameter group and the network state parameters, the network congestion risk component is calculated. Based on the semantic segmentation results of the video frames, the content mutation risk component is calculated.

6. The method according to any one of claims 1 to 3 and 5, characterized in that, The step of determining a video output parameter group from one or more candidate output parameter groups based on the score of each candidate output parameter group includes: Based on one or more of the following: the score of each candidate output parameter group, the maximum risk of each candidate output parameter group, the change amount corresponding to each candidate output parameter group, and the historical stability score of each candidate output parameter group, the priority score of each candidate output parameter group is calculated accordingly; the maximum risk is the maximum value among the terminal overload risk item, the network congestion risk item, and the content mutation risk item. Select the candidate output parameter group corresponding to the highest priority score as the video output parameter group.

7. The method according to any one of claims 1 to 3 and 5, characterized in that, The method further includes: During the output of the first video stream, network status parameters and terminal status parameters are monitored and acquired in real time. Based on the real-time acquired network status parameters and the bitrate in the video output parameter group, the target bitrate adjustment value is calculated. Based on the target bitrate adjustment value and the current bitrate of the first video stream, the bitrate of the first video stream is adjusted using a progressive adjustment strategy. Based on the real-time acquired terminal status parameters and network status parameters, as well as the frame rate in the video output parameter group, a frame rate adjustment target value is calculated, and the frame rate of the first video stream is adjusted to the frame rate adjustment target value.

8. An electronic device, characterized in that, The method includes a processor and a memory for storing a computer program capable of running on the processor, wherein, when the processor is used to run the computer program, it performs the steps of the method according to any one of claims 1 to 7.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images