A collaborative transcoding method and related equipment for large video on mobile terminals
By generating video structure description files and combining deep reinforcement learning and planning algorithms to optimize task allocation, the energy consumption and latency issues in large video processing on mobile terminals are solved, achieving efficient resource scheduling for terminal devices and smooth playback of streaming media.
Patent Information
- Application Number
- CN202611130355.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-28
- Publication Date
- 2026-11-17
AI Technical Summary
When mobile devices process large videos, their computing power and power consumption are limited, leading to prolonged high-load operation, which causes the device to overheat and the battery to drain rapidly. Meanwhile, cloud processing mode can cause significant bandwidth pressure and latency when uploading large video files in a mobile network environment.
By extracting video structure description files and combining terminal operating status and network parameters, an end-to-cloud collaboration strategy is generated. Transcoding tasks are adaptively allocated, and cloud offloading is dynamically triggered when the hardware status exceeds the threshold. Deep reinforcement learning networks and planning algorithms are used to optimize task allocation to minimize energy consumption and latency.
It improves the resource scheduling flexibility and processing efficiency of terminal devices, reduces device overheating and power consumption, reduces network latency and bandwidth usage, and ensures the continuity and smoothness of streaming media playback.
Smart Images

Figure CN123123578A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of video processing and streaming media transmission technology, and in particular to a collaborative transcoding method and related equipment for large video streams on mobile terminals. Background Technology
[0002] With the rapid popularization of mobile internet and smart devices, users are increasingly using mobile devices to shoot and share high-definition videos. To ensure a smooth viewing experience for users on multiple devices, the original video usually needs to be transcoded and segmented, converted into a streaming media format adapted to different network environments before being published.
[0003] In related technologies, video transcoding and streaming media publishing solutions typically employ two processing modes: the first relies entirely on the mobile terminal to compress, transcode, and slice the video locally, and then uploads the processed streaming media file to the server for publishing; the second uploads the original large video file directly and completely to the cloud, where the cloud server centrally performs subsequent transcoding and streaming media encapsulation operations.
[0004] However, for the first local processing mode, high-definition video data is massive and has complex spatiotemporal textures, while mobile terminals have relatively limited computing power and battery capacity. Prolonged local intensive transcoding calculations cause the terminal's CPU to remain under high load for extended periods, leading to device overheating and rapid battery consumption, impacting the terminal's concurrent performance. For the second cloud processing mode, the original large video files are enormous, and uploading them completely in a mobile network environment creates significant bandwidth pressure. Especially when network transmission rates are limited or fluctuating, the massive data transmission can result in extremely long delays for users waiting for the first playback. Summary of the Invention
[0005] This application provides a collaborative transcoding method and related equipment for large videos on mobile terminals, which can improve the efficiency of large video processing on mobile terminals.
[0006] Firstly, this application provides a collaborative transcoding and streaming media publishing method for large videos on mobile terminals. The method includes: extracting keyframe positions and segmentation complexity indicators from the source video to be shared, generating a structural description file for the source video; inputting the file size of the source video, hardware operating parameters of the mobile terminal, network transmission rate parameters, and the structural description file into a preset collaborative processing model, outputting an end-to-cloud collaborative strategy, which includes a first-play priority interval, a terminal processing task set, and a cloud processing task set; extracting video data corresponding to the first-play priority interval and the terminal processing task set, and performing compression encoding and segmentation on the extracted video data. The process involves segment encapsulation to generate initial streaming media segment files and initial streaming media index files; sending the initial streaming media segment files, initial streaming media index files, structure description files, cloud processing task sets, and the remaining video data corresponding to the cloud processing task sets in the source video to the cloud server; during the compression encoding and segment encapsulation operations, if the hardware operating status parameters are detected to exceed the preset hardware threshold, the remaining unexecuted tasks in the terminal processing task set and the unprocessed video data corresponding to the remaining tasks are sent to the cloud server; and receiving the streaming media playback link returned by the cloud server, which is used to instruct the preset playback terminal to play the source video.
[0007] In the above embodiments, the system extracts video structure description files and generates end-to-cloud collaboration strategies by combining terminal operating status and network parameters. This adaptively allocates video transcoding and slicing tasks between the mobile terminal and the cloud server. During execution, it dynamically triggers cloud offloading of unexecuted tasks by monitoring hardware operating status parameters, preventing device overheating or performance degradation due to prolonged high-load operation. Simultaneously, prioritizing local processing within the first-view priority interval ensures initial availability of streaming media, improving the resource scheduling flexibility and overall processing efficiency of the mobile terminal when processing large videos.
[0008] In conjunction with some embodiments of the first aspect, in some embodiments, the source video file size, mobile terminal hardware operating status parameters, network transmission rate parameters, and structural description file are input into a preset collaborative processing model to output an end-to-cloud collaborative strategy. Specifically, this includes: concatenating features from the source video file size, CPU utilization and remaining battery power in the hardware operating status parameters, network transmission rate parameters, and segmentation complexity indicators in the structural description file to construct a multi-dimensional state vector; inputting the multi-dimensional state vector into a pre-trained deep reinforcement learning network, and outputting an initial task allocation ratio through the policy network in the deep reinforcement learning network; based on the initial task allocation ratio and segmentation complexity indicators, using a preset planning algorithm to search for the optimal segmentation point on the time axis of the source video, with the optimization objective of minimizing the mobile terminal's energy consumption and overall processing latency, to generate an end-to-cloud collaborative strategy. The first-play priority interval is the video segment before the optimal segmentation point that meets the preset first-play buffer duration.
[0009] In the above embodiments, the system introduces a collaborative processing model that combines deep reinforcement learning networks with planning algorithms. The multi-dimensional state vector is input into the policy network to output the initial task allocation ratio, and the optimal split point is searched on the time axis based on the segmented complexity index. The original static task division process is transformed into a dynamic optimization process with the goal of minimizing energy consumption and overall latency. This allows the definition of the first-play priority interval to accurately match the inherent structure of the video content and the current device's carrying capacity, thereby improving the adaptability of the end-cloud collaborative strategy and the rationality of task allocation in complex and ever-changing operating environments.
[0010] In conjunction with some embodiments of the first aspect, in some embodiments, the optimization objective of minimizing the power consumption and overall processing latency of the mobile terminal is used to generate an end-to-cloud collaborative strategy. Specifically, this includes: constructing a joint optimization objective function, which includes a terminal power consumption penalty term, an overall processing latency penalty term, and a first-broadcast latency penalty term; adjusting the weight coefficient of the terminal power consumption penalty term based on the remaining battery power, wherein when the remaining battery power is lower than a preset battery power threshold, the weight coefficient of the terminal power consumption penalty term is adjusted to a first preset weight; adjusting the weight coefficient of the overall processing latency penalty term based on network transmission rate parameters, wherein when the network transmission rate parameter is lower than a preset rate threshold, the cloud processing task allocation ratio in the initial task allocation ratio is adjusted to a preset ratio, and the weight coefficient of the overall processing latency penalty term is adjusted to a second preset weight; substituting the initial task allocation ratio into the joint optimization objective function for iterative solution, and when the joint optimization objective function converges, using the solved task allocation result as the terminal processing task set and the cloud processing task set.
[0011] In the above embodiments, the system constructs a joint optimization objective function that includes terminal power consumption, overall processing latency, and first-broadcast latency penalties. For the actual operating conditions of limited mobile terminal power and network fluctuations, the system dynamically adjusts the weight coefficients of each penalty term based on the remaining power and network transmission rate parameters. When the power or network rate falls below a preset threshold, the system adaptively adjusts the weights and cloud processing ratio, guiding the iterative solution process towards reducing local power consumption or avoiding network congestion. This enhances the task scheduling mechanism's resilience and operational stability under extreme hardware or network conditions.
[0012] In conjunction with some embodiments of the first aspect, in some embodiments, the extraction of keyframe positions and segmentation complexity indices from the source video to be shared, and the generation of a structural description file for the source video, specifically includes: performing pre-decoding analysis on the source video to extract the position information of all I-frames in the source video as keyframe positions; calculating the spatial texture feature values and temporal motion vector feature values of video segments between adjacent I-frames in the source video; weightedly fusing the spatial texture feature values and temporal motion vector feature values to obtain the segmentation complexity index of the video segment; and encapsulating the keyframe positions, segmentation complexity index, and timestamps corresponding to the video segments according to a preset data exchange format to generate a structural description file.
[0013] In the above embodiments, the system performs pre-decoding analysis on the source video, weightedly fusing the spatial texture feature values and temporal motion vector feature values of video segments between adjacent I-frames, quantifying the encoding difficulty of the video content from both spatiotemporal dimensions. This mechanism encapsulates the extracted keyframe positions and segmentation complexity indicators into a structural description file, providing underlying data support for edge-cloud task allocation and cloud transcoding parameter configuration, improving the objectivity of the video structured description and the targeted nature of subsequent collaborative processing.
[0014] In some embodiments, in conjunction with the first aspect, the method includes: receiving an initial streaming media segment file, an initial streaming media index file, a source video structure description file, a cloud processing task set, and remaining video data in the source video sent by a mobile terminal; parsing the cloud processing task set and the structure description file to determine the target encoding parameters and slice duration corresponding to the remaining video data; performing compression encoding and slice encapsulation operations on the remaining video data using the target encoding parameters and slice duration to generate subsequent streaming media segment files; extracting the access paths of the subsequent streaming media segment files and concatenating the access paths to the initial streaming media index file in the timeline order of the source video to generate a complete streaming media index file; receiving the remaining tasks and the unprocessed video data corresponding to the remaining tasks sent by the mobile terminal, performing compression encoding and slice encapsulation operations on the unprocessed video data using the target encoding parameters and slice duration to generate new streaming media segment files, and supplementing the access paths of the new streaming media segment files to the complete streaming media index file; generating a streaming media playback link based on the complete streaming media index file, and sending the streaming media playback link to the mobile terminal for the mobile terminal to share or publish the source video.
[0015] In the above embodiments, the system receives and parses the structure description file and processing task set sent by the mobile terminal through the cloud server, takes over the remaining video data and unprocessed video data dynamically unloaded by the terminal, and performs compression encoding and segment encapsulation using target encoding parameters and segment duration. By extracting the access paths of subsequent and newly added segment files and concatenating them to the initial streaming media index file in chronological order, it ensures that streaming media fragments generated by asynchronous processing at both the end and cloud ends can be reassembled into a logically continuous and complete index, thus guaranteeing the integrity of streaming media content and the continuity of playback in distributed transcoding scenarios.
[0016] In conjunction with some embodiments of the first aspect, in some embodiments, the process of parsing the cloud processing task set and the structure description file to determine the target encoding parameters and slice duration corresponding to the remaining video data specifically includes: extracting the segmentation complexity index and keyframe position of each video segment corresponding to the remaining video data from the structure description file; matching the segmentation complexity index with a preset complexity threshold range to assign corresponding target quantization parameters and target bitrate to each video segment to obtain the target encoding parameters; calculating the time interval between adjacent keyframes based on the keyframe position, setting the slice duration to a preset integer multiple of the time interval, and ensuring that the starting frame of each video slice generated by the slice encapsulation operation is a keyframe.
[0017] In the above embodiments, the system dynamically allocates target encoding parameters and slice durations to each video segment based on the segmentation complexity index and keyframe positions in the structure description file. By matching the segmentation complexity index with a preset threshold range to allocate target quantization parameters and target bitrates, encoding resources are guided to favor complex scenes. Simultaneously, setting the slice duration to an integer multiple of the time interval between adjacent keyframes ensures that the starting frame of each video slice is a keyframe, preventing decoding dependency breaks caused by the non-coincidence of slice boundaries and image group boundaries, thus improving the encoding efficiency of cloud transcoding and the independent decoding capability of streaming media slices.
[0018] In conjunction with some embodiments of the first aspect, in some embodiments, after performing compression encoding and slice encapsulation operations on the remaining video data using target encoding parameters and slice duration to generate subsequent streaming media segment files, the method further includes: monitoring the real-time downlink bandwidth of the network when the preset playback terminal requests the source video through the streaming media playback link; when the real-time downlink bandwidth is lower than the target bitrate in the target encoding parameters for a preset period of time, calculating the downgraded bitrate based on the real-time downlink bandwidth; transcoding the unaccessed segments in the subsequent streaming media segment files using the downgraded bitrate to generate downgraded streaming media segment files; and updating the access path of the downgraded streaming media segment files to the complete streaming media index file so that the playback terminal can perform adaptive bitrate switching.
[0019] In the above embodiments, the system monitors the real-time downlink network bandwidth when the playback client requests the source video. When the network bandwidth remains below the target bitrate, it uses the calculated downgraded bitrate to perform forward transcoding on unaccessed segments. This mechanism generates downgraded streaming media segment files and synchronously updates the complete streaming media index file, transforming the network fluctuation response strategy from passive buffering and waiting to proactive bitrate adaptive downgrading. This provides the playback client with a path to switch to low bitrate segments, improving the smoothness of streaming media playback in weak network environments.
[0020] Secondly, embodiments of this application provide a collaborative transcoding device for large video on mobile terminals. The collaborative transcoding device for large video on mobile terminals includes: one or more processors and a memory; the memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the collaborative transcoding device for large video on mobile terminals to perform the method described in the first aspect and any possible implementation thereof.
[0021] Thirdly, embodiments of this application provide a computer program product containing instructions that, when the computer program product is run on a collaborative transcoding device for large video on a mobile terminal, causes the collaborative transcoding device for large video on a mobile terminal to perform the method described in the first aspect and any possible implementation thereof.
[0022] Fourthly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a collaborative transcoding device for large video on a mobile terminal, cause the collaborative transcoding device for large video on a mobile terminal to perform the method described in the first aspect and any possible implementation thereof.
[0023] Understandably, the collaborative transcoding device for large video on mobile terminals provided in the second aspect, the computer program product provided in the third aspect, and the computer storage medium provided in the fourth aspect are all used to execute the methods provided in the embodiments of this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods, and will not be repeated here.
[0024] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:
[0025] 1. This application generates an end-to-cloud collaboration strategy by combining terminal operating status and network parameters, adaptively allocates transcoding tasks, and dynamically monitors hardware status to trigger cloud offloading of unexecuted tasks, thereby improving the problem of device overheating and rapid power consumption caused by long-term high-load operation of the terminal.
[0026] 2. This application improves the problems of extremely high uplink bandwidth consumption, network latency, and transmission interruption caused by uploading large original videos by extracting keyframes and complexity to generate structure description files and combining reinforcement learning and planning algorithms.
[0027] 3. This application takes over the remaining video and dynamically unloads tasks through the cloud, and monitors real-time network bandwidth. In weak network conditions, it uses a downgraded bitrate to perform forward transcoding of unaccessed segments, actively achieving adaptive bitrate degradation, thus ensuring the smooth and continuous streaming media publishing and multi-terminal playback in complex network environments. Attached Figure Description
[0028] Figure 1 This is a flowchart illustrating a collaborative transcoding method for large video on mobile terminals in an embodiment of this application.
[0029] Figure 2 This is a schematic diagram of a physical device structure of a collaborative transcoding device for large video on mobile terminals, as described in this application. Detailed Implementation
[0030] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification of this application, the singular expressions “a,” “an,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to any or all possible combinations including one or more of the listed items.
[0031] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.
[0032] To facilitate understanding, the method provided in this implementation will be described in detail below, using the above scenario as an example. Please refer to [link / reference]. Figure 1 This is a flowchart illustrating a collaborative transcoding method for large video on mobile terminals in an embodiment of this application.
[0033] S101. Extract the keyframe positions and segmentation complexity indicators of the source video to be shared, and generate a structural description file of the source video.
[0034] Among them, the source video refers to the original video file to be shared that is stored locally on the mobile terminal; the keyframe position refers to the timestamp and byte offset position of all I-frames (i.e., intra-coded frames) on the time axis in the source video bitstream, expressed in millisecond precision or frame sequence number; the segmentation complexity index refers to the quantitative value that measures the encoding difficulty of video segments between two adjacent I-frames; the structure description file refers to the metadata file that uniformly encapsulates the keyframe position, segmentation complexity index and timestamps of each video segment. It can be stored in a preset data exchange format such as JSON, XML or binary key-value table, and can further carry auxiliary fields such as total video duration, resolution, orientation information, audio and video track identifiers, to provide underlying data basis for subsequent end-to-cloud task allocation and cloud encoding parameter configuration.
[0035] After a video sharing is initiated on a mobile terminal, the system first executes this step to obtain the inherent structural information of the source video in advance with low overhead. Specifically, this step performs lightweight pre-decoding analysis on the source video, parsing only the encoded structure information in the bitstream header without performing pixel-level reconstruction, ensuring the accuracy of I-frame location extraction while keeping the analysis latency within an acceptable range. For video segments between each adjacent I-frame, spatial texture feature values and temporal motion vector feature values are calculated separately, and then combined using weighted fusion to form a segmented complexity index.
[0036] The use of spatiotemporal dual-dimensional fusion, rather than a single feature, is to address the fact that relying solely on spatial texture might overestimate the complexity of slow-motion scenes, while relying solely on temporal motion might underestimate the encoding cost of highly detailed static scenes. Dual-dimensional fusion can more objectively reflect the actual transcoding difficulty of each video segment, enabling subsequent collaborative processing models to achieve fine-grained task allocation at the video content structure level. Once generated, the structure description file can be shared and reused on both the edge and cloud sides. Subsequent optimal segmentation point search, target encoding parameter allocation, and slice duration setting all use this file as the underlying data basis, eliminating the need for the cloud to re-analyze the source video and reducing redundant computational overhead.
[0037] In some preferred embodiments, the above steps specifically include: performing pre-decoding analysis on the source video, extracting the position information of all I-frames in the source video as keyframe positions; and calculating the spatial texture feature values and temporal motion vector feature values of video segments between adjacent I-frames in the source video.
[0038] The spatial texture feature value and the temporal motion vector feature value are weighted and fused to obtain the segmentation complexity index of the video segment; the key frame position, segmentation complexity index and the timestamp corresponding to the video segment are encapsulated according to the preset data exchange format to generate a structure description file.
[0039] Among them, pre-decoding analysis refers to a lightweight preprocessing operation that only performs frame type detection and frame header metadata parsing on the video bitstream without performing full pixel reconstruction; spatial texture feature values refer to the numerical values obtained by quantifying the static complexity of video frames in the spatial domain, which can be characterized by indicators such as DCT coefficient variance, edge gradient intensity, or intra-frame residual brightness; temporal motion vector feature values refer to the numerical values obtained by quantifying the intensity of motion between adjacent frames, which can be characterized by indicators such as inter-frame residual energy, average motion vector magnitude, or motion vector field variance; the preset data exchange format refers to the serialization and encapsulation specification of the structure description file, which can be JSON, XML, or a custom binary structured table to achieve a balance between readability and parsing efficiency.
[0040] This step outlines the specific implementation path for generating the structure description file, with each sub-operation logically interconnected. Specifically, the system locates frame type boundaries in the video stream through pre-decoding scanning, extracts the timestamp offsets of all I-frames as keyframe positions, divides the video into segments using adjacent I-frames as boundaries, calculates spatial texture feature values and temporal motion vector feature values respectively, and obtains the segmentation complexity index through weighted fusion.
[0041] The core of adopting spatiotemporal dual-dimensional fusion is that for action videos with intense motion but relatively flat image texture, relying solely on spatial texture scoring will significantly underestimate their encoding cost; conversely, for static scenes containing large areas of high-frequency texture, relying solely on inter-frame differences cannot accurately reflect their compression difficulty.
[0042] The weighting coefficients in weighted fusion can be configured with fixed preset values or determined through offline sample statistical training to adapt to the distribution characteristics of different types of video content; a typical example of configuring fixed preset values is: weighting spatial texture feature values. The weights of the time motion vector eigenvalues are set to 0.4. Set to 0.6, which is the piecewise complexity index. This ratio emphasizes the motion dimension and is suitable for short video scenarios that primarily feature dynamic content; for video content that mainly consists of indoor static shooting or high-detail textures, it can be appropriately adjusted. The weights were adjusted to 0.5 to 0.6 to accurately reflect the encoding cost; the weight coefficients could also be determined by offline regression training on sample sets of different video types to obtain more accurate content-adaptive segmentation complexity quantification results.
[0043] It should be noted that the calculation of the segmentation complexity index is not limited to the path described in this preferred embodiment, and can also be achieved based on alternative features such as scene switching frequency, bitrate fluctuation amplitude, or inter-frame residual intensity. Finally, the keyframe position, segmentation complexity index, and timestamps of each segment are packaged into a structured description file according to a preset data exchange format, providing a unified structured data foundation for subsequent end-to-cloud task allocation and cloud-based encoding parameter configuration.
[0044] S102. Input the source video file size, mobile terminal hardware operating status parameters, network transmission rate parameters, and structural description file into the preset collaborative processing model, and output the end-cloud collaborative strategy.
[0045] Among them, hardware operation status parameters refer to the real-time resource usage and device status information of the mobile terminal at the time of task startup, including at least CPU utilization and remaining power, and can also be expanded to collect dimensions such as device temperature, memory utilization, whether it is connected to a charging power source, and whether the application is running in the foreground, so as to more comprehensively reflect the real-time carrying capacity of the terminal; network transmission rate parameters refer to the estimated value of currently available uplink bandwidth, which is used to quantify the time cost of data transmission from the terminal to the cloud server.
[0046] The collaborative processing model refers to a pre-defined computing model that takes the aforementioned multi-dimensional parameters and structural description file as input, infers through a built-in three-layer decision-making mechanism, and outputs a cloud-edge collaborative strategy. Its internal structure consists of three functional modules connected sequentially: a feature construction layer, a strategy inference layer, and a segmentation optimization layer. The feature construction layer is responsible for normalizing or standardizing the source video file size, CPU utilization, remaining battery power, network transmission rate parameters, and the segmentation complexity indicators of each video segment, and then concatenating them in a pre-defined order into a fixed-length multi-dimensional state vector to eliminate the interference of differences in the dimensions of each field on the stability of subsequent inference. The strategy inference layer takes the aforementioned multi-dimensional state vector as input. The system employs a pre-trained deep reinforcement learning network to infer and output the initial task allocation ratio, enabling rapid global localization of task allocation directions between the terminal and the cloud. This layer can also be implemented using a rule engine, dynamic programming, or heuristic search algorithm to adapt to terminal scenarios with limited computing power. The segmentation optimization layer uses the coarse search range determined by the initial task allocation ratio as a constraint, and combines the segmentation complexity index of each video segment in the structure description file to accurately search for the optimal segmentation point on the source video timeline through a planning algorithm. Finally, it outputs a complete end-to-cloud collaborative strategy that includes the first-play priority interval, the terminal processing task set, and the cloud processing task set.
[0047] The end-to-cloud collaboration strategy is the output of the model, which explicitly includes three core components: the first-play priority interval (the video time period that prioritizes initial playability), the terminal processing task set (the set of video segments that are transcoded and sliced locally on the terminal), and the cloud processing task set (the set of video segments that are offloaded to the cloud server for processing). It may also include auxiliary decision-making information such as migration trigger thresholds or candidate segmentation point scores.
[0048] This step is the core decision-making stage of the entire collaborative processing flow, executed before the structure description file is generated and local encoding begins. Specifically, the collaborative processing model comprehensively analyzes the terminal's current remaining computing resources (CPU utilization and remaining battery power), network transmission efficiency (uplink bandwidth), the amount of data to be processed (file size), and the content difficulty of each video segment (segment complexity index), dynamically determining the task boundaries between the terminal and the cloud through an internal decision-making mechanism. Compared to fixed-ratio division, completely local processing, or uploading the original large video entirely to the cloud, this model can adaptively adjust task division based on the real-time status of the terminal and the network: when terminal resources are abundant and the network is unobstructed, it tends to expand the local processing range to reduce upload overhead; when the terminal is under high load or the network is limited, it prioritizes offloading high-complexity video segments to the cloud, thereby achieving a dynamic balance between terminal energy consumption, overall processing latency, and first-playback waiting time. The explicit output of the first-playback priority interval in the end-to-cloud collaborative strategy ensures that local encoding can complete the generation of the initial playable segments first, allowing the playback terminal to begin loading even before the video processing in the cloud is complete.
[0049] Before being deployed to mobile terminals, the above collaborative processing model optimizes the policy network parameters through the following offline training process.
[0050] At the training data construction level, complete trajectory samples containing terminal status, source video attributes, and task execution results are collected from historical transcoding task logs or simulation environments. Each trajectory record includes the terminal status at the time of task initiation (CPU utilization u_cpu, remaining battery power u_bat, uplink bandwidth u_bw, normalized source video file size f_size_norm, and segment complexity sequence of each video segment), the task allocation ratio α used, and the measured terminal energy consumption E_meas, overall processing latency D_meas, and first-playback latency D_fp_meas after task completion. After normalizing the three types of measured values to their respective reference maximum values, the results are calculated according to the reward function r = −(w_e · E_norm + w_d · D_total_norm + w_f · ) The sample reward is calculated using D_fp_norm, where the initial weights are w_e=0.4, w_d=0.4, and w_f=0.2. At least 10,000 combined trajectory samples covering low, medium, and high device load scenarios and slow motion, motion, and scene switching intensive video content are collected to ensure that the policy network has sufficient generalization ability under different state distributions.
[0051] Regarding the state vector encoding rules, the source video is divided into N video segments, and the actual value of N varies depending on the video duration and I-frame density. To uniformly map the variable-length segmented complexity sequence to a fixed-length representation, the system presets a maximum video segment limit of N_max=50: when N ≤ 50, the segmented complexity sequence is padded with N_max − N zero values for zero-padding; when N > 50, the segmented complexity sequence is sampled at equal intervals to reduce to 50 representative values; then u_cpu, u_bat, u_bw_norm (uplink bandwidth normalized to a 10Mbps reference value), and f_size_norm are sequentially concatenated before the complexity sequence to form a 54-dimensional multidimensional state vector s ∈ R^54, and each feature field is normalized to the [0, 1] interval to eliminate dimensional differences.
[0052] Regarding the network architecture, the policy network (Actor) adopts a three-layer fully connected structure. The input layer has a dimension of 54, the first hidden layer contains 256 neurons (with ReLU activation function), the second hidden layer contains 128 neurons (with ReLU activation function), and the output layer contains 1 neuron activated by Sigmoid, constraining the output value to the (0, 1) interval, i.e., the task allocation ratio α. The value network (Critic) shares the input layer and first hidden layer parameters with the policy network. The second hidden layer is independently set (128 neurons, ReLU activation), and the output layer contains 1 neuron (linear activation), outputting the value estimate V(s) of the current state, which is used to calculate the advantage function to stabilize the policy gradient estimate.
[0053] The training process employs the Proximal Policy Optimization (PPO) algorithm. The optimizer is Adam, with a learning rate of 3 × 10^-4. The PPO pruning coefficient ε is set to 0.2, the value function loss coefficient to 0.5, and the entropy regularization coefficient to 0.01 to prevent the policy network from prematurely converging to a local optimum. Each policy update randomly selects a small batch (batch size = 64) from the trajectory samples of the current round for gradient updates, with a maximum of 10 policy updates per round. When the policy loss is below 0.01 for 100 consecutive training rounds, the training is considered converged, and the Actor and Critic network parameters at that moment are saved as the final deployment weights.
[0054] After training, export the policy network parameters to TensorFlow Lite or ONNX format, compress them, and load them into the mobile terminal in read-only mode. Perform a single forward inference at the start of each video sharing task, and keep the inference latency within 50 milliseconds to avoid significantly affecting the overall task response time.
[0055] In some preferred embodiments, the above steps specifically include: concatenating the file size of the source video, the CPU utilization and remaining power in the hardware operating status parameters, the network transmission rate parameters, and the segmentation complexity index in the structure description file to construct a multi-dimensional state vector;
[0056] The multidimensional state vector is input into a pre-trained deep reinforcement learning network, and the initial task allocation ratio is output through the policy network in the deep reinforcement learning network.
[0057] Based on the initial task allocation ratio and segmentation complexity index, the optimal segmentation point is searched on the timeline of the source video using a preset planning algorithm. With the optimization goal of minimizing the energy consumption of the mobile terminal and the overall processing latency, an end-to-cloud collaboration strategy is generated. The first broadcast priority interval is the video segment before the optimal segmentation point that meets the preset first broadcast buffer time.
[0058] Among them, the multidimensional state vector refers to a fixed-length numerical vector formed by concatenating multiple heterogeneous input features in a specific order after normalization or standardization (for example, mapping CPU utilization, normalized file size, normalized uplink bandwidth, and segment complexity score sequences to a unified dimensional range and then concatenating them sequentially). It serves as the observation input for the deep reinforcement learning network. Normalization can eliminate the impact of dimensional differences between fields on the stability of network inference. The policy network refers to the network component in the deep reinforcement learning network responsible for outputting action decisions based on the current state. Its output initial task allocation ratio represents the initial estimated proportion of the terminal's local processing tasks to the total task volume. The planning algorithm refers to the optimization program that determines the search range based on the initial task allocation ratio and finds the optimal split point on the time axis by combining the segment complexity index. It can be implemented using dynamic programming or heuristic search.
[0059] The preset first-playback buffer duration refers to the minimum playable duration required for the playback end to complete the initial buffering. Its value can be preset according to the playback scenario, with a typical range of 3 to 10 seconds: For scenarios that emphasize quick availability, such as instant messaging or social media posting, it can be set to 3 to 5 seconds to prioritize shortening the perceived first-playback waiting time for users; For scenarios that require high smoothness, such as long video content management or professional publishing platforms, it can be set to 8 to 10 seconds to achieve a balance between first-playback response speed and buffer stability; Alternatively, it can be equivalently defined by the minimum number of playable segments (such as 2 to 3 segments) or the first segment playback duration to adapt to the first-playback guarantee logic under different segment duration configurations.
[0060] This step decomposes the collaborative processing model into a two-stage decision-making process: "coarse-grained ratio estimation – fine-grained segmentation point search." This two-stage collaboration avoids the excessive computational cost caused by directly exhaustively searching the high-dimensional task allocation space. Specifically, the system normalizes each input feature and concatenates them into a multi-dimensional state vector, which is then input into a pre-trained deep reinforcement learning network. The policy network outputs the initial task allocation ratio, completing the rapid localization of the global task allocation direction. Subsequently, a coarse search range for the optimal segmentation point is defined based on this ratio. Combining the segmentation complexity index of each video segment, a planning algorithm is used to perform a local fine-grained solution on the time axis with the goal of minimizing terminal power consumption and overall processing latency, thus determining the optimal segmentation point.
[0061] Once the optimal segmentation point is determined, the system selects video segments that meet the preset first-play buffer duration before it as the first-play priority interval, ensuring that local encoding can complete the generation of initial playable content in the shortest possible time. The planning algorithm can also simultaneously output multiple candidate segmentation points and their optimization target scores during the search process, reserving flexibility for the system to dynamically adjust task boundaries due to sudden changes in hardware status in subsequent execution phases.
[0062] It should be noted that, in addition to deep reinforcement learning networks, collaborative processing models can also be implemented using alternative methods such as rule engines, heuristic algorithms, or dynamic programming to adapt to the computing resource conditions of different terminals.
[0063] In other embodiments of this application, the above-mentioned edge-cloud collaboration strategy, which aims to minimize the power consumption and overall processing latency of the mobile terminal, specifically includes:
[0064] A joint optimization objective function is constructed, which includes a terminal energy consumption penalty term, an overall processing delay penalty term, and a first broadcast delay penalty term. The weight coefficient of the terminal energy consumption penalty term is adjusted according to the remaining power. When the remaining power is lower than the preset power threshold, the weight coefficient of the terminal energy consumption penalty term is adjusted to the first preset weight.
[0065] The weight coefficient of the overall processing latency penalty term is adjusted according to the network transmission rate parameter. When the network transmission rate parameter is lower than the preset rate threshold, the cloud processing task allocation ratio in the initial task allocation ratio is adjusted to the preset ratio, and the weight coefficient of the overall processing latency penalty term is adjusted to the second preset weight. The initial task allocation ratio is substituted into the joint optimization objective function for iterative solution. When the joint optimization objective function converges, the task allocation result obtained is used as the terminal processing task set and the cloud processing task set.
[0066] The joint optimization objective function refers to the comprehensive cost function that integrates multiple competing optimization objectives into a single solvable scalar through a weighted summation. Its basic form can be expressed as:
[0067]
[0068] in , , These are the weighting coefficients for the three types of penalty items; terminal energy consumption penalty item. This refers to the penalty component that quantifies the power consumption generated by local processing tasks. Its value can be modeled as a function of CPU utilization, local encoding duration, and power consumption rate; overall processing latency penalty term. This refers to the total latency from task initiation to the completion of all segment generation, which can be broken down into upload latency (determined by the remaining video data volume and uplink bandwidth), cloud queuing and transcoding waiting latency, and the sum of local remaining transcoding latency; first-run delay penalty item. It refers to the penalty component of the waiting time between the start of the quantification playback terminal and the readiness of the first playable segment, reflecting the initial playback response speed that the user can perceive; the first preset weight and the second preset weight are the aggravation adjustment coefficients assigned to the corresponding penalty items when specific conditions are triggered, which can be determined through offline test sample statistics or preset according to device model and application scenario.
[0069] Specifically, after constructing the joint optimization objective function, the system conditionally and dynamically adjusts the weight coefficients according to the current terminal status: the typical value of the preset power threshold is 20%. When the remaining power is lower than this threshold, α is adjusted to the first preset weight (e.g., 0.7, significantly higher than the default value of 0.4), driving the optimization solution to converge towards reducing the local encoding workload, effectively limiting the upper limit of the terminal processing task set; the typical value of the preset rate threshold is 1Mbps to 2Mbps (preset according to the minimum available bandwidth requirements of the target resolution and encoding format). When the network transmission rate parameter is lower than this threshold, the system first forcibly adjusts the cloud processing ratio in the initial task allocation ratio to the preset ratio (typically 30%, i.e., limiting the upload volume to no more than 30% of the total task volume, preventing excessive data offloading to the cloud under weak network conditions from aggravating transmission congestion), and at the same time adjusts β to the second preset weight (e.g., 0.65), increasing the proportion of the overall latency cost in the objective function, guiding the solution to avoid concentrating too much transmission burden on the currently limited network link.
[0070] After the adjustment is completed, the initial task allocation ratio is substituted into the joint optimization objective function for iterative solution. The task allocation result when the function converges is used as the final terminal processing task set and cloud processing task set.
[0071] S103. Extract the video data corresponding to the first-broadcast priority interval and the terminal processing task set, perform compression encoding and slicing encapsulation operations on the extracted video data, and generate the initial streaming media segment file and the initial streaming media index file.
[0072] The "first-play priority interval" refers to the video time segment that needs to be prioritized in the local encoding process, as defined in the end-to-cloud collaboration strategy. This interval includes video content before the optimal segmentation point that meets the preset first-play buffer duration, ensuring that the playback device can obtain the initial playable segments in the shortest possible time. The "terminal processing task set" refers to the set of all video segments whose compression encoding and segmentation are completed locally by the mobile terminal, as defined in the end-to-cloud collaboration strategy. The first-play priority interval is a subset of the terminal processing task set, which may also include other terminal-processed video segments after the first-play priority interval but before the optimal segmentation point. The "initial streaming media segment file" refers to a set of discrete media segments obtained after performing compression encoding and segmentation on the extracted video data. The "initial streaming media index file" refers to a metadata file that records the mapping relationship between the access paths of each initial segment (such as M3U8 entries in HLS format or MPD fragment descriptions in MPEG-DASH format) and the timeline order. Its format is compatible with the subsequent complete streaming media index file. Subsequent segment paths generated in the cloud will be appended to this file sequentially along the timeline, rather than constructing a separate independent index.
[0073] This step is the local encoding execution phase that the mobile terminal immediately initiates after the collaborative strategy is determined. The "video data corresponding to the first-play priority interval and the terminal processing task set" constitute the complete video data range for local processing by the terminal. These are not two independent sources, but rather an inclusion relationship between the task set and its internal priority subset. Specifically, after extracting the above video data, the system arranges the encoding queue according to the "first-play priority" principle: compression encoding and segmentation encapsulation are performed on the video segments within the first-play priority interval first, ensuring that the playback terminal can obtain the initial playable segments before the complete video has been fully transcoded, thus reducing the user-perceived first-play waiting latency; thereafter, the remaining video segments in the terminal processing task set are processed to generate all initial streaming media segment files.
[0074] After encoding, the system extracts the access paths of each segment and writes them into the initial streaming media index file in chronological order. This index can initially exist as a minimum playable version (containing only access entries for generated segments), and subsequent segment paths are gradually added as cloud processing progresses, eventually evolving into a complete streaming media index file. The initial segments generated on the terminal side and the subsequent segments generated on the cloud side share the same chronological index structure. The cloud does not need to reconstruct an unrelated index; it only needs to add path entries in chronological order, ensuring the logical integrity and continuity of streaming media fragments generated by asynchronous processing on both the terminal and cloud sides. This prevents timing gaps caused by switching processing sources when the playback terminal accesses the media in index order.
[0075] S104. Send the initial streaming media segment file, the initial streaming media index file, the structure description file, the cloud processing task set, and the remaining video data of the corresponding cloud processing task set in the source video to the cloud server.
[0076] The remaining video data refers to the original encoded data in the source video stream corresponding to the video segments covered by the cloud processing task set. Its extraction range is precisely defined by the start and end timestamps of each task entry in the cloud processing task set, and can be accompanied by byte offset annotations, so that the cloud can locate and read the target video segment without scanning the entire source video. The content sent in this step is divided into two categories in terms of functional attributes: the initial streaming media segment file and the initial streaming media index file are the processing results already produced in the local encoding stage of the mobile terminal, while the structure description file, the cloud processing task set, and the remaining video data are the context data that the cloud relies on for subsequent processing. Together, they constitute the complete input set for the cloud to seamlessly take over the transcoding task.
[0077] This step is triggered after the mobile terminal completes local initial encoding, generates initial streaming media segment files and initial streaming media index files, in order to minimize the time window for data waiting in the cloud. Specifically, the system uploads the above five types of data to the cloud server: the initial streaming media index file is sent synchronously with the initial segments. After receiving it, the cloud only needs to append the access paths of the subsequently generated segments to the file in the order of the source video timeline, without having to reconstruct an independent index, thus ensuring the continuity of streaming media segments generated by asynchronous processing between the end and the cloud on the logical timeline; the structure description file provides the cloud with segment complexity indicators and keyframe positions for each video segment, enabling it to independently complete the target encoding parameter configuration based on content characteristics, eliminating the redundant overhead of repeated analysis of the source video; the cloud processing task set clearly defines the range of video segments that need to be processed by the cloud, and together with the structure description file, constitutes the complete task execution specification for the cloud. The upload range of the remaining video data is strictly limited to the task boundaries of the cloud processing task set, avoiding the extremely high uplink bandwidth consumption caused by uploading a complete large video; during transmission, block upload and idempotent identification mechanisms can be adopted, combined with failure retransmission and breakpoint resumption strategies, to ensure the integrity and consistency of data transmission in weak network environments.
[0078] S201. Parse the cloud processing task set and structure description file to determine the target encoding parameters and slice duration corresponding to the remaining video data.
[0079] The target encoding parameters refer to the set of quantization configurations used by the cloud when performing compression encoding on the remaining video data. These parameters include at least the target bitrate (bits per second), target quantization parameters (QP value), target resolution, and target frame rate. The slice duration refers to the temporal granularity parameter used by the cloud during the slice encapsulation stage when dividing the encoded output stream into independent slices along the time dimension. Its value must be aligned with the keyframe position to ensure that the starting frame of each slice is an I-frame, thus enabling each slice to have independent decoding capabilities and supporting seamless switching at any slice entry point on the playback device. The cloud processing task set informs the cloud which video segments need to be processed, while the structure description file provides the content characteristics and structural information of these video segments. These two sets of data complement each other to completely determine the encoding parameters and slice duration.
[0080] This step is executed after receiving data sent by the mobile terminal in the cloud, and is the parameter preparation stage before the cloud starts compression encoding. Specifically, the system parses the cloud processing task set to obtain a list of video segments to be processed, and then extracts the segment complexity index and keyframe position of each corresponding video segment from the structure description file: the former is used to dynamically allocate differentiated encoding parameters to different video segments according to the content difficulty, so that high-complexity segments can obtain higher bitrate guarantees, and low-complexity segments can reduce resource consumption with lower bitrates; the latter is used to calculate the interval between adjacent I-frames to determine the slice duration, and prevent the slice boundary from not coinciding with the image group boundary, which would cause decoding dependency breakage.
[0081] In addition, specifications such as resolution, encoding format, and bitrate can be preset according to the terminal release scenario or target playback platform to meet the differentiated needs of different release channels for video specifications.
[0082] In some preferred embodiments, the above steps specifically include:
[0083] Extract the segmentation complexity index and keyframe position of each video segment corresponding to the remaining video data from the structure description file; match the segmentation complexity index with the preset complexity threshold range, assign the corresponding target quantization parameters and target bitrate to each video segment, and obtain the target encoding parameters; calculate the time interval between adjacent keyframes based on the keyframe position, set the slice duration to a preset integer multiple of the time interval, and ensure that the starting frame of each video slice generated by the slice encapsulation operation is a keyframe.
[0084] The preset complexity threshold range refers to a pre-configured segmentation rule that divides the continuous value space of the segmented complexity index into several discrete levels. Each level corresponds to a set of target quantization parameters and target bitrate. A typical implementation uses a three-level division (segmented complexity index normalized to the 0-1 range): when the segmented complexity index is below 0.35, it is classified as a low complexity level, with a target quantization parameter QP value of 32 to 36 and a target bitrate of 1Mbps to 2Mbps (based on 1080p / 30fps); when the segmented complexity index is between 0.35 and 0.70, it is classified as a medium complexity level, with a QP value of 24 to... 30. The target bitrate is 3Mbps to 5Mbps. When the segmentation complexity index is higher than 0.70, it is classified as a high complexity level, with a QP value of 18 to 23 and a target bitrate of 6Mbps to 10Mbps, in order to prioritize the output image quality of high dynamic or high texture scenes. The above threshold boundaries and bitrate ranges can be preset differently according to video category (such as life recording, sports, scene switching intensive), target resolution or historical sample statistics grouping. The number of levels can also be expanded to two or five levels to adapt to different precision requirements. The mapping between the target bitrate and quantization parameters can also be achieved by alternative methods such as lookup table rules or regression prediction models.
[0085] Preset integer multiples refer to setting the slice duration to an integer multiple (such as 1, 2 or 4 times) of the time interval between adjacent keyframes, so that the starting position of each slice strictly falls on the I-frame, ensuring that the slice can be decoded independently without referring to the preceding slice.
[0086] Specifically, the system extracts the segment complexity index of each video segment from the structure description file, matches them one by one with the preset complexity threshold range, and assigns the corresponding target quantization parameters and target bitrate according to the level: a smaller QP value and a higher bitrate are assigned to high-complexity video segments to prioritize the output image quality of high dynamic or high-texture scenes; a larger QP value and a lower bitrate are assigned to low-complexity video segments to reduce storage and bandwidth consumption without significantly reducing subjective quality, and guide encoding resources to tilt towards segments with higher content difficulty.
[0087] In addition to the threshold matching method mentioned above, the allocation of target bitrate and quantization parameters can also be achieved using alternative methods such as lookup table rules, regression prediction models, or templates pre-set according to device and network levels, to adapt to different accuracy requirements and computing power conditions. Regarding slice duration, the system calculates the slice duration as an integer multiple of the interval between adjacent I-frames based on the keyframe position, ensuring that the starting frame of each video slice is a keyframe. This allows each slice to be decoded independently without relying on preceding slices, providing a stable decoding entry point for adaptive bitrate switching and subsequent downgrading transcoding operations at the playback end. It also avoids decoding glitches or recovery delays caused by the slice starting position falling on a non-keyframe.
[0088] S202. Perform compression encoding and slice encapsulation operations on the remaining video data using the target encoding parameters and slice duration to generate subsequent streaming media segment files.
[0089] The subsequent streaming media segment files refer to a set of discrete media segments generated by the cloud after compressing and encapsulating the remaining video data using target encoding parameters and segment duration. The video content covered by these segments immediately follows the initial streaming media segment files on the source video timeline, and the two together cover the complete time span of the source video. The encapsulation format used by the subsequent streaming media segment files must be compatible with the segment format supported by the initial streaming media index file—such as both using fMP4 or MPEG-TS segments under the HLS protocol, or both using MP4 segments under the MPEG-DASH protocol—to ensure that the access path of the subsequent segments can be directly appended to the existing index file without the need for format conversion or index reconstruction.
[0090] This step is the substantive encoding execution phase after the cloud completes parameter configuration. Its output will be time-stitched with the initial streaming media index file to form the core content of the complete streaming media index. Specifically, the cloud sequentially calls the encoder according to the video segments specified in the cloud processing task set, compresses and encodes the remaining video data using the target encoding parameters, and encapsulates the segments according to their duration. Each segment file can be grouped into a designated storage directory using a unified naming rule with a unique video identifier as a prefix and a timeline offset as a suffix. Simultaneously, the time sequence identifier of each segment is recorded to facilitate subsequent extraction of paths according to the timeline and appending them to the index file. Subsequent streaming media segment files and the initial streaming media segment files logically belong to different time segments of the same source video. The unified index file connects the timeline, and the playback terminal is unaware of the switching of segment sources when accessing segments in the order of index entries. The end-cloud collaborative processing is completely transparent to the playback terminal.
[0091] In some preferred embodiments, after the above steps, the method further includes:
[0092] Monitor the real-time downlink bandwidth of the preset playback terminal when requesting source video through the streaming media playback link; when the real-time downlink bandwidth is lower than the target bitrate in the target encoding parameters for a preset period of time, calculate the downgrade bitrate based on the real-time downlink bandwidth;
[0093] The downgraded bitrate is used to transcode unaccessed segments in subsequent streaming media segments, generating downgraded streaming media segments; the access paths of the downgraded streaming media segments are updated in the complete streaming media index file so that the playback end can perform adaptive bitrate switching.
[0094] Among them, the real-time network downlink bandwidth refers to the currently available receiving bandwidth estimated by the playback end through continuous measurement of the segment download rate during the process of requesting segments through the streaming media playback link; the downgrade bitrate refers to the target output bitrate that can match the current network carrying capacity, calculated based on the real-time network downlink bandwidth. It can be obtained by multiplying the real-time downlink bandwidth by a preset safety factor (e.g., 0.8) to reserve a margin when there are fluctuation errors in bandwidth estimation, and to prevent the download rate of downgraded segments from still exceeding the network carrying capacity and causing secondary buffering.
[0095] The preset time period refers to the shortest confirmation time required to determine if bandwidth is continuously insufficient. By setting a duration threshold, false triggers caused by brief bandwidth fluctuations can be filtered out. Its value can be pre-configured according to the player buffer capacity and user experience requirements, with a typical range of 3 to 5 seconds: When the player buffer capacity is large (e.g., more than 15 seconds), the preset time period can be set to 5 seconds to reduce unnecessary downgrade transcoding caused by brief network fluctuations; when the player buffer capacity is small (e.g., 5 to 8 seconds), the preset time period can be shortened to 3 seconds to complete the generation of the downgraded version in time before the buffer is exhausted, reducing the risk of stuttering; the bandwidth detection sampling interval can be set to 500 milliseconds to 1 second, with the trigger judgment based on the continuous sampling points being lower than the target bitrate; the downgraded streaming media segment file refers to the low bitrate segment set generated by re-transcoding segments that have not yet been accessed by the playback end using the downgraded bitrate.
[0096] This step serves as a proactive adaptive expansion mechanism after subsequent streaming media segment files are generated in the cloud. It transforms the strategy for dealing with network fluctuations from passive buffering and waiting to proactive and forward-looking transcoding. Specifically, when the real-time network downlink bandwidth is lower than the target bitrate for a preset period of time, the system calculates the downgraded bitrate by multiplying the real-time downlink bandwidth by a safety factor, and re-transcodes the segments in the subsequent streaming media segment files that have not yet been accessed by the playback client. This step only downgrades unaccessed segments because accessed segments have already been buffered or rendered by the playback client, and re-transcoding them is neither meaningful nor computationally efficient. The unaccessed segments are precisely the content that the playback client will request soon. Generating downgraded versions in advance can provide the playback client with a switchable low-bitrate path when bandwidth is continuously limited, avoiding playback stuttering caused by download speeds exceeding network capacity.
[0097] After the access path of the downgraded segment is updated to the complete streaming media index file, the playback end completes the adaptive bitrate switching accordingly. When the bandwidth recovers, the bitrate can be prevented from switching frequently by setting the back-cut hysteresis time (i.e., the back-cut is triggered only after the bandwidth has recovered steadily and exceeded the high bitrate threshold for a period of time). This maintains the stability of the playback experience.
[0098] S203. Extract the access paths of subsequent streaming media segment files, and concatenate the access paths into the initial streaming media index file according to the timeline order of the source video to generate a complete streaming media index file.
[0099] The access path refers to the addressable identifier of each subsequent streaming media segment file in the storage service or content delivery network. It can be represented as a URL in HTTP / HTTPS format, a key value of the object storage service, or a relative access path, etc. The playback terminal uses this path to initiate segment data requests to the distribution node. The complete streaming media index file refers to the media index that summarizes and records the access paths of the initial streaming media segment file and subsequent streaming media segment files in the order of the source video timeline. It can adopt the M3U8 format of the HLS protocol or the MPD format of the MPEG-DASH protocol. Each segment entry must be sorted according to the time offset, and a unique path identifier is used to prevent duplicate writing. The update method is to append new entries sequentially along the timeline, rather than performing a full rewrite of the entire index file, so as to minimize the index update cost.
[0100] This step, performed after the subsequent streaming media segment files are generated, is crucial for integrating the processing results from both the cloud and endpoints and building a complete, addressable playback index. Specifically, the cloud traverses the subsequent streaming media segment files in chronological order from the storage directory, extracting the access paths for each segment. These paths are then sorted in ascending order using the segment's start timestamp as the key and appended one by one to the initial streaming media index file, completing the construction of the complete streaming media index file. The appending operation is strictly performed in chronological order to ensure that the segment entries in the index file completely correspond to the source video's timeline. This prevents playback from experiencing screen jumps or timing errors when the playback endpoint requests segments in the order of entries due to discrepancies between the path order and the actual playback sequence. This step incrementally appends to the initial streaming media index file, avoiding the additional overhead of a full rebuild. It also allows the index file to continuously evolve as the segment generation progresses—adding a new batch of access paths after each batch of subsequent segments is completed, providing the playback endpoint with the possibility of progressive access.
[0101] If it is necessary to adapt to the capabilities of multiple playback terminals, multiple index versions can be derived on demand based on the same batch of access paths, targeting different device capabilities, for high-definition playback scenarios and low-power devices to select as needed.
[0102] S105. During the compression encoding and slicing encapsulation operation, if the hardware operating status parameters are detected to exceed the preset hardware threshold, the remaining unexecuted tasks in the terminal processing task set and the unprocessed video data corresponding to the remaining tasks are sent to the cloud server.
[0103] Among them, the hardware threshold refers to the upper limit parameter of the monitoring used by the mobile terminal to determine the critical state of resources and trigger task migration during local encoding execution; the unexecuted remaining tasks refer to all task entries that the terminal processing tasks are concentrated on at the time of triggering migration but have not yet started compression encoding and slicing encapsulation processing. It is determined by comparing the initial list of task entries with the execution records of completed tasks. The migration range is marked by the start timestamp or task sequence number of the task entry to ensure that completed tasks are not processed repeatedly.
[0104] This step serves as a dynamic fallback mechanism for the edge-cloud collaboration strategy. It continuously monitors the hardware operating status throughout the local encoding process on the mobile terminal. Once a high-load or resource-critical state is detected, task migration is triggered to protect the terminal from the impact of continuous intensive encoding. Specifically, the system samples parameters such as CPU utilization, device temperature, and remaining battery power after each task entry is completed or at a fixed sampling period. When any parameter exceeds the corresponding preset hardware threshold, the system freezes any remaining tasks that have not yet been started, extracts their corresponding unprocessed video data, and sends the remaining task entries and unprocessed video data to the cloud server, using the I-frame position aligned with the keyframe as the migration cutting boundary. After the migration is triggered, the terminal synchronously updates its local task status and removes the migrated task entries from the pending queue to prevent the same video segment from being encoded repeatedly due to inconsistent status.
[0105] By using keyframe boundaries as migration and segmentation points instead of arbitrary byte positions, it is ensured that the video segments at the migration and segmentation points do not experience decoding dependency breaks, eliminating the need for additional frame patching or resetting of reference frame states during cloud-based continuation processing. This mechanism combines static pre-planning of tasks with runtime elastic unloading, enabling dynamic borrowing of cloud computing power in scenarios with unpredictable hardware state changes, thus avoiding device overheating and rapid power consumption issues caused by prolonged high-load operation of the terminal.
[0106] S204. Receive the remaining tasks and the unprocessed video data corresponding to the remaining tasks sent by the mobile terminal, perform compression encoding and slice encapsulation operations on the unprocessed video data using the target encoding parameters and slice duration, generate new streaming media segment files, and supplement the access path of the new streaming media segment files to the complete streaming media index file.
[0107] Among them, the newly added streaming media segment file refers to a set of streaming media segments generated by the cloud after performing compression encoding and segmentation encapsulation on the unprocessed video data received from the mobile terminal. The corresponding video segments belong to the time range defined by the initial cloud processing task set on the source video timeline. It is a direct continuation of the processing result of the terminal dynamic offloading task in S108. It belongs to the segment output of cloud processing along with the subsequent streaming media segment files, rather than a separate processing branch. The newly added streaming media segment file must adopt the same naming rules and path structure as the subsequent streaming media segment files, and use the timeline offset as an identifier to achieve unified sorting. This ensures that after the access path of the newly added segment is appended to the complete streaming media index file, all segment entries still strictly maintain the timeline order.
[0108] This step is the cloud's follow-up response after the mobile terminal triggers dynamic task migration, forming a complete closed loop of the end-to-cloud collaboration link with S108. Specifically, after receiving the remaining task items and unprocessed video data sent by the mobile terminal, the cloud first verifies whether the newly received task range already has path placeholders in the complete streaming media index file, and re-verifies whether the previously determined target encoding parameters are still applicable based on the received remaining tasks—if the video content characteristics or network status have changed when the terminal triggers the migration, the encoding parameters can be locally modified at this stage. Then, using the same target encoding parameters and slice duration as the initial cloud-processed task set, compression encoding and slice encapsulation are performed on the unprocessed video data to generate new streaming media segment files; the access paths of the new segments are extracted, sorted by time axis offset, and added to the complete streaming media index file.
[0109] Since the complete streaming media index file always manages all segment entries based solely on the source video timeline, the addition of new segment paths does not introduce additional sorting adjustment logic. When the playback end initiates segment requests in the order of index entries, segments generated from different sources and at different times on both the end and cloud are completely transparent to the playback end. Access to segments along the entire timeline will not cause playback interruptions due to mid-process task migration, thus ensuring the temporal continuity and content integrity of streaming media.
[0110] S205. Generate a streaming media playback link based on the complete streaming media index file, and send the streaming media playback link to the mobile terminal so that the mobile terminal can share or publish the source video.
[0111] The streaming media playback link refers to an accessible identifier generated with the storage location of the complete streaming media index file as the address target. Essentially, it's an HTTP / HTTPS URL pointing to a complete streaming media index file in M3U8 or MPD format. The playback client parses the index file pointed to by this link, sequentially obtains the access paths for each segment, and sends requests to the storage node or content delivery network to complete the streaming playback of the video. This link can be generated with authentication parameters (such as HMAC signature tokens, expiration timestamps), version numbers, or path signatures to prevent unauthorized access after the link expires. Furthermore, according to the format requirements of the publishing channel, the standard URL can be shortened into a shorter link to adapt to the link length requirements of different scenarios such as instant messaging, QR code sharing, or platform publishing interfaces.
[0112] Specifically, after storing the complete streaming media index file in object storage or a content delivery network, the cloud extracts its storage access address and appends a pre-computed authentication signature and validity period field to the address to generate a playback link carrying security parameters; then the link is sent to the mobile terminal via server push or response callback.
[0113] Once the complete streaming media index file is generated, an accessible playback link can be created immediately, eliminating the need for mobile devices to maintain a high-load local encoding state. This completely decouples the release timing from the terminal resource state, avoiding the sequential blocking caused by "the terminal must complete full processing before release," and enabling video shareability to be achieved instantly upon completion of end-cloud collaborative processing. In terms of multi-channel distribution, the complete URL can be used directly for webpage embedding or in-app playback scenarios, while the link length can be compressed through shortening for social messaging or QR code sharing scenarios. This allows the same complete streaming media index file to reach the audience with a link format most suitable for the target channel characteristics, improving the applicability and flexibility of cross-channel streaming media content release.
[0114] S106. Receive the streaming media playback link returned by the cloud server.
[0115] Specifically, after successfully receiving the playback link, the mobile terminal can immediately trigger a status notification (such as an in-app pop-up or system notification push) to inform the user that the video has been processed and is ready to be shared. If the link reception fails (such as due to network interruption causing response timeout or data loss), the terminal can initiate a new request to the cloud server through a preset retrieval mechanism, carrying an idempotent flag to prevent the server from generating indexes repeatedly or causing data inconsistencies. After the link reception is successful, a publishing result receipt can also be generated and reported to the business log module for subsequent tracking of the complete link status of this processing.
[0116] Thus, the entire solution forms a complete closed loop: the mobile terminal first completes the encoding of the first-broadcast priority range and the terminal processing task set locally, generating initial streaming media segment files and index files, and building a publishable foundation; the cloud continues to complete the transcoding of the remaining video data and dynamic offloading tasks, splices them together in chronological order to build a complete streaming media index file, and proactively performs forward-looking downgrade transcoding on unaccessed segments in weak network conditions; the playback terminal finally accesses the complete streaming media in the order of index entries through the playback link, obtaining a smooth and continuous playback experience. The collaborative processing between the terminal and the cloud is transparent to the playback terminal throughout the entire process, and the efficiency, completeness and continuity of the entire process are guaranteed.
[0117] The following specific numerical example will provide a complete explanation of the data flow and parameter configuration process in each step of the above method.
[0118] The basic parameters are set as follows: the source video file size is 1GB (1024MB), the resolution is 1080P, the frame rate is 30fps, and the total duration is 300 seconds; the current CPU utilization rate of the mobile terminal is 80%, the remaining battery is 60%, and the available uplink bandwidth is 8Mbps; the preset first playback buffer time is 5 seconds, the CPU utilization hardware threshold is 90%, the preset battery threshold is 20%, the preset rate threshold is 2Mbps, and the base unit for slice duration is 10 seconds.
[0119] Step S101: The system performs a lightweight pre-decoding scan on the source video, detecting I-frames at 0 seconds, 10 seconds, 20 seconds...290 seconds, and 300 seconds, totaling 31 I-frame boundaries. The source video is then divided into 30 video segments (s_1 to s_30), each 10 seconds long. The spatial texture feature value and temporal motion vector feature value of each segment are calculated and weighted (w_s=0.4, w_t=0.6) to obtain the segmentation complexity index for each segment. Among these, s_1 to s_... Segment 3 represents low complexity (with index values of approximately 0.28, 0.32, and 0.35 respectively), s_4 to s_7 represent medium complexity (with index values of approximately 0.51 to 0.62), and s_8 to s_10 represent high complexity (with index values of approximately 0.74, 0.79, and 0.82, corresponding to a motion scene). The remaining segments represent low to medium complexity. The keyframe positions, segment complexity indices, and timestamps of each segment are encapsulated in JSON format to generate a structure description file (approximately 12KB in size).
[0120] Step S102: The system constructs a 54-dimensional multidimensional state vector: where f_size_norm=1024 / 2048=0.50, u_cpu=0.80, u_bat=0.60, u_bw_norm=8 / 10=0.80. The segmentation complexity values of the 30 video segments are filled into dimensions 5 to 34 in sequence, and dimensions 35 to 54 are padded with zeros (because N=30 < N_max=50). This 54-dimensional vector is input into the deployed policy network (54 → FC(256, ReLU) → FC(128, ReLU) → FC(1, Sigmoid)). The forward inference takes about 18 milliseconds, and the policy network outputs an initial task allocation ratio α=0.27, corresponding to a terminal processing time of approximately 0.27 × 30 ≈ Eight video segments; then the system uses round(0.27×30)=8 as the center and Δ=3 as the search radius to form a candidate segmentation point set J_set={5, 6, 7, 8, 9, 10, 11}, and calculates the joint cost L(j) for each candidate segmentation point j; when j=7, the terminal processes s_1 to s_7, which are low to medium complexity segments with low energy consumption, and the high complexity s_8 to s_10 can be completely handled by the cloud; when j=8, the terminal needs to take on the high complexity segment s_8, and the terminal energy consumption increases significantly, L(8) > L(7); After traversal comparison, L(j*) reaches its minimum value when j*=7 (corresponding to 70 seconds on the time axis), and the optimal split point is determined to be 70 seconds; Since the duration of s_1 is 10 seconds, which meets the preset first broadcast buffer duration of 5 seconds, the first broadcast priority interval is set to [0, 10s] (i.e. s_1); The final output end-cloud collaboration strategy is: first broadcast priority interval [0, 10s], terminal processing task set {s_1, s_2, ..., s_7} (covering 0 seconds to 70 seconds), and cloud processing task set {s_8, s_9, ..., s_30} (covering 70 seconds to 300 seconds).
[0121] Step S103: The terminal arranges the encoding queue according to the "first broadcast priority" principle, performs compression encoding and segmentation encapsulation on s_1 (segmentation complexity 0.28, low complexity) with a target bitrate of 1.5Mbps, and generates the first initial streaming media segment file (size approximately 1.875MB = 1.5Mbps × 10s / 8); after s_1 is encoded, the terminal continues to complete the encoding of the remaining terminal processing task sets in the order of s_2, s_3...s_7. Each segment selects a target bitrate based on its segmentation complexity index (s_4 to s_7 are medium complexity, with a target bitrate of approximately 3Mbps to 4Mbps); after encoding is completed, the access paths of each initial segment are extracted and written into the initial streaming media index file in M3U8 format in timeline order.
[0122] Step S104: The terminal sends the above 5 types of data to the cloud server: the initial streaming media segment file (the 3 completed segments s_1 to s_3, with a total of about 10MB of encoded output), the initial streaming media index file, the structure description file (12KB), the cloud processing task set (task entries corresponding to {s_8, ..., s_30}) and the raw video data of s_8 to s_30 (23 / 30 × 1,024MB ≈ 785MB); with an uplink bandwidth of 8Mbps, the transmission of the 785MB of raw video data proceeds in parallel with the subsequent encoding on the terminal, without waiting for the upload to be completed before starting cloud processing.
[0123] Steps S201 to S202: The cloud parses the structure description file and extracts the segment complexity index of each segment from s_8 to s_30; among them, s_8 to s_10 (complexity 0.74 to 0.82) are classified as high complexity, and are assigned a target quantization parameter QP=20 and a target bitrate of 8Mbps; the segments from s_11 to s_30 are mostly medium to low complexity, and the target bitrate is configured between 2Mbps and 4Mbps; the slice duration is set to 1 times the 10-second interval between adjacent I-frames (i.e., slice every 10 seconds), ensuring that the starting frame of each slice is an I-frame; the cloud uses the above parameters to perform compression encoding and slice encapsulation on the received s_8 to s_30 sequentially to generate subsequent streaming media segment files.
[0124] Step S105: After completing the encoding of s_1 to s_3, the terminal immediately begins encoding s_4. Due to the increased complexity of s_4 and subsequent segments (0.51 to 0.62), the CPU load rises to 93% after approximately 3.5 seconds, exceeding the preset hardware threshold of 90%. The system determines that the tasks currently still in the terminal processing task set that have not yet been executed are {s_4, s_5, s_6, s_7} (corresponding to a time range of 30 to 70 seconds). It immediately extracts the original video data of s_4 to s_7 (4 / 30 × 1, 024 MB ≈ 136 MB) with I-frames as the boundary, and sends it to the cloud server along with the task entries {s_4, s_5, s_6, s_7} through an independent uplink queue. The terminal marks the above 4 task entries as "migrated" and removes them from the local pending queue to prevent duplicate encoding. At this point, the video segments that the terminal has actually completed encoding locally are s_1 to s_3 (0 to 30 seconds), and the remaining tasks have been taken over by the cloud.
[0125] Steps S204 to S205: After receiving the task unloading of s_4 to s_7, the cloud verifies that there are no path placeholders in this range in the complete streaming media index file. It performs compression encoding and segment encapsulation on s_4 to s_7 with the same target encoding parameters and segment duration as s_8 to s_30, generating new streaming media segment files. It extracts the access paths of the new segments, sorts them by time axis offset, and adds them to the complete streaming media index file. At this time, the complete streaming media index file covers the access entries of 30 segments from s_1 to s_30, covering the complete time axis from 0 seconds to 300 seconds of the source video. Based on this, the cloud generates an M3U8 playback link with HMAC authentication signature and sends the link to the mobile terminal through server callback.
[0126] Step S106: After receiving the playback link, the mobile terminal triggers an in-app notification to inform the user that the video has been processed, and the user can then share the link to the target platform. The playback terminal parses the M3U8 index file and requests each segment in chronological order—segments s_1 to s_3 are generated by local encoding on the terminal, while segments s_4 to s_30 are generated by cloud encoding. The two types of segments are connected along the timeline through a unified M3U8 index file, and the playback terminal is unaware of the switching of segment sources, resulting in a seamless and uninterrupted streaming media playback. In the above example, the actual amount of video encoded locally by the terminal is reduced from the originally planned 70 seconds to 30 seconds. The dynamic task migration mechanism minimizes the CPU high-load period, protecting the mobile terminal's operating status and remaining battery power.
[0127] The following describes the collaborative transcoding device for large video on mobile terminals according to the embodiments of this invention from the perspective of hardware processing. Please refer to [link to relevant documentation]. Figure 2 This is a schematic diagram of a physical device structure for a collaborative transcoding device for large video on a mobile terminal, as described in this application.
[0128] It should be noted that, Figure 2 The structure of the collaborative transcoding device for large video on mobile terminals shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.
[0129] like Figure 2As shown, the collaborative transcoding device for large video on mobile terminals includes a Central Processing Unit (CPU) 201, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 202 or programs loaded from storage section 208 into Random Access Memory (RAM) 203, such as performing the methods described in the above embodiments. The RAM 203 also stores various programs and data required for system operation. The CPU 201, ROM 202, and RAM 203 are interconnected via a bus 204. An Input / Output (I / O) interface 205 is also connected to the bus 204.
[0130] The following components are connected to I / O interface 205: input section 206 including audio input devices, push-button switches, etc.; output section 207 including a liquid crystal display (LCD) and audio output devices, indicator lights, etc.; storage section 208 including a hard disk, etc.; and communication section 209 including a network interface card such as a LAN (Local Area Network) card, modem, etc. Communication section 209 performs communication processing via a network such as the Internet. Drive 210 is also connected to I / O interface 205 as needed. Removable media 211, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 210 as needed so that computer programs read from them can be installed into storage section 208 as needed.
[0131] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 209, and / or installed from removable medium 211. When the computer program is executed by central processing unit (CPU) 201, it performs the various functions defined in the present invention.
[0132] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0133] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those shown in the drawings.
[0134] Specifically, the collaborative transcoding device for large video on mobile terminals in this embodiment includes a processor and a memory. The memory stores a computer program, and when the computer program is executed by the processor, it implements the collaborative transcoding method for large video on mobile terminals provided in the above embodiment.
[0135] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in the collaborative transcoding device for large video on mobile terminals described in the above embodiments; or it may exist independently and not assembled into the collaborative transcoding device for large video on mobile terminals. The storage medium carries one or more computer programs that, when executed by a processor of the collaborative transcoding device for large video on mobile terminals, cause the collaborative transcoding device to implement the collaborative transcoding method for large video on mobile terminals provided in the above embodiments.
[0136] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.
[0137] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".
[0138] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.
Claims
1. A collaborative transcoding method for large videos on mobile terminals, applied to mobile terminals, characterized in that, The method includes: Extract the keyframe positions and segmentation complexity indices of the source video to be shared, and generate a structural description file of the source video; The file size of the source video, the hardware operating status parameters and network transmission rate parameters of the mobile terminal, and the structure description file are input into a preset collaborative processing model, and the end-cloud collaborative strategy is output. The end-cloud collaborative strategy includes a first-play priority interval, a terminal processing task set and a cloud processing task set. Extract the video data corresponding to the first-broadcast priority interval and the terminal processing task set, perform compression encoding and slicing encapsulation operations on the extracted video data, and generate initial streaming media segment files and initial streaming media index files; The initial streaming media segment file, the initial streaming media index file, the structure description file, the cloud processing task set, and the remaining video data in the source video corresponding to the cloud processing task set are sent to the cloud server. During the compression encoding and slicing encapsulation operation, if the hardware operating status parameters are detected to exceed a preset hardware threshold, the remaining unexecuted tasks in the terminal processing task set and the unprocessed video data corresponding to the remaining tasks are sent to the cloud server. The system receives a streaming media playback link returned by the cloud server, which is used to instruct a preset playback terminal to play the source video.
2. The method according to claim 1, characterized in that, The process of inputting the file size of the source video, the hardware operating status parameters and network transmission rate parameters of the mobile terminal, and the structure description file into a preset collaborative processing model, and outputting an end-cloud collaborative strategy, specifically includes: The file size of the source video, the CPU utilization and remaining power in the hardware operating status parameters, the network transmission rate parameters, and the segmentation complexity index in the structure description file are concatenated to construct a multi-dimensional state vector. The multidimensional state vector is input into a pre-trained deep reinforcement learning network, and the initial task allocation ratio is output through the policy network in the deep reinforcement learning network. Based on the initial task allocation ratio and the segmentation complexity index, a preset planning algorithm is used to search for the optimal segmentation point on the timeline of the source video. The optimization objective is to minimize the energy consumption of the mobile terminal and the overall processing latency. The end-to-cloud collaboration strategy is generated, and the first-play priority interval is the video segment before the optimal segmentation point that meets the preset first-play buffer duration.
3. The method according to claim 2, characterized in that, The optimization objective of generating the end-to-cloud collaboration strategy, which aims to minimize the energy consumption and overall processing latency of the mobile terminal, specifically includes: A joint optimization objective function is constructed, which includes a terminal energy consumption penalty term, an overall processing delay penalty term, and a first-broadcast delay penalty term. The weighting coefficient of the terminal energy consumption penalty item is adjusted according to the remaining power. When the remaining power is lower than the preset power threshold, the weighting coefficient of the terminal energy consumption penalty item is adjusted to the first preset weight. The weight coefficient of the overall processing delay penalty item is adjusted according to the network transmission rate parameter. When the network transmission rate parameter is lower than the preset rate threshold, the cloud processing task allocation ratio in the initial task allocation ratio is adjusted to the preset ratio, and the weight coefficient of the overall processing delay penalty item is adjusted to the second preset weight. The initial task allocation ratio is substituted into the joint optimization objective function for iterative solution. When the joint optimization objective function converges, the task allocation result obtained is used as the terminal processing task set and the cloud processing task set.
4. The method according to claim 1, characterized in that, The step of extracting the keyframe positions and segmentation complexity indices of the source video to be shared, and generating a structural description file for the source video, specifically includes: The source video is pre-decoded and analyzed to extract the position information of all I-frames in the source video as the keyframe positions; Calculate the spatial texture feature value and temporal motion vector feature value of video segments between adjacent I-frames in the source video; The spatial texture feature value and the temporal motion vector feature value are weighted and fused to obtain the segmentation complexity index of the video segment; The keyframe positions, segmentation complexity indicators, and timestamps corresponding to the video segments are encapsulated according to a preset data exchange format to generate the structure description file.
5. A collaborative transcoding method for large videos on mobile terminals, applied to cloud servers, characterized in that... The method includes: Receive the initial streaming media segment file, initial streaming media index file, source video structure description file, cloud processing task set, and remaining video data in the source video sent by the mobile terminal; The cloud processing task set and the structure description file are analyzed to determine the target encoding parameters and slice duration corresponding to the remaining video data; The remaining video data is compressed and sliced using the target encoding parameters and the slice duration to generate subsequent streaming media segment files. Extract the access paths of the subsequent streaming media segment files, and concatenate the access paths into the initial streaming media index file according to the timeline order of the source video to generate a complete streaming media index file; The system receives the remaining tasks and the unprocessed video data corresponding to the remaining tasks sent by the mobile terminal, performs compression encoding and segmentation encapsulation operations on the unprocessed video data using the target encoding parameters and the segment duration, generates a new streaming media segment file, and adds the access path of the new streaming media segment file to the complete streaming media index file. A streaming media playback link is generated based on the complete streaming media index file, and the streaming media playback link is sent to the mobile terminal so that the mobile terminal can share or publish the source video.
6. The method according to claim 5, characterized in that, The step of parsing the cloud processing task set and the structure description file to determine the target encoding parameters and slice duration corresponding to the remaining video data specifically includes: Extract the segmentation complexity index and keyframe position of each video segment corresponding to the remaining video data from the structure description file; The segmentation complexity index is matched with a preset complexity threshold range, and corresponding target quantization parameters and target bitrates are assigned to each video segment to obtain the target encoding parameters. The time interval between adjacent keyframes is calculated based on the keyframe position, the slice duration is set to a preset integer multiple of the time interval, and it is ensured that the starting frame of each video slice generated by the slice encapsulation operation is the keyframe.
7. The method according to claim 6, characterized in that, After the step of performing compression encoding and segmentation encapsulation operations on the remaining video data using the target encoding parameters and the segment duration to generate subsequent streaming media segment files, the method further includes: Monitor the real-time downlink network bandwidth when a preset playback terminal requests the source video through the streaming media playback link; When the real-time network downlink bandwidth is lower than the target bitrate in the target coding parameters for a continuous preset time period, the downgraded bitrate is calculated based on the real-time network downlink bandwidth. The downgraded bitrate is used to transcode the unaccessed segments in the subsequent streaming media segment files to generate downgraded streaming media segment files; The access path of the downgraded streaming media segment file is updated to the complete streaming media index file so that the playback terminal can perform adaptive bitrate switching.
8. A collaborative transcoding device for large video on mobile terminals, characterized in that, The collaborative transcoding device for large video on mobile terminals includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the collaborative transcoding device for large video on mobile terminals to perform the method as described in any one of claims 1-7.
9. A computer-readable storage medium comprising instructions, characterized in that, When the instruction is executed on a collaborative transcoding device for large video on mobile terminals, the collaborative transcoding device for large video on mobile terminals performs the method as described in any one of claims 1-7.
10. A computer program product, characterized in that, When the computer program product is run on a collaborative transcoding device for large video on mobile terminals, the collaborative transcoding device for large video on mobile terminals performs the method as described in any one of claims 1-7.
